Hi! My name is Ziyu Chen. I am a full-stack engineer at Colorkrew. I love learning and writing about data engineering.

Today, I would like to discuss transformations and actions in Spark. Of course, you can dive into the world of Spark and perform ETL processes without knowing how they differ from each other. However, once you are in charge of optimizing pre-existing ETL processes or building new ones from scratch, not understanding their differences might sabotage the efficiency of the ETL processes that you create. In the end, mediocrely-designed inefficient ETL processes might even increase the operation cost of your organization. Therefore, a thorough comprehension of the essence of transformations and actions becomes necessary if you want to build high-performance ETL processes using Spark.

Dependency Injection - With Laravel

Dependency injection is a commonly used design pattern in object oriented programming. Through some pre-established conventions, we are able to manage the creation of our dependencies more easily. We can declare, replace, or even mock the dependencies as needed without the need to change the code that relies on the dependency.

Any frontend developer knows the frustration of seeing the infamous “is not a function” message pop up. It reminds you that, even though your code might achieve your goal, you will not be able to assure its consistency or reliability. Without strict typing there is no stopping tiny slip-ups due to a lack of coffee - or sleep, depending on your dedication.

Hello! I’m Da-hara from the branding project.

I’m delivering you the latest interview article first time in a while!

This month, one big project came to its completion. The project was rewarded in the Award presentation, where the company praises successful outcomes of the previous period.

We had an interview with Mr. Tagami, the No.1 Project Manager (PM) called “the man who never fails.” We had him talk about the history of the 20 years long project that had been contributing to the company, what the team learned through, and the hardships and delights of the members.