Those who want to work for a world-changing startup from Japan?

Starting your career as a new graduate is one of the most important life events for many people.

Over the past 30 years, countries around the world have accelerated their economic ties, leading to globalization.

Meanwhile, politically, the world has changed from the relatively stable international situation of the latter half of the 20th century to the VUCA (Volatility, Uncertainty, Complexity, and Ambiguity) era, due to the confrontation between liberal capitalism and former communist countries and the increasing presence of the Global South.

There’s no denying that we have already ushered in the era of big data. An enormous amount of information is generated every second. While decision-makers can gain invaluable insights from this ever-growing data, its sheer volume also poses considerable challenges to data engineers–greater demand for storage spaces, the need to handle increasingly complex data formats, and highly unpredictable network traffic. Luckily, recent years have witnessed the creation of various technologies devoted to efficiently digesting big data, and Kafka is one of them. Today, I will demonstrate how Kafka works to help kick-start your Kafka journey.

If you read a lot about innovation and technology, you’ve probably heard or seen some publication involving the acronym SXSW.

The acronym means South by Southwest and it is an innovation festival. We can say that it is one of the biggest events of our time involving people from all over the world who debate themes that involve innovation, media, music, behavior and interactivity. The purpose of the event is to expose and provide space for people in the technology area to share their ideas and experiences. In this post we are going to talk in more detail about SXSW.

As a data engineer, you probably have heard about Hadoop. It is one of the most popular frameworks for distributed processing of large data sets. It is less costly and more secure than other frameworks. At its center is a programming model called MapReduce. Today we will take a closer look at MapReduce to understand the inner working of Hadoop.

As a data engineer, you probably have heard of the data integration methodology called ETL (Extract-Transform-Load). It has been around for a while, and many data engineers have used this methodology to build data pipelines. However, ETL is not the only option up our sleeves. Recently, ELT has also been gaining a lot of popularity. In this article, I will compare ETL and ELT to help you understand their respective advantages and drawbacks so that you can choose the methodology more suitable for your project.

Hi, I’m Alessandro.

The Colorkrew Biz team recently went on a company camp at the Enoshima Beach House to make up for the disruptions caused by COVID-19. While the team had done well with remote work, the employer knew that in-person interactions and team bonding were important.

We spent two days in a dreamy Beach House in the stunning Enoshima, partying and enjoying yourselves with our teammates. But that’s not all - in between all the fun, we also had the opportunity to tackle interesting technical challenges and come up with improvements for our product. It was the perfect blend of work and play!

Few would disagree that the word “lazy” has a negative connotation. We usually describe someone who is not willing to work hard as lazy. However, not all laziness is undesirable, and sometimes we prefer laziness to diligence, especially in the computer science world. One example is lazy evaluation. Today, we will closely inspect why lazy evaluation is essential to Spark’s high performance and how lazy evaluation works in Spark.

If you have been in the world of big data long enough, you probably have heard about Parquet files. You might even have used it while thinking to yourself: “why can’t we just use CSV files?” Today, I will debunk the mystery of Parquet files and explain why a growing number of data scientists prefer Parquet files to CSV files.

Hi! My name is Ziyu Chen. I am a full-stack engineer at Colorkrew. I love learning and writing about data engineering.

Today, I would like to discuss transformations and actions in Spark. Of course, you can dive into the world of Spark and perform ETL processes without knowing how they differ from each other. However, once you are in charge of optimizing pre-existing ETL processes or building new ones from scratch, not understanding their differences might sabotage the efficiency of the ETL processes that you create. In the end, mediocrely-designed inefficient ETL processes might even increase the operation cost of your organization. Therefore, a thorough comprehension of the essence of transformations and actions becomes necessary if you want to build high-performance ETL processes using Spark.

Dependency Injection - With Laravel

Dependency injection is a commonly used design pattern in object oriented programming. Through some pre-established conventions, we are able to manage the creation of our dependencies more easily. We can declare, replace, or even mock the dependencies as needed without the need to change the code that relies on the dependency.

Any frontend developer knows the frustration of seeing the infamous “is not a function” message pop up. It reminds you that, even though your code might achieve your goal, you will not be able to assure its consistency or reliability. Without strict typing there is no stopping tiny slip-ups due to a lack of coffee - or sleep, depending on your dedication.

Hi all, This is Ning Li from Colorkrew. I am a lead engineer who oversees several in-house service developments in Colorkrew. In this blog, please let me introduce the technical onboarding in Colorkrew.

We have been working hard to improve this progress in recent years, so that our new members can join the team smoothly and get excellent work experience. We are considering 3 points.

Hello! I’m Yusuke, the HR guy with the dreadlocks. On December 1, 2021, we launch “Culture Deck.”

➡https://www.colorkrew.com/about/culture_deck/

In this blog, I’d want to explain why I came up with the concept of producing a Culture Deck in the first place.

■What exactly is a Culture Deck?

A Culture Deck is a document that describes and summarizes a company’s corporate philosophy, management concepts, and anticipated employee conduct in a thorough manner. More and more firms are developing similar products, although Netflix is perhaps the most well-known. Netflix’s Culture Deck has been called “the most important document ever to come out of the Valley”.

On December 1, 2021, We updated our vision.

「 Color Your Work with Excitement 」

We wish to make working people’s daily life more exciting. We wish to provide color and variety to people’s lives. Our new vision was created with this in mind.

The Vision Project Again

This is the third renewal of our vision since I came to work at Colorkrew in 2010.