Colorkrew Blog

What data structure would you use to determine whether an entity is in a given dataset with speed? Many would answer “Hashtable” without hesitation. Hashtable performs well when accessing an entity among a group of entities: the average time complexity of its read operations is constant. Nevertheless, using Hashtables can be rather costly when dealing with large datasets. Hashtables need to keep track of every entity in the dataset, resulting in a linear space complexity.

For many software engineers, endpoints for retrieving a list of entities are probably their favorite to implement–all they need to do is build an SQL statement that grabs all the available items, along with some other minor tasks. It’s all fun and games until the number of entities in the database becomes a bottleneck, where each GET request returns tens of thousands of items. You start to see server-side logs bombarded with error messages highlighted in red. Your dear users, previously satisfied with your application, begin to complain about the increasingly lackluster performance. Your product manager is on the brink of losing it and taking out their frustration on you. What?! You say you want to prevent all this from happening? Pagination is what you need! It can drastically improve your endpoints’ performance and save your servers from overload. Today, I will define pagination and introduce the most common pagination techniques.

Recently, I joined a new project at Colorkrew that uses Golang as its backend language. Almost everyone on the team, including myself, had no experience with Golang, so we had to start from zero. Learning Golang has been a rewarding experience so far. After all, it is one of the most in demand programming languages now. However, writing code in Golang is not all sunshine and roses. Unopinionated as I am, I still have my fair share of complaints, the biggest being the recommended way of error handling in Golang.

Many frontend engineers probably have experienced the head-scratching moment when their React components become so gigantic that they keep losing their train of thought when navigating through their frontend code. To salvage readability, one of the most common practices is to split up the humongous React components into smaller pieces. Traditionally, they would do this by chopping up the JSX elements. This method can effectively improve the readability of React components whose sole responsibility is to display data. However, on top of displaying data, some React components are also responsible for retrieving data from the backend. The more effective way to make such React components readable is to separate them into Containers and Views. In this article, I will demonstrate how to divide a React component into a Container and a View based on the principle of separation of concerns.

It is almost impossible for a backend engineer to be oblivious to RESTful API in the 21st century, as it is one of the most popular API types. We can mostly attribute its popularity to its scalability, flexibility, and simplicity, all of which are highly sought-after qualities in modern APIs. If you also find these qualities desirable, you should consider building APIs following the RESTful principles. In this article, I will explain what counts as a RESTful API and demonstrate how to design one from scratch.

The first programming paradigm that most programmers in the 21st century learn is probably Object Oriented Programming (or OOP for short). Object Oriented Programming allows us to transform real-world entities into highly abstract models with descriptive properties and executable methods to simulate their behaviors in the real world. Despite its numerous advantages, such as readability, reusability, and maintainability, it also introduces immense complexity to state management. The difficulty in keeping track of the states of objects has led head-scratching exhausted programmers to churn out error-prone code and cause bugs that would take hours, or even days, to locate and fix. “Is there a way out of this undesirable situation,” you might ask. Here is the good news: a lesser-known yet widely-used programming paradigm called Functional Programming (or FP for short) can help turn the tide around and decrease the number of bugs we cause. In this article, I will explain what Functional Programming is and why it can help us avoid bugs.

Incorporating OAuth (short for Open Authorization) into an application might seem somewhat intimidating and disheartening for many entry-level software engineers. After all, they need to spend hours—or even days—implementing a complicated authentication process with the correct configuration, only to realize they have merely completed a tiny feature. Despite all the hard work it entails, OAuth is a safe and efficient way for users to grant websites and applications access to their personal information. Therefore, it is worth the effort to comprehend how OAuth works. In this article, I will break down all the steps involved in the OAuth process to help you kick-start your OAuth journey.

If you became a frontend engineer after 2015, chances are that you have used or at least heard about Promise. As I have covered in my previous blog, Promise can help us specify sequential relations between operations in asynchronous programming in a readable and maintainable manner. Promise object in JavaScript has methods, such as then and catch, that can help us organize the sequential execution of operations in a pattern called Promise Chaining. Clean and organized as Promise Chaining is, some people—especially those more accustomed to synchronous programming—find it unintuitive and confusing. Lucky for them, ECMAScript 2017 introduced a new feature that allows us to write asynchronous code as if it were synchronous. This new feature consists of two keywords: async and await. In this article, I will cover the fundamentals of async/await and how to transition from Promise Chaining to using async/await.

If you have done any web development after 2015, chances are that you have heard of the concept of Promise. It wouldn’t be an overstatement to claim that Promise is ubiquitous in modern-day front-end codebases. However, many web developers—especially those who do not have much experience in front-end development—have been using Promise without thoroughly understanding its inner working. This has led to countless misuses of Promise and consequent bugs. In this article, I will walk you through what motivated the creation of Promise, how Promise works, and how we should use it in our code, under the assumption that you don’t have much experience in front-end development.

Just like real life, the world of computer science is replete with trade-offs. Relational databases are no exception. When interacting with relational databases, we face the dilemma between data consistency and transaction concurrency. The former guarantees the data is trustworthy, while the latter ensures relational databases can conduct transactions swiftly. Both are desirable qualities of relational databases, but we cannot simultaneously achieve them to the fullest extent. Today, I will discuss how isolation levels can help us structure our decision-making regarding this dilemma.

In the previous blog, I introduced the famous CAP Theorem (please give it a read if you haven’t already before you start reading this one). It involves a trilemma of needing to give up one of the following three qualities: consistency, availability, and partition tolerance. Since all three are desirable features of modern-day distributed systems, determining which one to relinquish has become one of the most important and delicate decisions for designers of complicated distributed systems. While the CAP Theorem is widely-known in computer science, its extension, the PACELC theorem, has received less attention. Today, I will shine a long-overdue spotlight on the PACELC theorem.

The world that we live in is far from perfect. We constantly find ourselves in dilemmas, sometimes even trilemmas, that require us to make trade-offs. When shopping, we can only choose two out of “cheap,” “fast,” and “good.” In economics, a government cannot enjoy “sovereign monetary policy,” “fixed exchange rate,” and “free capital flow” at the same time. It can only achieve two of them by giving up the third. Similarly, the CAP (consistency, availability, and partition tolerance) theorem involves an equally head-scratching trilemma that has troubled computer scientists and software engineers ever since distributed computing became a popular solution to large-scale computation. Today, we will dive deep into the CAP theorem and learn how to make wise trade-offs based on our needs.

There’s no denying that we have already ushered in the era of big data. An enormous amount of information is generated every second. While decision-makers can gain invaluable insights from this ever-growing data, its sheer volume also poses considerable challenges to data engineers–greater demand for storage spaces, the need to handle increasingly complex data formats, and highly unpredictable network traffic. Luckily, recent years have witnessed the creation of various technologies devoted to efficiently digesting big data, and Kafka is one of them. Today, I will demonstrate how Kafka works to help kick-start your Kafka journey.

As a data engineer, you probably have heard about Hadoop. It is one of the most popular frameworks for distributed processing of large data sets. It is less costly and more secure than other frameworks. At its center is a programming model called MapReduce. Today we will take a closer look at MapReduce to understand the inner working of Hadoop.