ByteByteGo

I’ve been writing the system design newsletter for 12 months. Here are the 5 most popular ones: 👇 1. From 0 to Millions: A Guide to Scaling Your App 2. A Crash Course in Caching 3. API Architectural Styles 4. How does ChatGPT work? 5. 8 Data Structures That Power Your Databases Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/3FEGliw .

How to design a 𝐬𝐞𝐜𝐮𝐫𝐞 web API access for your website? . . When we open web API access to users, we need to make sure each API call is authenticated. This means the user must be whom they claim to be.   In this post, we explore two common ways: 1. Token based authentication 2. HMAC (Hash-based Message Authentication Code) authentication   The diagram below illustrates how they work.   𝐓𝐨𝐤𝐞𝐧 𝐛𝐚𝐬𝐞𝐝 Step 1 - the user enters their password into the client, and the client sends the password to the Authentication Server.   Step 2 - the Authentication Server authenticates the credentials and generates a token with an expiry time.   Steps 3 and 4 - now the client can send requests to access server resources with the token in the HTTP header. This access is valid until the token expires.   𝐇𝐌𝐀𝐂 𝐛𝐚𝐬𝐞𝐝 This mechanism generates a Message Authentication Code (signature) by using a hash function (SHA256 or MD5).   Steps 1 and 2 - the server generates two keys, one is Public APP ID (public key) and the other one is API Key (private key). Step 3 - we now generate a HMAC signature on the client side (hmac A). This signature is generated with a set of attributes listed in the diagram.   Step 4 - the client sends requests to access server resources with hmac A in the HTTP header.   Step 5 - the server receives the request which contains the request data and the authentication header. It extracts the necessary attributes from the request and uses the API key that’s stored on the server side to generate a signature (hmac B.)   Steps 6 and 7 - the server compares hmac A (generated on the client side) and hmac B (generated on the server side). If they are matched, the requested resource will be returned to the client.   Question - How does HMAC authentication ensure data integrity? Why do we include “request timestamp” in HMAC signature generation? --  Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social #systemdesign #coding #interviewtips  .

How does 𝐆𝐨𝐨𝐠𝐥𝐞 𝐀𝐮𝐭𝐡𝐞𝐧𝐭𝐢𝐜𝐚𝐭𝐨𝐫 (or other types of 2-factor authenticators) work?   Google authenticator is commonly used for logging into our accounts when 2-factor authentication is enabled. How does it guarantee security?   Google Authenticator is a software-based authenticator that implements a two-step verification service. The diagram below provides detail.    There are two stages involved: 🔹 Stage 1 - The user enables Google two-step verification  🔹 Stage 2 - The user uses the authenticator for logging in, etc.   Let’s look at these stages.   𝐒𝐭𝐚𝐠𝐞 1 Steps 1 and 2: Bob opens the web page to enable two-step verification. The front end requests a secret key. The authentication service generates the secret key for Bob and stores it in the database.   Step 3: The authentication service returns a URI to the front end. The URI is composed of a key issuer, username, and secret key. The URI is displayed in the form of a QR code on the web page.   Step 4: Bob then uses Google Authenticator to scan the generated QR code. The secret key is stored in the authenticator.   𝐒𝐭𝐚𝐠𝐞 2 Steps 1 and 2: Bob wants to log into a website with Google two-step verification. For this, he needs the password. Every 30 seconds, Google Authenticator generates a 6-digit password using TOTP (Time-based One Time Password) algorithm. Bob uses the password to enter the website.   Steps 3 and 4: The front end sends Bob's password to the backend for authentication. The authentication service reads the secret key from the database and generates a 6-digit password using the same TOTP algorithm as the client.   Step 5: The authentication service compares the two passwords generated by the client and the server, and returns the comparison result to the front. Bob can proceed with the login process only if the two passwords match.   Is this authentication mechanism 𝐬𝐚𝐟𝐞?  🔹 Can the secret key be obtained by others?  We need to make sure the secret key is transmitted using HTTPS. The authenticator client and the database store the secret key, and we need to ensure the secret keys are encrypted. 🔹 Can the 6-digit password be guessed by hackers? No. The password has 6 digits, so the generated password has 1 million potential combinations. Plus, the password changes every 30 seconds. If hackers want to guess the password in 30 seconds, they need to enter 30,000 combinations per second.   Over to you: What are some of the other 2-factor authentication devices you used? --  Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social #systemdesign #coding #interviewtips  .

What is SSO (Single Sign-On)? . . Basically, Single Sign-On (SSO) is an authentication scheme. It allows a user to log in to different systems using a single ID. The diagram below illustrates how SSO works. Step 1: A user visits Gmail, or any email service. Gmail finds the user is not logged in and so redirects them to the SSO authentication server, which also finds the user is not logged in. As a result, the user is redirected to the SSO login page, where they enter their login credentials. Steps 2-3: The SSO authentication server validates the credentials, creates the global session for the user, and creates a token. Steps 4-7: Gmail validates the token in the SSO authentication server. The authentication server registers the Gmail system, and returns “valid.” Gmail returns the protected resource to the user. Step 8: From Gmail, the user navigates to another Google-owned website, for example, YouTube. Steps 9-10: YouTube finds the user is not logged in, and then requests authentication. The SSO authentication server finds the user is already logged in and returns the token. Step 11-14: YouTube validates the token in the SSO authentication server. The authentication server registers the YouTube system, and returns “valid.” YouTube returns the protected resource to the user. The process is complete and the user gets back access to their account. Over to you:  Question 1: have you implemented SSO in your projects? What is the most difficult part? Question 2: what’s your favorite sign-in method and why? --  Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social #systemdesign #coding #interviewtips  .

Understanding Database Types To make the best decision for our projects, it is essential to understand the various types of databases available in the market. We need to consider key characteristics of different database types, including popular options for each, and compare their use cases. --  Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social #systemdesign #coding #interviewtips  .

If you use JSON files, you'll probably like this tool 👇 Nested JSON files are hard to read. 𝐉𝐬𝐨𝐧𝐂𝐫𝐚𝐜𝐤 generates graph diagrams from JSON files and makes them easy to read. Additionally, the generated diagrams can be downloaded as images. --  Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social #systemdesign #coding #interviewtips  .

How do Apple Pay and Google Pay handle sensitive card info? The diagram below shows the differences. Both approaches are very secure, but the implementations are different. To understand the difference, we break down the process into two flows. 1 Registering your credit card flow 2 Basic payment flow 1. The registration flow is represented by steps 1~3 for both cases. The difference is: Apple Pay: Apple doesn’t store any card info. It passes the card info to the bank. Bank returns a token called DAN (device account number) to the iPhone. iPhone then stores DAN into a special hardware chip. Google Pay: When you register the credit card with Google Pay, the card info is stored in the Google server. Google returns a payment token to the phone. 2. When you click the “Pay” button on your phone, the basic payment flow starts. Here are the differences: Apple Pay: For iPhone, the e-commerce server passes the DAN to the bank. Google Pay: In the Google Pay case, the e-commerce server passes the payment token to the Google server. Google server looks up the credit card info and passes it to the bank. In the diagram, the red arrow means the credit card info is available on the public network, although it is encrypted. --  Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social #systemdesign #coding #interviewtips  .

How do live streaming platforms like YouTube Live, TikTok Live, or Twitch work?    Live streaming is challenging because the video content is sent over the internet in near real-time. Video processing is compute-intensive. Sending a large volume of video content over the internet takes time. These factors make live streaming challenging.    The diagram below explains what happens behind the scenes to make this possible.    Step 1: The streamer starts their stream. The source could be any video and audio source wired up to an encoder    Step 2: To provide the best upload condition for the streamer, most live streaming platforms provide point-of-presence servers worldwide. The streamer connects to a point-of-presence server closest to them.    Step 3: The incoming video stream is transcoded to different resolutions, and divided into smaller video segments a few seconds in length.    Step 4: The video segments are packaged into different live streaming formats that video players can understand. The most common live-streaming format is HLS, or HTTP Live Streaming.    Step 5: The resulting HLS manifest and video chunks from the packaging step are cached by the CDN.    Step 6: Finally, the video starts to arrive at the viewer’s video player.    Step 7-8: To support replay, videos can be optionally stored in storage such as Amazon S3. --  Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social #systemdesign #coding #interviewtips  .

Why is Nginx called a “𝐫𝐞𝐯𝐞𝐫𝐬𝐞” proxy? . . The diagram below shows the differences between a 𝐟𝐨𝐫𝐰𝐚𝐫𝐝 𝐩𝐫𝐨𝐱𝐲 and a 𝐫𝐞𝐯𝐞𝐫𝐬𝐞 𝐩𝐫𝐨𝐱𝐲. 🔹 A forward proxy is a server that sits between user devices and the internet. A forward proxy is commonly used for:  1️⃣ Protect clients 2️⃣ Avoid browsing restrictions 3️⃣ Block access to certain content 🔹 A reverse proxy is a server that accepts a request from the client, forwards the request to web servers, and returns the results to the client as if the proxy server had processed the request. A reverse proxy is good for: 1️⃣ Protect servers 2️⃣ Load balancing 3️⃣ Cache static contents 4️⃣ Encrypt and decrypt SSL communications Over to you: What’s the difference between a reverse proxy and a load balancer? What are some of the most popular proxy servers? --  Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social #systemdesign #coding #interviewtips  .

How will you design the Stack Overflow website? If your answer is on-premise servers and monolith (on the right), you would likely fail the interview, but that's how it is built in reality! 𝐖𝐡𝐚𝐭 𝐩𝐞𝐨𝐩𝐥𝐞 𝐭𝐡𝐢𝐧𝐤 𝐢𝐭 𝐬𝐡𝐨𝐮𝐥𝐝 𝐥𝐨𝐨𝐤 𝐥𝐢𝐤𝐞 The interviewer is probably expecting something on the left side. 1. Microservice is used to decompose the system into small components. 2. Each service has its own database. Use cache heavily. 3. The service is sharded. 4. The services talk to each other asynchronously through message queues. 5. The service is implemented using Event Sourcing with CQRS. 6. Showing off knowledge in distributed systems such as eventual consistency, CAP theorem, etc. 𝐖𝐡𝐚𝐭 𝐢𝐭 𝐚𝐜𝐭𝐮𝐚𝐥𝐥𝐲 𝐢𝐬 Stack Overflow serves all the traffic with only 9 on-premise web servers, and it’s on monolith! It has its own servers and does not run on the cloud. This is contrary to all our popular beliefs these days. Over to you: what is good architecture, the one that looks fancy during the interview or the one that works in reality? --  Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social #systemdesign #coding #interviewtips  .