Sweet Codey

𝙀𝙫𝙚𝙧 𝙬𝙤𝙣𝙙𝙚𝙧𝙚𝙙 𝙝𝙤𝙬 𝙫𝙞𝙙𝙚𝙤 𝙨𝙩𝙧𝙚𝙖𝙢𝙞𝙣𝙜 𝙬𝙤𝙧𝙠𝙨? 🎥✨ 𝗜𝘁’𝘀 𝗮 𝘁𝗵𝗿𝗲𝗲-𝘀𝘁𝗲𝗽 𝗽𝗿𝗼𝗰𝗲𝘀𝘀: 1. Client Request: The client sends a request to play a specific video. 📲 2. Manifest File: The server responds with a manifest file containing the locations of video chunks. 📄 3. Streaming: The client streams these chunks from the CDN using the HLS protocol. 🌐 The manifest file lists chunk locations in various formats and qualities. For example, if you request a chunk in MP4 format, the CDN delivers it in the requested quality—be it 4K or 240p—depending on your internet connection. This adaptive streaming ensures a smooth viewing experience, even as your connection fluctuates! Curious to learn more? Check out the full explanation here: https://lnkd.in/g6R5ABbx

𝙀𝙫𝙚𝙧 𝙬𝙤𝙣𝙙𝙚𝙧𝙚𝙙 𝙝𝙤𝙬 𝙖 𝙘𝙤𝙣𝙩𝙚𝙣𝙩 𝙥𝙧𝙤𝙘𝙚𝙨𝙨𝙤𝙧 𝙬𝙤𝙧𝙠𝙛𝙡𝙤𝙬 𝙚𝙣𝙜𝙞𝙣𝙚 𝙤𝙥𝙚𝙧𝙖𝙩𝙚𝙨? 🤔 It involves four key services: 1. Content Chunker Service: Breaks down videos into smaller chunks and sends event notifications. 📦 2. Format Converter Service: Converts chunks into compatible formats like MP4 and MOV. 🎞️ 3. Quality Converter Service: Adjusts chunks into various quality levels (4K, 720p) for optimal viewing. 🌐 4. CDN Uploader Service: Uploads the final chunks to the CDN and logs their locations in the database. 🚀 This seamless process ensures videos are ready for smooth streaming! 𝗪𝗮𝗻𝘁 𝘁𝗼 𝗹𝗲𝗮𝗿𝗻 𝗺𝗼𝗿𝗲? 𝗖𝗵𝗲𝗰𝗸 𝗼𝘂𝘁 𝘁𝗵𝗲 𝗳𝘂𝗹𝗹 𝗲𝘅𝗽𝗹𝗮𝗻𝗮𝘁𝗶𝗼𝗻 𝗵𝗲𝗿𝗲: https://lnkd.in/gakzskVD

𝙀𝙫𝙚𝙧 𝙬𝙤𝙣𝙙𝙚𝙧𝙚𝙙 𝙝𝙤𝙬 𝙫𝙞𝙙𝙚𝙤𝙨 𝙖𝙧𝙚 𝙪𝙥𝙡𝙤𝙖𝙙𝙚𝙙 𝙩𝙤 𝙥𝙡𝙖𝙩𝙛𝙤𝙧𝙢𝙨 𝙡𝙞𝙠𝙚 𝙔𝙤𝙪𝙏𝙪𝙗𝙚? 𝙄𝙩 𝙖𝙡𝙡 𝙨𝙩𝙖𝙧𝙩𝙨 𝙬𝙝𝙚𝙣 𝙮𝙤𝙪 𝙝𝙞𝙩 𝙩𝙝𝙚 𝙪𝙥𝙡𝙤𝙖𝙙 𝙗𝙪𝙩𝙩𝙤𝙣! 🚀 First, the video metadata—like title and format—is sent to the API gateway, which routes it to the content upload service. Once the server receives this info, it generates a session URL for the actual video upload. 📂 Then, using this session URL, the client sends the video content in chunks to object storage. After the upload, a message containing the video ID is sent to the message queue, setting off a series of automated processing steps. The content processor breaks the video into smaller chunks and converts them into various formats and resolutions. Finally, these chunks are stored in a CDN, ensuring quick access for viewers. 🌐 𝗖𝘂𝗿𝗶𝗼𝘂𝘀 𝘁𝗼 𝗹𝗲𝗮𝗿𝗻 𝗺𝗼𝗿𝗲 𝗮𝗯𝗼𝘂𝘁 𝘃𝗶𝗱𝗲𝗼 𝗽𝗿𝗼𝗰𝗲𝘀𝘀𝗶𝗻𝗴? 𝗖𝗵𝗲𝗰𝗸 𝗼𝘂𝘁 𝘁𝗵𝗲 𝗳𝘂𝗹𝗹 𝗲𝘅𝗽𝗹𝗮𝗻𝗮𝘁𝗶𝗼𝗻 𝗵𝗲𝗿𝗲:https://lnkd.in/gWTx__HF

𝙀𝙫𝙚𝙧 𝙬𝙤𝙣𝙙𝙚𝙧 𝙝𝙤𝙬 𝙘𝙤𝙣𝙩𝙚𝙣𝙩 𝙪𝙥𝙡𝙤𝙖𝙙 𝙬𝙤𝙧𝙠𝙨 𝙨𝙚𝙖𝙢𝙡𝙚𝙨𝙨𝙡𝙮 𝙤𝙣 𝙥𝙡𝙖𝙩𝙛𝙤𝙧𝙢𝙨 𝙡𝙞𝙠𝙚 𝙔𝙤𝙪𝙏𝙪𝙗𝙚? The process involves efficient routing and session management to ensure smooth uploads! 🛠️ 𝗛𝗶𝗴𝗵-𝗟𝗲𝘃𝗲𝗹 𝗗𝗲𝘀𝗶𝗴𝗻 𝗢𝘃𝗲𝗿𝘃𝗶𝗲𝘄: When a user clicks the upload button, the process begins with a request to the API gateway. 🔗 𝗔𝗣𝗜 𝗚𝗮𝘁𝗲𝘄𝗮𝘆: Think of it as the receptionist of a hotel—managing and routing requests to the right services. In this case, it directs the upload request to the content upload service via a load balancer. 📋 𝗖𝗼𝗻𝘁𝗲𝗻𝘁 𝗨𝗽𝗹𝗼𝗮𝗱 𝗣𝗿𝗼𝗰𝗲𝘀𝘀: 1. Routing: The API gateway sends the upload request to the Content Upload Service. 2. Metadata Storage: The content upload service adds video metadata (title, format, etc.) to the Videos Database. 3. Confirmation: The client receives a successful confirmation along with a session URL for further uploads. 🔗 Check out the full video explanation here:https://lnkd.in/gz9gFN3G

𝙀𝙫𝙚𝙧 𝙬𝙤𝙣𝙙𝙚𝙧 𝙝𝙤𝙬 𝙥𝙡𝙖𝙩𝙛𝙤𝙧𝙢𝙨 𝙡𝙞𝙠𝙚 𝙔𝙤𝙪𝙏𝙪𝙗𝙚 𝙨𝙩𝙧𝙚𝙖𝙢 𝙫𝙞𝙙𝙚𝙤𝙨 𝙨𝙢𝙤𝙤𝙩𝙝𝙡𝙮 𝙬𝙞𝙩𝙝𝙤𝙪𝙩 𝙗𝙪𝙛𝙛𝙚𝙧𝙞𝙣𝙜? The secret lies in efficient data streaming and chunk management! When a client requests to watch a video, the server sends a manifest file containing locations of video chunks, enabling seamless playback. 💡 𝗪𝗵𝘆 𝗠𝗮𝗻𝗶𝗳𝗲𝘀𝘁 𝗙𝗶𝗹𝗲𝘀? Manifest files list all video chunk locations, allowing clients to fetch each part efficiently rather than downloading the entire video at once. 📂 𝗦𝘁𝗿𝗲𝗮𝗺𝗶𝗻𝗴 𝗣𝗿𝗼𝗰𝗲𝘀𝘀: 1. Initial Request: The client sends a GET request to the V1 watch endpoint, asking to play a specific video. 2. Response: The server responds with a manifest file that contains the locations of video chunks stored in the Content Delivery Network (CDN). 🚀 𝗔𝗱𝗮𝗽𝘁𝗶𝘃𝗲 𝗦𝘁𝗿𝗲𝗮𝗺𝗶𝗻𝗴: Using the HLS protocol, the video quality adjusts based on the user’s internet speed, ensuring uninterrupted viewing experience. 🔗 Check out the full video explanation here:https://lnkd.in/ga2WbmNQ

🚀 Ever wondered how Twitter/X searches work so fast? Let us break it down for you in very simple language. Picture this: a client types in "quantum computing" to search on our platform. Let’s break down the journey of this search: 1️⃣ The client’s search request goes to the API Gateway, where it gets validated and passed on to the Search Service via a Load Balancer. 2️⃣ The Search Service looks at the keywords ("quantom" and "computing"). It checks if these keywords are in the Cache first. If not, it moves to the Index DB. Nothing found for "quantom"? No worries. 3️⃣ The Query Correction Service jumps in, spots the typo, and suggests the correct word, "quantum." 4️⃣ With corrected keywords "quantum" and "computing," the search goes back to the Cache. If still not found, it retrieves data from the Index DB and updates the Cache for future searches (using a Read-Through Mechanism). 5️⃣ The Search Service now has the tweet IDs containing both "quantum" and "computing." It checks the Tweets DB Cache for these specific tweet IDs. If not found there, the Tweets DB Cache updates itself so that future searches are quicker. 6️⃣ The Search Service passes these relevant tweet IDs to the Ranking Service. 7️⃣ The Ranking Service arranges the tweets, ordering them from newest to oldest. 8️⃣ Finally, these sorted tweets are returned to the client as search results. This setup helps us deliver search results quickly and accurately, while keeping things efficient behind the scenes. Found this helpful? Learn more in our Udemy course - https://lnkd.in/e4HkYZN3 , where over 𝟭𝟯,𝟱𝟬𝟬+ 𝗹𝗲𝗮𝗿𝗻𝗲𝗿𝘀 are learning System Design with us. Use 𝗦𝗗_𝗠𝗔𝗦𝗧𝗘𝗥 to get a 𝟲𝟬% 𝗱𝗶𝘀𝗰𝗼𝘂𝗻𝘁!

𝙒𝙝𝙚𝙣 𝙪𝙨𝙚𝙧𝙨 𝙪𝙥𝙡𝙤𝙖𝙙 𝙘𝙤𝙣𝙩𝙚𝙣𝙩 𝙩𝙤 𝙥𝙡𝙖𝙩𝙛𝙤𝙧𝙢𝙨 𝙡𝙞𝙠𝙚 𝙔𝙤𝙪𝙏𝙪𝙗𝙚, 𝙚𝙫𝙚𝙧 𝙬𝙤𝙣𝙙𝙚𝙧𝙚𝙙 𝙝𝙤𝙬 𝙡𝙖𝙧𝙜𝙚 𝙫𝙞𝙙𝙚𝙤 𝙛𝙞𝙡𝙚𝙨 𝙜𝙚𝙩 𝙝𝙖𝙣𝙙𝙡𝙚𝙙? Uploading hours-long videos in a single request isn't practical. Here's a breakdown of how the API design handles this through chunked uploads and resumable sessions, making sure uploads continue smoothly, even when the connection drops. 📤 𝗔𝗣𝗜 𝗗𝗲𝘀𝗶𝗴𝗻 𝗳𝗼𝗿 𝗨𝗽𝗹𝗼𝗮𝗱𝗶𝗻𝗴 𝗖𝗼𝗻𝘁𝗲𝗻𝘁 Step 1: User uploads metadata via a POST request. The server provides a resumable URL for uploading the video in chunks. Step 2: The client uses this URL to upload video chunks using PUT requests, ensuring the video is uploaded piece by piece. This resumable process ensures even large videos get uploaded efficiently, without interruptions. For a detailed explanation, check out the video here-https://lnkd.in/gZkD_68v

𝙀𝙫𝙚𝙧 𝙬𝙤𝙣𝙙𝙚𝙧 𝙝𝙤𝙬 𝙢𝙪𝙘𝙝 𝙙𝙖𝙩𝙖 𝙛𝙡𝙤𝙬𝙨 𝙩𝙝𝙧𝙤𝙪𝙜𝙝 𝙨𝙮𝙨𝙩𝙚𝙢𝙨 𝙡𝙞𝙠𝙚 𝙔𝙤𝙪𝙏𝙪𝙗𝙚 𝙞𝙣 𝙧𝙚𝙖𝙡-𝙩𝙞𝙢𝙚? In this section, we'll explore network capacity, focusing on both data entering (ingress) and exiting (egress) the system. 💻 𝗡𝗲𝘁𝘄𝗼𝗿𝗸 𝗘𝘀𝘁𝗶𝗺𝗮𝘁𝗶𝗼𝗻 Ingress: With 240 TB of data uploaded daily, the ingress rate equals 2.7 GB per second. Egress: With 1 billion video views daily, the egress reaches 7 TB per second. 🔗 For more detailed insights, check out the full explanation here:https://lnkd.in/gM6_7DnC

𝙀𝙫𝙚𝙧 𝙬𝙤𝙣𝙙𝙚𝙧 𝙝𝙤𝙬 𝙥𝙡𝙖𝙩𝙛𝙤𝙧𝙢𝙨 𝙠𝙚𝙚𝙥 𝙫𝙞𝙙𝙚𝙤𝙨 𝙞𝙣𝙨𝙩𝙖𝙣𝙩𝙡𝙮 𝙖𝙫𝙖𝙞𝙡𝙖𝙗𝙡𝙚 𝙬𝙞𝙩𝙝𝙤𝙪𝙩 𝙙𝙚𝙡𝙖𝙮𝙨? The secret lies in efficient memory caching. In our system, we estimate the required cache memory to ensure fast video access. 💡 Why Cache? Cache speeds up data retrieval, minimizing the slow access time from databases. 📊 Daily Cache Estimate: With 240 TB of new data uploaded daily, we store 1% in cache—about 2.4 TB—for faster access. 🔗 Check out the full video explanation here:https://lnkd.in/gMDdV-S6

𝙀𝙫𝙚𝙧 𝙩𝙝𝙤𝙪𝙜𝙝𝙩 𝙖𝙗𝙤𝙪𝙩 𝙝𝙤𝙬 𝙢𝙪𝙘𝙝 𝙨𝙩𝙤𝙧𝙖𝙜𝙚 𝙖 𝙥𝙡𝙖𝙩𝙛𝙤𝙧𝙢 𝙡𝙞𝙠𝙚 𝙔𝙤𝙪𝙏𝙪𝙗𝙚 𝙧𝙚𝙦𝙪𝙞𝙧𝙚𝙨 𝙩𝙤 𝙝𝙖𝙣𝙙𝙡𝙚 𝙩𝙝𝙚 𝙚𝙣𝙙𝙡𝙚𝙨𝙨 𝙛𝙡𝙤𝙬 𝙤𝙛 𝙫𝙞𝙙𝙚𝙤 𝙪𝙥𝙡𝙤𝙖𝙙𝙨? Let’s break down the staggering numbers! ⚡ 𝗖𝗮𝗽𝗮𝗰𝗶𝘁𝘆 𝗦𝘁𝗼𝗿𝗮𝗴𝗲 𝗘𝘀𝘁𝗶𝗺𝗮𝘁𝗶𝗼𝗻 ⚡ To keep up with YouTube-like scale, we estimate the storage requirements: 📹 𝗩𝗶𝗱𝗲𝗼 𝗦𝗶𝘇𝗲: The average video upload size is around 600 MB. 📈 𝗨𝗽𝗹𝗼𝗮𝗱𝘀 𝗣𝗲𝗿 𝗗𝗮𝘆: From the throughput estimation, we know 0.4 million videos are uploaded daily. 💾 𝗗𝗮𝗶𝗹𝘆 𝗦𝘁𝗼𝗿𝗮𝗴𝗲: Multiply 600 MB by 0.4 million uploads = 240 TB of new video data stored every day! ⏳ 𝟭𝟬-𝗬𝗲𝗮𝗿 𝗘𝘀𝘁𝗶𝗺𝗮𝘁𝗲: Over 10 years, this totals to 876 petabytes of data (240 TB x 365 days x 10 years). That’s an immense amount of data—enough to fill thousands of hard drives every year! How do platforms manage such massive storage? 🔗https://lnkd.in/gjaQudWC