Data Engineer Things

☠ Data Death Cycle: The Silo Trap 🖋️ Author: Hugo Lu 🔗 Read the article here: https://lnkd.in/eYkdcQ22 ------------------------------------------- ✅ Follow Data Engineer Things for more insights and updates. 💬 Hit the 'Like' button if you enjoyed the article. ------------------------------------------- #dataengineering #sql #generativeai #dataanalytics

⏳ I spent 5 hours learning how ClickHouse built their internal data warehouse. 🖋️ Author: Vu Trinh 🔗 Read the article here: https://lnkd.in/dT3Wq-cR ------------------------------------------- ✅ Follow Data Engineer Things for more insights and updates. 💬 Hit the 'Like' button if you enjoyed the article. ------------------------------------------- #dataengineering #clickhouse #datawarehouse #dataanalytics

🖼 12 Unique Ways to Create Spark DataFrames (You Won’t Believe the 10th Option!) 🖋️ Author: Rahul Madhani 🔗 Read the article here: https://lnkd.in/eTgQMcpb ------------------------------------------- ✅ Follow Data Engineer Things for more insights and updates. 💬 Hit the 'Like' button if you enjoyed the article. ------------------------------------------- #dataengineering #spark #python #sql

The AI bottleneck isn't data; it's people 💡. A recent conversation with Yaakov Bressler inspired me to explore the crucial question: Do you have enough data to use AI, or is something else holding you back? My latest article dives into this topic, examining: 📚 - The critical importance of skilled data teams for AI success 🔬 - The misconception of data quantity and the relevance of the Small Data SF movement (sometimes, less is more!) ⚙️ - The capabilities and limitations of no-code and low-code AI platforms https://lnkd.in/esazXxqv Check it out and share your thoughts! Join the Data Engineer Things community by Xinran Waibel to find more insightful articles 📚. #dataengineering #ai #artificialintelligence #data #smalldata #nocode #lowcode #bigdata #dataanalytics #dataanalysis #dataengineer #dataengineerthings #lessismore 🫶 𝘚𝘱𝘦𝘤𝘪𝘢𝘭 𝘵𝘩𝘢𝘯𝘬𝘴 𝘵𝘰 𝘠𝘢𝘢𝘬𝘰𝘷 𝘢𝘯𝘥 𝘴𝘩𝘰𝘶𝘵𝘰𝘶𝘵 𝘵𝘰 Small Data SF

✨ Why Does the "Executor Out of Memory" Error Happen in Apache Spark? 🖋️ Author: Vishal Barvaliya 🔗 Read the article here: https://lnkd.in/esmk9A9r ------------------------------------------- ✅ Follow Data Engineer Things for more insights and updates. 💬 Hit the 'Like' button if you enjoyed the article. ------------------------------------------- #dataengineering #datascience #spark #python

💻 APIs for Data Engineers (Part 2) - Interacting with APIs: The Requests Library and the Structure of an API Request 🖋️ Author: Aminat Lawal 🔗 Read the article here: https://lnkd.in/e_PWzcba ------------------------------------------- ✅ Follow Data Engineer Things for more insights and updates. 💬 Hit the 'Like' button if you enjoyed the article. ------------------------------------------- #dataengineering #api #programming #python

Are you a Data Engineer working with #AI? We'd love to hear from you. Share your work with greater DE community by writing for our publication.

🎉 A fun experiment: using Marquez as a lineage tool for Celery 🖋️ Author: Marin Aglić Čuvić 🔗 Read the article here: https://lnkd.in/e57drbVg ------------------------------------------- ✅ Follow Data Engineer Things for more insights and updates. 💬 Hit the 'Like' button if you enjoyed the article. ------------------------------------------- #dataengineering #python #celery #datalineage

🚗 Uber’s Big Data Revolution: From MySQL to Hadoop and Beyond 🖋️ Author: Vu Trinh 🔗 Read the article here: https://lnkd.in/e_i_2ShF ------------------------------------------- ✅ Follow Data Engineer Things for more insights and updates. 💬 Hit the 'Like' button if you enjoyed the article. ------------------------------------------- #dataengineering #bigdata #dataanalytics #data