dltHub

🚀 Learning Fridays: Portable Data Lake Workshop announcement 🚀 7th of November, Sign up to get the content async if you cannot make it. Link: https://lnkd.in/evkUM_kp What's a portable data lake? Read more here: https://lnkd.in/dB5PNXpY 🌊💻 What you’ll learn: Building your own portable data lake: We'll use dlt, Parquet, and DuckDB to create an open, lightweight data lake. Embedded catalog: Explore using Ibis to manage metadata without relying on heavy catalogs like those required by Iceberg. Integration possibilities: Learn how to export transformed data to platforms like Snowflake or BigQuery. Don't miss out—this is your chance to step into the open-source data movement, breaking free from vendor lock-in and embracing flexible, cloud-agnostic data solutions.

🚀 Learning Fridays: Portable Data Lake Workshop announcement 🚀 7th of November, Sign up to get the content async if you cannot make it. Link: https://lnkd.in/evkUM_kp What's a portable data lake? Read more here: https://lnkd.in/dB5PNXpY 🌊💻 What you’ll learn: Building your own portable data lake: We'll use dlt, Parquet, and DuckDB to create an open, lightweight data lake. Embedded catalog: Explore using Ibis to manage metadata without relying on heavy catalogs like those required by Iceberg. Integration possibilities: Learn how to export transformed data to platforms like Snowflake or BigQuery. Don't miss out—this is your chance to step into the open-source data movement, breaking free from vendor lock-in and embracing flexible, cloud-agnostic data solutions.

Streamline your data pipeline with the powerful combo of #SQLMesh and dltHub! This integration accelerates data workflows by enabling seamless metadata handovers, automating scaffolding, and supporting incremental processing. 💻 Boost efficiency, reduce errors, and unlock the full potential of your 𝒑𝒐𝒔𝒕𝒎𝒐𝒅𝒆𝒓𝒏 data stack. 😉 🔗 https://lnkd.in/gZ3SaV43 #DataEngineering #dlt #DataPipelines

If you are in San Francisco on November 6th and free 5-7pm, check out my buddy's Matthaus Krzykowski workshop and demos on Open Data Lakes hosted by Foundation Capital and dltHub. Josh Wills will be there with his demo, as well as Alexander Butler and Marcin Rudolf, dltHub's CTO. Venue: Foundation Capital San Francisco 2 Shaw Alley, 4th Floor San Francisco, 94105 RSVP link: https://lnkd.in/e7cyib-r

𝐓𝐡𝐞 𝐦𝐨𝐝𝐞𝐫𝐧 𝐝𝐚𝐭𝐚 𝐬𝐭𝐚𝐜𝐤 𝐭𝐨𝐨𝐥𝐬 𝐝𝐨𝐧'𝐭 𝐭𝐚𝐥𝐤 𝐭𝐨 𝐞𝐚𝐜𝐡 𝐨𝐭𝐡𝐞𝐫 𝐞𝐧𝐨𝐮𝐠𝐡! Modern data stack tools are fighting to push boundaries, and this fight sometimes gets them in conflict with each other. You may have heard of a recent disagreement between Tobiko and dbt Labs, competitors in the T layer, around Coalesce. In happier news, Tobiko team are working on a SQLMesh + dlt metadata handover that enables SQLMesh to generate scaffolds with built in incremental models for dlt pipelines. Why is this awesome and pushing boundaries? Because metadata handovers make open source tools highly interoperable. Open apis on those tools enable other vendors or developers to use them as de facto standards and build better pipelines. Check out the amazing work being done by Tobiko team on this project: dlt-SQLMesh generator: https://lnkd.in/eSNFBBcC

A portable data lake is what you need to go from local to prod without hitches.

I’m officially an Advanced ELT Specialist, thanks to the dltHub workshop! 🎉 The workshop was an amazing experience and helped me dive deeper into handling data extraction and loading easily with dlt. If you work with data, I highly recommend it! 💯 Big thanks to Alena Astrakhantseva and Hiba Jamal for preparing the workshop and providing such practical examples. If anyone’s interested, I’ll share the link to the upcoming workshops in the first comment.

I’m happy to share that I’ve obtained a new certification: dlt Advanced ELT Specialist from dltHub! I've been working with dlt over the last month to standardise and simplify our extract and load processes, while hoping to push the project to production soon

I'm thrilled to announce … Nope. It is with great pride … Uhmm, still nope. Fantastic news! … Oh, gosh, definitely no. How about that: if you want to master ELT using Python you should definitely attend the official workshop from dltHub. I’ve completed one in August and now I’m a certified 😎 Advanced ELT specialist! Thanks to Alena Astrakhantseva and Hiba Jamal for preparing the workshop and providing all the examples.