Chaos Genius’ Post

Quick Tip: Whenever you make design decision, make sure that you use the 'right tool for its purpose'. With respect to Databricks and Power BI :-- 1. Implement the 'Medallion Architecture' in Databricks and perform the following operations in the appropriate layers, Bronze Layer -- Raw data Ingestion ('As-is' source format) Silver Layer -- Cleaned/Filtered/Transformed data. Here is where you can even have your 'facts' and 'dims' (Star Schema Model) Gold Layer -- Denormalized views/ Ready for reporting / Business Aggregated data / Business specific naming conventions 2. Once the Gold layer is ready -- bring in the required 'Gold' tables into Power BI Dataset and start creating relationships/model between the Gold tables and create KPIs using DAX measures. Power BI Dataset is an area where you need to create your dynamic calculations (i.e. calculation based on the user selections of the filters/slicers in the report). Here is where DAX measures comes into place and it is a perfect use for the 'Dynamic based calculation/measures'. Power BI Dataset is also an area -- where you need to create a relationship between the Gold layer tables ( from Databricks ) to enable the filter/data flow and enable the instructiveness in the report canvas area. 3. Visualization -- Use Power BI Reports to display the insights in a meaningful visual format that can be sliced and diced with different data points/visual elements available in the report page. Conclusion: We must ensure that we use the right tool according to its purpose. We should not be creating any calculated columns / calculated tables / aggregated tables within Power BI Dataset instead it should be done at the upstream end/Databricks end (at the gold layer) which is the right area to perform the operations. Similarly, we must not create any dynamic calculations in the Databricks end as it is difficult to pre-compute the results based on the different possible selections made by the end users in the report side, instead these kinds of dynamic calculations should be created using Power BI DAX measure inside Power BI Dataset (downstream side). #databricks #powerbi #medallion architecture #gold tables #DAX measure

Databricks Lakeview Dashboard, why it matters ? 🤔 Few weeks ago, I had the privilege of presenting Lakeview Dashboards feature to the Databricks Montreal User Group amazing community. Since then, I have continued to test this feature, and let me tell you, I LOVE IT 😄 Here's what you need to know about Lakeview Dashboards: 1️⃣ Unlike Databricks SQL, Lakeview Dashboards features a canvas where you can build visuals, just as you will do with your favorite visualization tool (Power BI, Tableau, Qlik, etc.). For those who have tried the legacy Databricks SQL, you know how tedious it was to build queries beforehand. 2️⃣ The dataset is optimized. This means that if your dataset contains fewer than 64 thousand rows, it is encapsulated into the dashboard, providing exceptional refresh performance. For larger dataset, client requests are sent to the backend using wrapped queries ; you can expect response times to be up to 10x faster using the Lakeview Dashboard compared to Databricks SQL. 3️⃣ Flawless Unity Catalog integration. The dashboards will display UC lineage for data governance, as well as give you the ability to control which data is shown to your users, depending on their privileges. 4️⃣ Lakeview Dashboards are also optimized for sharing. You can share with users who are in the current workspace, another workspace, or even those at the account level. Many thanks to Vincent Fortier and his amazing team for all they are doing at #databricks. #lakeview #lakeviewdashboards #databrickssql Image source : databricks.com

The best #DataIntelligencePlatform with the best BI tool! What a combination!! In this blogpost Andrey Mirskiy, Yatish Anand, and I share our insights and recommendations on efficiently adopting existing Power BI semantic models on #DatabricksSQL 🚀 I truly appreciate Andrey's mentorship. With his guidance, my understanding of this topic steadily increased(and I really hope it continues like this). https://lnkd.in/dHTUC5n4 #Databricks #PowerBI #BusinessIntelligence

If you need a definitive guide to PowerBI semantic models on Databricks SQL, check out this awesome blogpost co-written by my fellow colleague Mohammad Shahedi !

🚀 Loading Databricks Delta Tables into Power BI 🚀 As data grows, integrating Databricks Delta tables with Power BI unlocks powerful real-time insights. Let’s explore how to load Delta tables into Power BI and maximize your analytics. 🔷 Steps to Get Started: 1️⃣ Connect to Databricks: - In Power BI Desktop, go to Get Data > Azure > Azure Databricks. - Enter your Databricks URL and authenticate with your Access Token. 2️⃣ Access Your Delta Tables: - Query Delta tables stored in Databricks. Make sure the Delta tables are formatted for integration. 3️⃣ Load Data: - Load the Delta tables into Power BI. Use Power Query Editor to clean and transform your data. 4️⃣ Optimize Queries: - Leverage Databricks SQL to optimize queries. Use filters and aggregations to minimize load times in Power BI. - Delta Lake’s ACID transactions and time travel ensure data consistency. 5️⃣ Create Reports: - Build visuals and dashboards in Power BI using the data from Databricks. Set up relationships and measures for rich, interactive reports. 🔶 Why Integrate Databricks Delta with Power BI? - Unified Analytics: Combining Delta Lake’s big data architecture with Power BI’s visuals provides scalable insights. - Real-Time Data: Delta Lake’s transaction logs enable real-time data analytics, ensuring your Power BI reports are always up-to-date. - Data Integrity: Delta Lake’s ACID transactions maintain accuracy and reliability in your data pipelines, reducing reporting errors. 🔍 Best Practices: - Partition Data: Use data partitioning in Delta Lake to improve query performance. - DirectQuery: Use DirectQuery in Power BI to minimize data duplication and fetch fresh data from Databricks. - Time Travel: Leverage Delta Time Travel to access historical data for comparative analysis in Power BI. 🔗 To learn more, check out the official guide on integrating Power BI with Databricks: [Databricks Documentation](https://lnkd.in/dtWXXSPE) Bringing Databricks Delta tables into Power BI empowers data professionals with **fast, reliable, and scalable** analytics. Whether tracking key metrics or performing real-time monitoring, this powerful integration takes your analytics to new heights! 💡 #Databricks #PowerBI #DataEngineering #DeltaLake #Analytics #DataVisualization #BusinessIntelligence #BigData

🚀 Exciting news for data enthusiasts! This innovative connector seamlessly integrates Tableau with Databricks Delta Sharing, revolutionising data access and collaboration. Don't miss out on this transformative solution reshaping the future of data analytics! 💡📊 #DataAnalytics #Tableau #Databricks #Innovation

Yeah, what Nicholas Mann said! In all seriousness, for those organizations that are committed to maximizing their investment in #snowflake and accelerating their business, this is the most powerful trio available. #partnershipsmatter

When you share a dataset with Delta Sharing, Tableau users can now directly connect to it! That's the benefit of an open protocol for data sharing, which Delta Sharing uniquely provides. Don't build your collaboration architecture on something closed. Databricks & Tableau 🤝

Lakeview Dashboards are now GA, and I’ve been diving into them the past few weeks. It’s a slightly different paradigm but offers an immensely valuable framework for building BI reports directly on Databricks. Check out this weeks blog all about the best practices and pro tips and tricks for building performant, highly functional, and beautiful Lakeview Dashboards! We cover all these best practices with an example we can all use - analyzing usage on system tables. This blog has lots of general gold nuggets and also some great out of the box design patterns everyone can use! Post: https://lnkd.in/dh_MuNb8 #BI #Databricks #SQL

A lot of my customers ask for two things when they are working with or evaluating Databricks: - how to easily report on their data - how to do it cost effectively With the recent GA of Lakeview Dashboards, Cody Austin Davis wrote up a great blog that answers both of those questions. *Added bonus - our AI assistant can do a lot of the heavy lifting for you. 💪🏼 Check it out! #databricks #ai