Mark A.’s Post

You know a Semantic Layer would be hugely valuable, but how do you actually build such a thing? Pro tip: Crafting a Semantic Layer is about building iterative velocity alongside accuracy, so that when your stakeholders ask about Revenue MoM grouped by Attribution Channel, you can answer instead of adding a ticket to the backlog. Start with these four steps: 1. Identify a Data Product that is impactful: Find something that is in heavy use and high value, but fairly narrow scope. Don’t start with a broad executive dashboard that shows metrics from across the company because you’re looking to optimize for migrating the smallest amount of modeling for the highest amount of impact that you can. For example, a good starting place would be a dashboard focused on Customer Acquisition Cost (CAC) that relies on a narrow set of metrics and underlying tables that are nonetheless critical for your company. 2. Catalog the models and their columns that service the Data Product, both in dbt and the BI tool, including rollups, metrics tables, and marts that support those. Pay special attention to aggregations as these will constitute metrics. 3. Melt the frozen rollups in your dbt project, as well as variations modeled in your BI tool, into Semantic Layer code. 4. Create a parallel version of your data product that points to Semantic Layer artifacts, audit, and then publish. Creating in parallel takes the pressure off, allowing you to fix any issues and publish gracefully. You’ll keep the existing Data Product as-is while swapping the clone to be supplied with data from the Semantic Layer. Dig deeper into the step-by-step process of how to ship a Semantic Layer in pieces at our link in the comments.

🚀 Unlock the power of your data with Sigma’s new Data Models! Now, you can build scalable, reusable datasets that make analytics faster and more accessible for everyone—no SQL required. 💡

🚀 Unlock the power of your data with Sigma’s new Data Models! Now, you can build scalable, reusable datasets that make analytics faster and more accessible for everyone—no SQL required. 💡 Say goodbye to bottlenecks and hello to real-time insights. Discover how Sigma is simplifying data modeling and helping teams collaborate like never before. Dive into the details here

🚀 Unlock the power of your data with Sigma’s new Data Models! Now, you can build scalable, reusable datasets that make analytics faster and more accessible for everyone—no SQL required. 💡 Say goodbye to bottlenecks and hello to real-time insights. Discover how Sigma is simplifying data modeling and helping teams collaborate like never before. Dive into the details here

The modern data stack problem 3/3: Scale Over the past week, I've highlighted two of the most pressing data challenges faced by scale-ups and large enterprises: skills and speed. These issues converge on a third, crucial challenge. As your data operation scales, untangling analytics engineering bottlenecks becomes a priority. ↓ You often skill analysts and hire analytics engineers (♡) to maintain consistent data models and follow proper engineering workflows. ↓ But as your company grows, so does the number of analytics engineers, and consequently, the costs. ↓ Eventually, the migration project to dbt will conclude. But business needs will continuously require data model changes, and at the same time, require analysts to return to their core expertise: analyzing data. ↓ And whenever you let go from your strict procedures, you find the same issues with duplicates, inconsistencies, and silos reemerging, now in your dbt models. Here's a surprising fact: → Nearly every large company I speak with ends up shifting dbt responsibilities back to engineers, away from analysts! *** As we bring dbt into our daily practices, we find that inconsistencies and conflicts remain an issue, and sometimes might become exacerbated by the additional workflow to keep dbt and the BI tools in sync. We have the engineering tools and solid understanding for data modeling within dbt and syncing up with BI tools like Looker, Tableau, or Power BI. But a reverse flow -- capturing the inception of core business logic crafted by analysts and shifting it into dbt -- remains a territory awaiting innovation. *** Thanks for following up on this mini-series on the modern data stack. Check out this past week's posts in the comments. What do you think? P.S. Euno expands dbt's governance capabilities into the BI layer, enabling analytics engineers to govern business logic proactively and retroactively without slowing down business analysts. DM me if this resonates with you.

🚀 Unlock the power of your data with Sigma’s new Data Models! Now, you can build scalable, reusable datasets that make analytics faster and more accessible for everyone—no SQL required. 💡 Say goodbye to bottlenecks and hello to real-time insights. Discover how Sigma is simplifying data modeling and helping teams collaborate like never before. Dive into the details here

🚀 Unlock the power of your data with Sigma’s new Data Models! Now, you can build scalable, reusable datasets that make analytics faster and more accessible for everyone—no SQL required. 💡 Say goodbye to bottlenecks and hello to real-time insights. Discover how Sigma is simplifying data modeling and helping teams collaborate like never before. Dive into the details here

Sigma has always had reusable data sets and governed content publication, but its great to see us moving deeper and deeper into providing our customers with options for data modeling. We can integrate with dbt or cube if you want to do the work there -- or use our data modeling layer to pair that logic with your BI layer. Agility requires flexibility and Sigma is accelerating teams faster than any other platform.

You certainly don’t need a metrics or semantic layer to generate dashboards or execute a data analysis. However, adopting a "metrics-as-code" framework has the potential to revolutionize data production and consumption workflows. Let’s delve into the production side of the equation. Even if your use case involves producing basic metric calculations and dashboards, transitioning from SQL scripts to a well-defined metrics layer significantly amplifies the day-to-day capabilities of a data or BI team. With this transition, you can now: 1) retrieve metric variants effortlessly that every organization has to build manually - examples include metrics by flexible time windows or calculations such as cumulative, to-date, and period-over-period. 2) access the power of the “metrics cube” which encompasses the full combinatorial space of metrics and their associated dimensional attribute cuts. Building reports requiring overlapping slices of cubes becomes far less painstaking compared to writing numerous queries and ensuring consistent business logic. 3) reduce maintenance costs - a well constructed metrics architecture should allow you to modify metrics logic, add or edit attributes such that downstream datasets and dashboards update seamlessly. On the consumption side, a metrics back-end can enable an entirely new operating system and way of working for the organization. By crafting metric trees from well-defined metric objects, entire business processes can now be represented backed by rich datasets. After all, businesses are intricately connected input-output metric systems. Consequently: 1) The C-suite, business domains, and data teams can transparently align on the controllable inputs and resultant outputs, as well as the initiatives and experiments needed to achieve desired metric outcomes. 2) Algorithms can leverage the metric tree structure in combination with the rich collection of datasets to streamline or even automate business reviews, metric deep dives, pre-post, or A/B experimentation analysis. 3) Delightful no-code user experiences can be developed for operators who were previously heavily reliant on data teams and their mastery of SQL and data models. Productivity and empowerment gains often stem from introducing new abstractions. Transitioning to a "metrics-first" world where these objects can be captured in code, composed flexibly, and operated upon represents the next logical evolution on top of data platforms.