What's your method for measuring data quality?

Actually, I would suggest that even before selecting data quality dimensions, the first step is to solicit input from the data consumers to understand their concerns about the usability of the data. That provides context as to both the expectations associated with dimensions and corresponding thresholds for acceptability. For example, the number of customer records that are missing telephone numbers is only relevant to data consumers (e.g., outbound sales representatives) wishing to use telephone numbers (e.g., to make sales calls). Having a measurable dimension is meaningless if that measure is irrelevant. Measuring a wooden beam's length won't tell me that its weight tolerance is insufficient to bear the second floor's load.

As you are capturing these dimensions for your data - a key set of meta data need to tagged across these datasets. Things like : a) Lineage b) Business Context(transactions) under which they were captured c) Who captures them (user entered or call Center agent) When you start capturing the gaps in data quality you would exactly know what needs to be fixed. You would suddenly identify data quality issue patterns against users / business transactions which needs to be fixed. In any organisation of considerable size there is too much data everywhere and without context there is a chance you would end up chasing your tail trying to fix bulk loads of data without being able to show (targeted)benefits out of it.

The basic thing is to share the definitions of data quality dimensions and creating awareness across the organization. Explain with examples would be a great buy in.

In addition, interoperability and reusability are core aspects of data quality to be measured. Without these two essential components of DQ, usability of data suffers. If data cannot be managed and accessed across a system, the value of data suffers. Data silos will continue to persist creating disjointed and isolated data assets.

One thing that is often overlooked is that not all DQ dimensions are created equal, and further, not all are always required. The dimensions listed are available to use and rules can be defined and classified accordingly, but a balance needs to be achieved in the application of multiple rules to single datasets and/or elements. Accuracy is the most difficult to achieve and a General Ledger reconciliation is often looked to as an example, but not everything needs an accuracy rule when a completeness or consistency will suffice. A big part is knowing how the data will be used, by whom, for what purpose. Then you can define the right rule(s) and establish the right thresholds as mentioned below.

When establishing data quality metrics, you have to decide what you're going to do if you find you don't have 100% quality. Errors can happen and we don't always have perfection, but you have to understand what the impact is when you don't have 100%. If your compliance rate is 95%, then 5% is non-compliant. Is that acceptable? Is it 5% non-compliant to a subset of employees, and that information can be clearly communicated to them so they don't use that data? It's important to remember that a KPI is just a number. You still need to understand the implications of the data behind that number.

Run profiling periodically and establish results (or calculated measures) in reports, would definitely help quantifying the data quality issues.

KPIs are for organisations who are quite matured in their DG journeys and honestly how many of them really are ? For organisations who are looking to newly start quantifying their data quality, you need to start by finding issues and figuring out cost of those issues and how fixing these issues would result in savings or create new opportunities or deliver better insights - once you can prove the ROI - Organisations would get the value of these KPIs and provide adequate funding for it.

A mistake that I have observed is making the rule results binary by establishing a 100%/0% (aka: all or nothing) defect tolerance. This is a recipe for an influx of data defects that could quickly overwhelm the organization and mislead to conclusions of having a horrible data environment. In reality, data might not be perfect every time and missing a single field out of 300 million may be within the risk tolerance for the particular use of the data. In a regulatory reporting environment, the need for the highest quality data is paramount - whereas if it is being used for an internal management status report, different DQ rules and different results may be acceptable. This can further be rolled-up into overall scoring at various levels.

The proposed vision is Ok, but it's way too technical. If you want to win the attention of your business: 1. keep an eye on zero-tolerance errors (there are very few for sure) 2. measure all others in "potential risks", i.e. money to loose (dollars, euros, pounds). If you cannot - probably you don't need any fancy KPIs - a quantity of errors detected may be sufficient. To have the correction process KPIs it's usually sufficient to measure the average time needed to make a correction or a number of corrections made by day.

In addition, tracking data integrity issues as a core principle in data quality is important. Asking questions such as: How much does the data consistency, accuracy, and reliability remain within the entire data life-cycle? What does the data quality data tell us about the integrity of the data used in decision-making within the organization?

Profiling results can get stored in database which will enable for analysis. A report may also be designed to show the ups and downs happening on periodical results.

Surfacing data quality metrics as scorecards or dashboards for all teams to access creates a transparent feedback loops. When common metrics and expectations are communicated, a data quality initiative can be strengthened from the ground up.

One of the challenges I have faced is business involvement in defining the data quality rules. Please ensure architects and tech leads don’t define these rules. They are for the business and should be defined by the business.

A short addition : 1. DQ is frequently a by-product of BI ot other data consolidation initiatives. Don't do DQ separately if possible. 2. Nobody is talking about the organisation of data correction process. It is way more important than the dashboards-KPIs. The need is not to report, but to inform and apply the corrections.

I believe this is where the data stewardship process comes into play. That should be the part of improvement plan. A possible prioritization may help narrow down the focus.

With scorecards and dashboards, data quality improvement actions can also be clearly communicated. To achieve acceptable data quality, communication with clear expectations is necessary.

In my experience, Attaching Data Quality KPIs to KRA of senior executive helps to drive Data Quality and ensures that operating staff also participate in improving Data quality.

Take proactive approach by training the HR on data requirements and making few fields compulsory on data input tool to improve data quality.

Expect new data quality issues will arise. Support teams by updating communication issues, flagging data quality concerns with direct action items. Data observability functions can help surface data quality issues to alert stakeholders.

Communicating data quality needs to be tailored to the audience. Some people want a high-level report so they have an idea what the data quality level is and that it's tracking in the right direction. Some people need to see specifics on what data has problems so they can avoid that data in their analysis. If there are problems with the data, people need to know what the problem is, how it will be solved, the timeline for resolution, and then additional communications with status updates or notification of the resolution.

Ensuring you have identified your stakeholders before hand is important - Remember every stakeholder will look at the stats differently ; it is important to interview each of them before you start to understand what does good data quality mean to them. This will allow you to customise and cater to their needs.

Why not let the tools do it for you? Informatica has a SaaS Data Quality tool that provides out of the box KPIs via a dashboard. It’s a beautiful thing. In short, don’t overthink it. If tech is the answer - use tech!