What is the best way to validate statistical results in your Data Visualization?

Validate statistical results in data visualization through methods like cross-validation, resampling, sensitivity analysis, expert review, replication, significance tests, outlier handling, comparative visualizations, and peer review for robust and reliable findings.

Statistical results in data visualizations? You need to make sure your analysis is accurate, but how do you data visualizations are not always about displaying statistical results. In fact, it's not common that statistical results are visualized. You can show a scatterplot of 2D data you clustered into two groups by using a side-effect color for each group but the result of a regression isn't something you plot. In statistical analysis plots are used for exploratory analysis and there are lots that are used as diagnostics, but usual they don't display the result of an analysis.

Before validating statistical results, it is best to have a thorough understanding of the matter, and having background knowledge, context and previous validated results (if available) helps. Ensure that the statistical data is reliable, relevant and sufficiently large enough before using them for analysis because inferior inputs often lead to undesirable outputs, lots of data cleaning, or even neccesitate a do-over after a lot of work has been done. For the multiple sources of data, it helps to use a mixture of primary and secondary sources of data. Since time is usually a limiting factor, it helps to mostly use data gathered by others, but the data should be assessed to find out how relevant they are to the current analysis.

Data visualization can help to reveal not only trends in the data, but also flag out issues with the data, especially data which come from relatively unknown sources or were used for other purposes. When doing exploratory data analysis (EDA), it is very useful to find out the shape and distribution of the data, usually by table form of the standard statistical methods like mean, median, range and so on. Also, currently available can use multiple charts to map up the shape of the data by different fields. Depending on the purpose of the analysis and how critical it is, validation can be one of the most important and time-consuming areas of the data analysis.

Errors and bias can be difficult to detect, and usually require a fair amount of checking, cross-referencing, experience etc. As there might be many data sources and it is very difficult to ensure all the data sources are complete, accurate, relevant etc., one should focus on the significance and possible impact, and prioritise the impactful ones to check deeply. Ensuring measurement and other systems such as databases and recording systems are accurate is critical, thus they should be tested routinely. If the underlying data and measurements are wrong and undetected, through scalability, it would cause immense issues for all downstream processes. Ad-hoc issues like human error can be managed via reconciliation, cross-referencing etc.

Feedback can be split into many categories, and for non-data feedback we can look at internal and external stakeholders. In this case, I define internal stakeholders as those who have a direct influence on the outcomes of the analysis, or the collaborators if you will. This refers to the project team members, reviewers, approvers etc. The purposes of getting feedback from this group is to seek improvement ideas, validation, support and buy-in so that the eventual analysis can be trusted. For external stakeholders, their feedback should be secondary but still necessary, especially stakeholders who will be affected. They can provide viewpoints and context which might or might not be relevant to the data analysis using the statistical data.

On paper, it is easy to mention various measures of validating statistical data, but in real-life it is often complex, time-consuming and impossible to validate completely. Oftentimes, validation of data should be based on preventative rather than reactionary measures, such as allowing only numbers for contact numbers for manual recording systems to prevent typos. To ensure completeness, mandatory fields should be set where appropriate. Statistical data should be handled differently according to the purposes needed for them, and any validation should take this into account. For example, a report might be used with minimal data cleaning and processing for one analysis but require 20 processing steps for another analysis to be useful.