Is your data cheating on you ?

At least a few of you would have seen this headline recently – iPhone sales bring down chewing gum sales.

Source : www.recode.net

They had a reason for it – The consumers who were used to picking up chewing gums while waiting at the check out lines now are spending more time looking at the iPhones.

For a minute everyone who is even remotely connected with data and numbers would have nodded. What an amazing insight !

Pause for a moment. Just because the numbers correlate, can we say that one is the cause of the other? 

 Has the data cheated on you ?

 Just because chewing gum sales fell in the same way as iPhone sales picked up can we say one caused the other?

Look at some other cases - The case of autism and the presence of a chemical called glyphosate (agent that is used to protect GMO plants).

There is substantial real estate on the internet dissecting this claim. Some interesting graphs draw near perfect correlation between number of Jim Carrey movies and prevalence of autism cases.

If you scour the Internet, you will get more of these. Look at some more interesting ones – US Spending on Science, Space and Technology correlates with Suicides with strangulation, hanging and suffocation to Number of people who drowned by falling into a pool correlates with films Nicolas Cage appeared.

 Both are classic cases of correlation but not necessarily causation.

 However, just because the numbers showed a relation, the graphs looked like overlapping you cannot attribute them to each other.

 If you torture data enough, it will agree to anything. In the above cases it not much of a torture, but you are cheating on data more than data is cheating on you?

 What can you do to not to get into this trap while looking at your data – be it your revenue, marketing, customer behavior, trends and others?

Correlation does not imply causation.

 What can we do to eliminate these correlation causation issues while analyzing data ?

 1.    Be unbiased – At least be aware of your biases. Are you against GMO and seriously feel to contribute towards autism. Nothing wrong. But does that cloud your hypothesis, your analysis ?

2.    Be your devil’s advocate – Well almost. If you have a causal story – chewing gum sales have come down with iPhone sales – try to see if it holds good in all circumstances ? is that the only one that holds good ? Don’t just look at it to substantiate your story.

3.    Normalize the data – Have you normalized the data ? say for population growth ? inflation? Currency variations ?

4.    Let data speak - As some experts say, don’t torture them to speak what you want to hear, let them speak not under duress.

 With data being called the new oil we are bound to do lots of analysis. Be kind enough not to torture data, watch for the correlation – causal trap, then we’ll have less cheating and betrayals :)

 Share your interesting stories of cheating on data or that other way around.

 Or to further explore read this click bait Sex Makes You Rich :) (Not joking, there was a study that correlated sex and money)

 ___________________________________________________

Read my other articles here

Stay connected