How credible are Election Opinion Polls ??

Most exit polls in the aftermath of the Bihar results ended up looking an exercise in high-school sampling. What surprises me is that the exit polls from 6 different sources after the Bihar elections actually looked like a mirror image of each other with some minor numerical adjustments. The bigger surprise was that each of these polls were substantially off the mark. I am not even talking about the Chanakya poll, where they probably tried to test their dame luck too far; but the others who have a decent track record. 

It is ironical that the only agency that actually got its prediction for Bihar right was so unconvinced about its own methodology that it decided not to air its poll results. So finally, the people with the understanding did not have the conviction and people with little understanding had too much conviction. What surprises me is that in the central elections, Delhi elections and Bihar elections the media exit poll estimates have been miles away from the real picture. What could be the reasons for this sudden deterioration in exit poll quality.

1. Missing the bigger picture seems to be the key. In any election that  showed a distinctly substantial shift in trend, the exit polls seem to have missed out the big picture. Take the case of Delhi. Nobody even got anywhere close to the kind of clear sweep that the AAP managed. Union elections were no different. While polls were talking about the NDA scraping towards the halfway mark, the BJP on its own reached a comfortable majority. Bihar was the worst of the lot. It looked less like an exit poll and more like a broker consensus estimates, which is typically a consensus of wrong estimates. Not a single worthwhile insight came from any of the exit polls in Bihar at a time when any person with bare minimum political understanding would have told you that Narendra Modi will never be able to engineer the 5% vote shift required to win Bihar from Nitish.

2. Making a hash of the sample. This is the most common problem in any statistical sample. You either do not take an adequate sample size or do not intelligently stratify the sample or you ask daft questions or most likely avoid asking the key questions. In the case of Bihar exit polls, it seems to be a combination of all the above factors. A good size sample with a proper stratification would have enabled you to get critical insights like the RJD outperformance, Congress performance etc..

3. Lack of quality psephologists. A senior journalist once told me that the best of psephologists have either taken up politics or business as a full time profession. Hence the new lot of psephologists either do not have the perspective or the ability to understand the finer aspects of politics in a complex state like Bihar. That is the key reason why polls tend to get their view correct in simpler states like Madhya Pradesha and Rajasthan but make a mess when it comes to more complex states like Bihar and Uttar Pradesh.

4. Indian media is getting aggressive on monetizing news. Frankly, I would like to believe otherwise, but that almost seems to be the obvious conclusion. A few sample cases. On the counting day in Bihar, the NDA was leading for the first 2 hours. I have never seen this kind of a wrong trend in the first two hours especially when the final result was the exact opposite. That is impossible unless the media was plain incompetent or had a hidden agenda. How could 6 polls come up with similar prediction with marginal differences when the truth was miles away. It is time for the media to introspect.

The Bihar elections has brought the Indian media's credibility to an all time low. It is now being billed as insouciant, incompetent and corrupt bunch of people. The most worrying feedback for the Indian media came from a voter last year. I walked across to a young voter who had just been interviewed by a media channel and asked him what he had told the interviewer. His answer said it all. "Kya Fark padta saab. Yeh saare channel wale bikau hai". Probably, the biggest reason the media is getting wrong outcomes is that the people do not believe in the integrity of these exit polls any longer and hence do not care. That shows in the quality of output. This sense of disillusionment about the media among the Indian population is something the Indian media must really worry about...