Today, Maurice de Hond (the most famous Dutch ‘pollster’) said that as of now the Freedom Party (PVV) of Geert Wilders is the biggest party in the Netherlands. Question: How can a party have become smaller or larger, without any kind of formal elections? Answer: Because a tiny fraction of the population has said so.
I am surprised how much attention polls get in media, in many countries. Every week, a new poll is published and treated as some kind of reliable, trustworthy, perhaps even (quasi-)scientic source of information. A certain politician may have made a public appearance, or made a mistake, or whatever, after which the next polls are used to see what the consequences have been by comparing them to earlier polling results.
I love statistics and how they can help us to make sense of the world. But I am sceptical of how much it can tell us in this matter, for a variety of reasons. I don’t think that polling institutions necessarily do a bad job; perhaps they do their job as good as they possibly can. The problem is that inherent difficulties exist that can only with qualitative judgments be “fixed”.
Laudable, Maurice de Hond himself mentions a lot of limitations to his polls. However, he mentions the limitations that he is able to address. He leaves out some limitations that pertain to the crucial question: How confident are we that these polling results would indeed be similar to results of actual (hypothetical) elections, if they were organised today? I will mention three of those difficulties, which are most easily explained, but there are more.
Polls collect their data by asking people for which party they would vote. This can be done in person, via telephone calls, or, which is more and more often the case, via the Internet. The first question may be: Are these panels, these subgroups of society, representative of the entire electorate? A first problem is that it is quite likely that the propensity to accept to be a member of a panel correlates with political choices. I am sorry to generalise, and bear with me for the sake of the argument (the argument would only collapse if voters are on average exactly equal in many respects for every party, which is very obviously not the case), but I believe that PVV-voters are, on average, less educated and less interested in the day-to-day political debates, which means that they are less likely to be part of the poll.
Then, people may lie, which causes biases if voters from a certain party are more likely to lie than others. It is of course perfectly understandable that people who vote PVV may be reluctant to disclose this. This would underestimate the number of votes the PVV would get, if voters would choose the PVV more easily in the anonimity of the ballot box. In general, all kinds of subtle factors may cause that voting behaviour via Internet or phone calls differs from what happens in the ballot box.
Also, polls correct their results for all kinds of characteristics, such as gender, ethnicity, education, and so on, so that their panel is representative of society. But what they do not take into account is the elections turnout. This is important because the likelihood that one will cast a vote correlates with personal characteristics, such as age, education, or job status. So, if the turnout is relatively high, this means that relatively many people who are young, have a lower education, and are unemployed will cast a vote.
The pollsters recognise such shortcomings, of course. They even address them. But this is where, by necessity, qualitative judgments and corrections come into play. For obvious reasons, pollsters have not disclosed in details how they correct them, but I don’t believe that there is any objective way to do so.
Perhaps the most repulsive element of polling models, or rather of the media that use the results and invite people like Maurice de Hond to their talkshow, is that they get away with it when they are wrong. There is hardly a way in which they can be, using Popperian terminology, falsified. Why? Because even for an amateur it would be difficult to be extremely wrong; there are only 150 seats to be distributed, and much patterns have remained quite stable over time. Take a look at history, use some Fingerspitzengefühl, and there you go. In the case you are wrong, some standard excuses can be provided. Pollsters were really-really-really wrong, but see here, here, here, here, and here: all over the frontpages again, as if nothing happened.
So why is this so crucially important? First of all because people are deceived. They are being told a story, which is presented as if it were factual, but which is anything but factual. And secondly, they influence election results. They can do so in an infinite amount of subtle and unobservable ways.
But to see how, we can just take a look at the last elections in the Netherlands. Most polls indicated that it would be a close call between the VVD and the PvdA. This was a clear situation of the Right versus the Left, which we had not seen for a very long time. Instead of voting on the person or the party that best represented their views and interests, many people therefore engaged in strategic voting. Voters suspected that their vote on, for instance, D66 would not matter so much, since it would be about and between the VVD and the PvdA. If they strongly disliked either one of these parties (or either of the ideological extremes), they may have considered giving their vote to the other party, instead of to D66 (or any of the other parties).
It is my suspicion that this is largely responsible for the huge electoral success of the VVD and the PvdA. The VVD and the PvdA got way more seats than was predicted, ironically likely because of the influence earlier polling results exerted themselves! Polls thus gravely influence the very phenomenon they try to reseach, which sounds like an immense and toxic case of Heisenberg’s uncertainty principle to me.
It is even more ironic that of all potential sources of information, Maurice de Hond himself supplies some quantitative evidence. De Hond stated that 17% voted strategically, and that without strategic voting the VVD would have won only 34 seats and the PvdA 26 seats – a total of 60 seats, not nearly enough to form a two-party government as they do now. It is very plausible that the current government is in power largely because of previous polling results.
De Hond, after the elections, indicated that he was aware of this phenomenon. Moreover, he indicates that strategic voting (as I indicated above) cannot be caught by his models – and therefore his models “were not wrong”. So, his models induced strategic voting, which caused that his models gave wrong predictions – but the models still weren’t wrong. Do you still get it? How can this not be an extremely malicious string of events?
Disregarding issues of manipulation (if polling results are so important, it is not difficult to see that a lot of very powerful people get an interest in tweaking the qualitative judgments made within polling models), I’d like to suggest two things.
(1) Please, media, take polling results as what they are: some fancy but phony statistics, at best.
(2) Consider observing a moment of election silence, and prohibit polls in the last days before the actual election.
Update 14 December 2012: See here, the polling hypocrisy strikes again. Five days ago it was frontpage news that the PVV was the ‘biggest’ party. Well, things have changed: the PvdA is the biggest now, with 31 seats, whereas the PVV only has 20 (!). Utter nonsense.