European Commission uses this survey to monitor the evolution of public opinion in the Member States. This may help in making policy decision. The following question was asked in 2007:
Taking everything into consideration, would you say that the country has on balance benefited or not from being a member of the European Union?Eurobarometer 2007
It turned out that 69% of the respondents were of the opinion that the country had benefited from the EU. A similar question was included at the same time in the Dutch opinion poll Peil.nl (Maurice de Hond). However, the question was preceded by another question that asked respondents to select the most important disadvantages of being a member of the EU. Among the items in the list were the fast extension EU, the possibility of Turkey becoming a member state, the, introduction of the Euro, the waste of money by the European Commission, the loss of identity of the member states, the lack of democratic rights of citizens, veto rights of member states, and possible interference of the European Commission with national issues. As a result, only 43% of the respondents considered membership of the EU beneficial.
Most opinion polls in The Netherlands use online panels. Examples are Peil.nl (conducted by Maurice de Hond) and Politieke Barometer (conducted by Synovate). All these polls suffer from under-coverage: People without Internet access will never be able to participate in such a poll. If people with Internet differ from those without Internet, polls results will be biased.
According to Eurostat (2007) Internet access is high in The Netherlands. More than four out of five households have an Internet connection. Internet coverage is also high in the Scandinavian countries Sweden and Denmark. Coverage is very low in the Balkan countries Romania and Bulgaria. Only approximately one out of five households there has Internet access. Notwithstanding the high Internet penetration in The Netherlands, there are still groups where access is lower than average. This particularly relates to the low-educated, the elderly and non-natives. See Bethlehem (2008) for a more detailed analysis.
The principles of probability sampling are fundamental for modern survey research. The first ideas emerged a little more than a century ago. For an overview, see e.g. Bethlehem (2009). Reliable estimates can only be computed if a participants are selected by means of probability sampling. Furthermore, only then the accuracy of estimates can be computed.
Unfortunately, most only polls are not based on probability sampling. The survey questionnaire is simply put on the web. Participants are those people who happen to have Internet, visit the website and decide to participate. The poll organization is not in control over the selection process. Therefore, no unbiased estimates can be computed nor can the accuracy of estimates be determined. These polls are called self-selection polls.
Various market research organizations used self-selection opinion polls to predict the outcome of the general elections in The Netherlands in 2006. The results of the three major polls are summarized by Bethlehem (2008). For example, Peil.nl (Maurice de Hond) made a wrong prediction for the Labour Party and the right-wing populists (Wilders). And a survey carried out by Statistics Netherlands (based on probability sampling) produced correct predictions.
Probability sampling has the additional advantage that it provides protection against certain groups in the population attempting to manipulate the outcomes of the poll. Self-selection does not have this safeguard. An example of this effect could be observed in the election of the 2005 Book of the Year award (Dutch: NS Publieksprijs), a high-profile literary prize. The winning book was determined by means of a poll on a website. People could vote for one of the nominated books or mention another book of their choice. More than 90,000 people participated in the survey. The winner turned out to be the new interconfessional Bible translation launched by the Netherlands and Flanders Bible Societies. This book was not nominated, but nevertheless an overwhelming majority (72%) voted for it. This was due to a campaign launched by (among others) Bible societies, a Christian broadcaster and Christian newspaper. Although this was all completely within the rules of the contest, the group of voters could clearly not be considered representative for the Dutch population.
It should be stressed that increasing the sample size will not decrease the problems caused by under-coverage and self-selection. An example of a large online opinion poll in The Netherlands was 21minuten.nl, a survey supposed to supply answers to questions about important problems in Dutch society. Within a period of six weeks in 2006 about 170.000 people completed the online questionnaires. The news program NOVA of the Dutch public television station Nederland 3 presented the results of this poll and claimed them to be representative because of the large number of participants. This statement is simply untrue.
The American Association for Public Opinion Research (AAPOR) is the leading professional association for public opinion researchers. It has warned its members against the use of self-selection polls:
Only when a Web-based survey adheres to established principles of scientific data collection can it be characterized as representing the population from which the sample was drawn. But if it uses volunteer respondents, allows respondents to participate in the survey more than once, or excludes portions of the population from participation, it must be characterized as unscientific and is unrepresentative of any population.American Association for Public Opinion Research
A study across 19 online panels of Dutch market research organizations showed that most of them use self-selection, see Vonk et al. (2006). It became also clear that participants structurally differ on average from the Dutch population.
Poll organizations sometimes attempt to correct for the negative effects of under-coverage and self-selection by applying so-called weighting adjustment procedure. Participants are assigned weights. Those in under-represented groups get a larger weight than those in over-represented groups. Bethlehem (2008) shows that such a correction procedure is absolutely no guarantee that the poll results will be reliable.
In conclusion, one can say that many opinion polls are quick and dirty. They are quick because they are carried out by means of the Internet. And they are dirty because their outcomes are unreliable.