Without going into the smallest details of the measurements of the past decade – quite a few very high-quality studies have been published on the subject – we will now only try to shed a different light on the results of a symbolic research. In the case of Brexit – but to a lesser extent the 2020 US presidential election – the press did not clarify the likelihood of the possible scenarios.
After reviewing the confidence interval of the research (this is a probability interval, because if we estimate from a sample, we never know the exact value, the result will be somewhere between an upper and a lower value), the reader can soon be convinced that the winner could not be clearly estimated. In other words, the press presented the given candidate/outcome as a clear winner instead of the uncertainty correctly suggested by the majority of research firms. In this case, however, we cannot speak of a measurement error, as the deviation was within the error limit. The “failure” was much more a product of the press, which was magnified by the prominent political importance of the vote. In connection with the Budapest mayoral election in 2024, it was not possible to predict the victory of any of the candidates, since the competition was so close. In the end, the result was less than half a hundred votes, in such and similar situations it is unrealistic to expect polling companies to accurately estimate the winner. Of course, methodological errors can also occur – especially during the 2020 US presidential election – so, for example, inappropriate weighting and the underrepresentation of certain groups (we will write about these in more detail later) can also contribute to the error.
Many factors must be taken into account when interpreting survey data. Perhaps the most important thing is that we are not talking about “prediction”, but about an estimate fixed at a given time, trying to reflect the current climate of opinion, working with a sample representing the entire population. If a survey is conducted 30 days before a “current” election, it cannot result in a meaningful projection of the final result, but it is suitable for interpreting the current political situation. A perfect example of this is the result of the Budapest mayoral election in 2019, because, contrary to previous expectations, Gergely Karácsony defeated István Tarlós. In this case, the scandal known as the Borkai affair can be held responsible for the election fiasco. In addition, as mentioned earlier, the data has a margin of error and a reliability. In the event of an expected close result, it cannot be clearly stated who will be the winner.
Of course, there are indeed methodological difficulties that impair the reliability of research, in which case surveys that are very close to voting are equally distorted. Perhaps it is worth mentioning briefly here that the time of data collection and the accuracy of the estimate are not necessarily related to each other. Below, we present the measurement difficulties that are independent of the time of the research.
Methodological difficulties and limitations
According to Ádám Stefkovics, director of the Political Research Institute of the Századvég Foundation, one of the biggest problems of public opinion polls today is that the conditions of classic probability sampling are less and less met. The underrepresentation of certain social groups can lead to a distortion of the results of the survey, which is why it is extremely important to cover the target population as much as possible during sampling. The already mentioned US presidential election is a prime example of this, as people with lower education were underrepresented in the sample, and this led to inaccurate prognoses. An additional difficulty in relation to representativeness is that people who are actively involved in politics are more easily included in the sample, as they are more likely to respond to inquiries.
In the case of protest parties such as Mi Hazánk, this phenomenon can often lead to an underestimation of support, while for other parties with a more politically active voter base, it can lead to an overestimation.
In Hungary, address-based selection and so-called random-digit dialing (RDD) are the most common sampling methods. Due to the rapid rise in prices starting in the 2000s, the necessary data were no longer collected by interviewers. Data collection companies purchase marketing-based mobile phone databases, which then serve as a sampling frame. However, no data is available on the quality of the databases obtained in this way, so possible distortions cannot be filtered out. Although weighting can help, if you don’t have an approximate picture of the composition of the sampling frame, then such correction procedures are not able to guide the obtained data in the right direction either.
Another problem is the “noise” generated by surveys, because not only professional polling companies, but
the willingness to answer was inflated due to marketing research with, at best, only partial professional experience and tools.
Not to mention that free time has increased in value. In contrast to the end of the last century, people no longer experience it positively when they are asked for their opinion on the phone, rather it comes across as irritation.
In connection with the latter, the increasing refusal to answer can also be highlighted. There are many reasons for this, one of which is political rejection. This is due to the decrease in the public’s trust in pollsters. Many people – not without reason – associate certain institutes with certain parties, and this can even lead to knowingly false answers given during the survey, as the questioner may attempt to mislead the interviewer due to political motives.
The 2024 EP election and public opinion polls
Despite all the difficulties, the domestic polling companies did not perform badly at all in connection with the 2024 EP election. The announcement of partpreference.hu drew attention to the fact that during the last public measurements before voting, the polling institutes estimated the real results approximately correctly. As written: “Examining the >>most probable list result<< of each research, it can be concluded that almost all of them accurately predicted the support of the parties with the greatest support." If we take into account that the data were recorded at different times, and that some measurements were taken well before the elections in a politically very turbulent environment, the “accuracy” of the data is remarkable.
partpreferencia.hu also established a ranking among the institutes, for which the negative and positive difference between the measured and real support of each party was squared and then added up. The extent of the differences was then compared. The Center for Fundamental Rights – despite the fact that it worked with early data collection – took a very high place and won a bronze medal. With this, he rejected such left-wing institutes as Median, Republicon Institute or even Publicus.
The future of polling (?)
Although, as the 2024 EP election shows, it is too early to bury the domestic polling companies, it cannot be ignored that the profession is forced to respond to the challenges of the 21st century. The aforementioned Ádám Stefkovics, in his newly published volume, examined the methodological problems of public opinion polls. A book based on the results of the author’s PhD research (Where to next, Hungarian election research?) it not only provides a thorough overview of the domestic history, challenges and dilemmas of the profession, but also aims to contribute to the improvement of the measurement results with proposed solutions.
Stefkovics primarily recommends a wider application of the hybrid methodology, while emphasizing that in order to increase the willingness to answer, domestic researchers will be forced to provide various (even financial) incentives to the respondents. In his book, he also points out the role and responsibility of the press in connection with the restoration of trust in public opinion research, and also warns against the politicization of the profession, which also has a negative effect on trust in research.
The seriousness of the problem is shown by the words of Hann Endre, the head of Medián, who stated that before the 2022 parliamentary elections, several studies were prepared that were only meant to manipulate public opinion in the desired direction.
The author recommends closer cooperation between the scientific sphere and the public opinion polling profession to counter negative processes like this. In addition to his already presented reform proposals, Stefkovics also supports the use of data generated in the administrative and digital space, as a correction and addition to the primary data collection.
The volume provides insight into the past, present and, hopefully, future of election research in an exciting, accessible, yet extremely informative way, not only for domestic experts, but also for the wider strata interested in public opinion polls.
The authors of the article are employees of the Center for Fundamental Rights.
(Cover image: Columbán Kitti / Index)
#Index #Homeland #domestic #polling #profession #buried
2024-07-08 06:38:44