Quota-controlled Surveys: an Empirical Approach to their Total Error
Article Sidebar
Main Article Content
Abstract
The majority of survey data available for public opinion research and electoral forecasting through representative samples, in Spain and Latin America, has been obtained with the final selection of interviewees through quotas, a non-probabilistic method. The reliability of the estimates made from this data is poorly understood and has little theoretical support: we know quite a bit about its biases, but almost nothing about its variance and "margin of error." This research empirically studies the variability of the samples obtained in the periodic barometers of the Center for Sociological Research of Spain (CIS), taking advantage of the length and regularity of some of its series estimating stable parameters. We find that the growth of variance is moderate and is within what is expected in a complex design.
Article Details
Citas en Dimensions Service
References
Referencias
Abundis, F., Becerra, L., Berrueto, F., Berumen, E.F., De La Rosa, M., Durand, C., Flores, C., Juárez, J., Moreno, A., y Penagos, D. (2017). La precisión de las encuestas electorales: un paradigma en movimiento. Instituto Nacional Electoral. https://www.ine.mx/wp-content/uploads/2019/11/la_precision_de_las_encuestas_Vol-II.pdf
Baker, R. J., Brick, M. Bates, N.A., Battaglia, M., Couper, M.P., Dever, J.A., Gile, K.J. y Tourangeau, R. (2013). Summary report of the AAPOR task force on non-probability sampling. Journal of Survey Statistics and Methodology, 1(2), 90–143. https://doi.org/10.1093/jssam/smt008
Berinsky, A J. (2006). American Public Opinion in the 1930s and 1940s.The Analysis of Quota-Controlled Sample Survey Data. Public Opinion Quarterly, 70(4), 499–529. https://doi.org/10.1093/poq/nfl021
Cantril, H. (1948). Opinion Trends in World War II: Some Guides to Interpretation. Public Opinion Quarterly, 12(1), 30–44. https://doi.org/10.1086/265918
Curtice, J. y Sparrow, N. (1997). How Accurate are Traditional Quota Opinion Polls. International Journal of Market Research, 39(3), 1-14. https://doi.org/10.1177/147078539703900302
Davis, J. A., y Smith, T. W. (1992). The NORC General Social Survey: A user's guide. Sage Publications, Inc. https://doi.org/10.4135/9781483345246
Deville, J.C. (1991). A Theory of Quota Surveys. Survey Methodology, 17(2), 163-181.
Díaz De Rada, V. y Martínez Martín, V. (2014). Random Route and Quota Sampling: Do They Offer Any Advantage over Probably Sampling Methods?. Open Journal of Statistics, 4, 391-401. http://doi.org/10.4236/ojs.2014.45038
Durand, C., Johnson, T., Moreno, A. y Traugott, M. (2018). Report of the WAPOR Committee Reviewing the Pre‐election Polls in the 2017 Presidential Election in Chile. World Association for Public Opinion Research (WAPOR). https://wapor.org/wp-content/uploads/Final-WAPOR-Report-onChile-2017-Election.pdf
Frankel, M.R. y Frankel, L.R. (1987). Fifty Years of Survey Sampling in the United States. The Public Opinion Quarterly, 51(2) Suppl., S127-S138. https://doi.org/10.1093/poq/51.4_PART_2.S127
Ganninger, M. (2006). Estimation of Design Effects for ESS Round II. European Social Survey Documentation. https://www.europeansocialsurvey.org/docs/round2/methods/ESS2_estimation_of_design_effects.pdf
Gaziano, C. (2005). Comparative Analysis of Within-Household Respondent Selection Techniques. Public Opinion Quarterly, 69(1), 124-157. https://doi.org/10.1093/poq/nfi006
Gschwend, T. (2005). Analyzing Quota Sample Data and the Peer-Review Process. French Politics, 3, 88-91. https://doi.org/10.1057/palgrave.fp.8200068
Kish, L. (1998). Quota sampling: Old Plus New Thought. Web Survey Methodology. http://www.websm.org/uploadi/editor/doc/1458891348Kish_1988_Quota_Sampling.pdf
Kish, L. (1965). Survey Sampling. John Wiley.
Lohr, S.L. (2010). Sampling: Design and Analysis. Brooks/Cole
Lynn, P. y Jowell, R. (1996). How Might Opinion Polls be Improved? The Case for Probability Sampling. Journal of the Royal Statistical Society, 159(1), 21-28. https://doi.org/10.2307/2983465
Martínez, V. (1999). Diseño de encuestas de opinión: Barómetro CIS. QÜESTIÍO, 23(2), 343-362. http://hdl.handle.net/2099/4111
Mateos, A. y Penadés, A. (2013). Las encuestas electorales en la prensa escrita (2008-2011). Errores, sesgos y transparencia. Metodología de Encuestas, 15, 99-119. http://casus.usal.es/pkp/index.php/MdE/article/view/1039
Moreno, A., Aguilar, R., y Romero, V. (2014). Estimaciones de encuestas preelectorales en México: en busca de las principales fuentes de error. Revista Latinoamericana de Opinión Pública, (4), 44-93. http://hdl.handle.net/10366/142669
Moser, C. y Stuart, A. (1953). An experimental study of quota sampling. Journal of the Royal Statistical Society, series A 116, (4), 349-405. https://doi.org/10.2307/2343021
Page, B.I. y Shapiro, R.Y. (1992). The Rational Public: Fifty Years of Trends in Americans’ Policy Preferences. University of Chicago Press.
Shirani-Mehr, H., Rothschild, D., Goel, S. y Gelman, A. (2018). Disentangling Bias and Variance in Election Polls. Journal of the American Statistical Association, 113(522), 607-614. https://doi.org/10.1080/01621459.2018.1448823
Stephan, F. F. y Mccarthy, P.J. (1958). Sampling Opinions. An Analysis of Survey Procedure. John Wiley.
Sturgis, P., Baker,N., Callegaro, M., Fisher, S., Green, J., Jennings, W., Kuha, W., Lauderdale, B. y Smith, P. (2016). Report of the Inquiry into the 2015 British general election opinion polls. Market Research Society and British Polling Council. https://eprints.ncrm.ac.uk/id/eprint/3789/
Sturgis, P., Kuha, J., Baker,N., Callegaro, M., Fisher, S., Green, J. Jennings, W., Lauderdale, B.E. y Smith, P. (2018). An assessment of the causes of the errors in the 2015 UK general election opinion polls. Journal of the Royal Statistical Society, series A 181, (3), 757-781. https://doi.org/10.1111/rssa.12329
Sudman, S. (1966). Probability Sampling with Quotas. Journal of the American Statistical Association, 61(315), 749-771. https://doi.org/10.1080/01621459.1966.10480903
Selb, P. Chen, S., Körtner, J. & Bosch, P. (2023) Bias and Variance in Multiparty Election Polls, Public Opinion Quarterly 87 (4): 1025–1037, https://doi.org/10.1093/poq/nfad046
Stephenson, C. B. (1979). Probability Sampling With Quotas: An Experiment. The Public Opinion Quarterly, 43(4), 477–496. http://www.jstor.org/stable/2748547
Taylor, H. (1995). Horses for Courses: How Survey Firms in Different Countries Measure Public Opinion with Very Different Methods. International Journal of Market Research, 37(3), 1-9. https://doi.org/10.1177/14707853950370030
Taylor, H. (1997). The Very Different Methods Used to Conduct Telephone Surveys of the Public. International Journal of Market Research, 39(3), 1-13. https://doi.org/10.1177/14707853970390030
Revista Mexicana de Opinión Pública por Universidad Nacional Autónoma de México se distribuye bajo una Licencia Creative Commons Atribución-NoComercial-SinDerivar 4.0 Internacional.
Basada en una obra en http://revistas.unam.mx/index.php/rmop.