Web Survey Bibliography
Relevance & Research Question
The market research industry is wedded to quota controls. We apply Age and Gender quotas without a second thought as to why or indeed whether they are doing any good at all. Our argument is that, in the modern online sampling world, a different set of stratifications must be applied and our old assumptions simply do not apply. Why not? The answer, in common with so many of the problems in sampling in online research, lies in the frame. The frame in traditional research was close to the population; therefore a quota controlled random sample would tend to produce samples that, within the quota strata, also contained representative numbers of all other attitudes and behaviours. This is not the case with online access panels.
Methods and Data
Our experiment uses our US panel; the topic, eye colour, is unrelated to Age and Gender but is strongly related to Ethnicity. We have conducted 2 samples. The first strictly controlled on Age, Gender and Region, the second controlled on Ethnicity alone. Our Age Gender Region ‘nat rep’ sample should underestimate the number with brown eyes. The Ethnicity we expect to estimate eye colour extremely well. At the same time a third sample will be drawn which is simply “random enough”. Our expectation is that this sample will also under-perform on eye colour but will equal the findings from “nat rep” sample 1. A second experiment will be undertaken where the variable of interest is unrelated to anything –left- or right-handedness. Our hypothesis is that all three samples will perform equally well.
The results are precisely as predicted.
Researchers, particularly in the commercial world, apply quota controls to ensure “representivity” as a matter of practice, they do it because they have been told to, it is part of the folklore of market research. This is not sustainable in a world where we are no longer dealing with essentially incomplete frames. More science and less folklore needs to be applied to make the best of an increasing unscientific world.
Conference Homepage (abstract) / (presentation)
Web Survey Bibliography (6476)
- Solving the Unintentional Mobile Challenge; 2013; Peterson, G., Mechling, J., LaFrance, J., Ham, G.
- Mobile Research Risk: What Happens to Data Quality When Respondents Use a Mobile Device for a Survey...; 2013; Baker-Prewitt, J.
- Internet-Based Recruitment to a Depression Prevention Intervention: Lessons From the Mood Memos Study...; 2013; Morgan, A. J., Jorm, A. F., Mackinnon, A. J.
- Challenges for Researchers Investigating Contraceptive Use and Pregnancy Intentions of Young Women Living...; 2013; Herbert, D. L., Loxton, D., Bateson, D., Weisberg, E., Lucke, J. C.
- Computer science security research and human subjects: Emerging considerations for research ethics boards...; 2013; Buchanan, E. A., Aycock, J., Dexter, S., Dittrich, D., Hvizdak, E. E.
- A standard for test reliability in group research; 2013; Ellis, J. L.
- Using a web-based survey tool to undertake a Delphi study: Application for nurse education research; 2013; Gill, F. J., Leslie, G. D., Grech, C., Latour, J. M.
- The comparison of road safety survey answers between web-panel and face-to-face; Dutch results of SARTRE...; 2013; Goldenbeld, C., de Craen, S.
- Addressing Survey Nonresponse Issues: Implications for ATE Principal Investigators, Evaluators, and...; 2013; Welch, W. W., Barlau, A. N.
- Pros and cons of virtual interviewers – vote in the discussion about surveytainment; 2013; Póltorak, M., Kowalski, J.
- Addressing Disclosure Concerns and Analysis Demands in a Real-Time Online Analytic System; 2013; Krenzke, T., Gentleman, J. F., Li, J., Moriarity, C.
- The General Survey System Initiative at RTI International: An Integrated System for the Collection and...; 2013; Thalji, L., Mitchell, S., Hill, C. A., Suresh, R., Speizer, H., Pratt, D.
- Consolidation and Standardization of Survey Operations at a Decentralized Federal Statistical Agency; 2013; Nealon, J., Gleaton, E.
- Examination of the equivalence of self-report survey-based paper-and-pencil and internet data collection...; 2013; Weigold, A., Weigold, I. K., Russell, E. J.
- An Assessment of Incentive Versus Survey Length Trade-offs in a Web Survey of Radiologists; 2013; Ziegenfuss, J. Y., Niederhauser, B. D., Kallmes, D., Beebe, T. J.
- Clarifying Categorical Concepts in a Web Survey.; 2013; Redline, C. D.
- Using Online and Paper Surveys - The Effectiveness of Mixed-Mode Methodology for Populations Over 50; 2013; De Bernardo, D. H., Curtis, A.
- The fish model: What factors affect participants while filling in an online questionnaire?; 2013; Mohamed, B., Lorenz, A., Pscheida, D.
- Interview Duration in Web Surveys: Integrating Different Levels of Explanation; 2013; Rossmann, J., Gummer, T.
- The monetary value of good questionnaire design; 2013; Tress, F.
- Technical and methodological meta-information on current practices in online research: A full population...; 2013; Burger, C., Stieger, S.
- Using interactive feedback to enhance response quality in Web surveys. The case of open-ended questions...; 2013; Emde, M., Fuchs, M.
- Reducing Response Order Effects in Check-All-That-Apply Questions by Use of Dynamic Tooltip Instructions...; 2013; Kunz, T., Fuchs, M.
- Slide to ruin data: How slider scales may negatively affect data quality and what to do about it; 2013; Funke, F.
- Measuring wages via a volunteer web survey – a cross-national analysis of item nonresponse; 2013; Steinmetz, S., Annmaria, B.
- Identifying and Mitigating Satisficing in Web Surveys: Some Experimental Evidence; 2013; Blumenstiel, J. E., Rossmann, J.
- Does one really know?: Avoiding noninformative answers in a reliable way.; 2013; de Leeuw, E. D., Boevee, A., Hox, J.
- Online Mixed Mode Surveying using a Responsive Design; 2013; Kissau, K.
- Sensitive Topics in PC and Mobile Web Surveys; 2013; Mavletova, A. M., Couper, M. P.
- Mobile Research Performance: How Mobile Respondents Differ from PC Users Concerning Interview Quality...; 2013; Schmidt, S., Wenzel, O.
- Who responds to website visitor satisfaction surveys?; 2013; Andreadis, I.
- Measuring working conditions in a volunteer web survey; 2013; de Pedraza, P., Villacampa, A.
- Sampling online communities: using triplets as basis for a (semi-) automated hyperlink web crawler.; 2013; Veny, Y.
- Prison break: Releasing offline experiments from methodological constraints by transforming them into...; 2013; Förstel, H., Manthei, K., Mohnen, A., Berger, G.
- Comparison of psychometric properties of internet versions of the Marlowe-Crowne Social Desirability...; 2013; Vesteinsdottir, V., Reips, U. -D., Joinson, A. N., Porsdottir, F.
- Why are you leaving me?? - Personality predictors of answering drop out in an online-study; 2013; Thielsch, M., Nestler, S., Back, M.
- Propensity Score Weighting – Can Personality Adjust for Selectivity?; 2013; Glantz, A., Greszki, R.
- Research Design as an Influencing Factor for Reliability in Online Market Research; 2013; Wengrzik, J., Theuner, G.
- Ethics, privacy and data security in web-based course evaluation; 2013; Salaschek, M., Meese, C., Thielsch, M.
- Seducing the respondent – how to optimise invitations in on-site online research?; 2013; Póltorak, M., Kowalski, J.
- Influence of mobile devices in online surveys; 2013; Maxl, E., Baumgartner, T.
- E-questionnaire in cross-sectional household surveys; 2013; Karaganis, M.
- GESIS Online Panel Pilot: Results from a Probability-Based Online Access Panel; 2013; Kaczmirek, L., Bandilla, W., Schaurer, I., Struminskaya, B., Weyandt, K.
- Online Survey – Research with children on advertising impact; 2013; Funkenweh, V., Busch, J., Amthor, A. L., Boeer, A., Gaedke, J.
- HTML5 and mobile Web surveys: A Web experiment on new input types; 2013; Funke, F.
- Metadata on the demographics of online research: Results from a full-range study of available online...; 2013; Burger, C., Stieger, S.
- How the screen-out influence the dropout of a commercial panel; 2013; Bartoli, B.
- Beyond methodology - some ethical implications of "doing research online"; 2013; Heise, N.
- Innovation in Data Collection: the Responsive Design Approach; 2013; Bianchi, A., Biffignandi, S.
- Break-off and attrition in the GIP amongst technologically experienced and inexperienced participants...; 2013; Blom, A. G., Bossert, D., Clark, V., Funke, F., Gebhard, F., Holthausen, A., Krieger, U., Wachenfeld...