Web Survey Bibliography
Relevance & Research Question: For several years, the Federal Statistical Office (FSO) has been working on the systematic implementation of questionnaire testing. A pretest laboratory was established in 2007 and complemented by an eye tracker in 2009. Questionnaires of online surveys are now increasingly evaluated by qualitative testing methods and redesigned to reduce the burden for respondents and to increase data quality of official statistics.
Methods & Data: Pretesting online questionnaires shall improve their usability, functionality and comprehensibility. At the FSO, a three step approach is applied: Firstly, we observe eye movements and facial expressions (in real-time), while respondents deal with the questionnaire. Secondly, we conduct cognitive interviews afterwards in order to discover the reasons why respondents proceeded the way they did. Thirdly, we evaluate the process of self-completing by eye tracking data (e. g. ‘Areas of Interest’) and the sequence of mouse clicks.
Results: Each source of information has its strengths and weaknesses: Generally, it is challenging to analyze eye tracking data. It is for example difficult to assess whether a longer fixation duration indicates problems or simply a higher interest in a question. Consequently, the interpretation might be misleading without profound background knowledge. From a different angle, results derived from cognitive interviews are of minor value if the answers of respondents seem to be determined by effects like acquiescence, social desirability or limited capacity for remembering and verbalizing cognitive processes. By linking our sources of information (“triangulation”) we are able to provide more valid pretesting results and recommendations for improving online questionnaires.
Added Value: When online questionnaires are tested at the FSO, cognitive interviews are conducted after eye tracking itself. Combining both methods has given us insights into users’ behaviour when reading off screen and their expectations concerning navigation. The analyses illustrate whether respondents perceive links to detailed explanations, skip instructions or entire lists of response options. General advice is provided on wording and design principles for improving online questionnaires. Our findings lead us directly to Steve Krug’s (2006) saying: “Don’t make me think!”
Conference Homepage (abstract)
Web survey bibliography - Germany (361)
- Mobile, webmail, desktops: Where are we viewing email now?; 2011
- Assessing personality traits through response latencies using item response theory; 2011; Ranger, J., Ortner, T. M.
- Web-based rating scales: HTML 5 and other innovations; 2011; Funke, F.
- E-dater, Artificial Actors, and German Households; 2011; Hebing, M.
- Seeing Through the Eyes of the Respondent: An Eye-tracking Study on Survey Question Comprehension; 2011; Lenzner, A., Kaczmirek, L., Galesic, M.
- Eye Tracking in testing questionnaires: What’s the added value?; 2011; Tries, S.
- Improving validity in web surveys with hard‐to‐reach targets: Online Respondent Driven Sampling...; 2011; Mavletova, A. M.
- Ignoring the compatibility of online questionnaires may bias the psychological composition of your sample...; 2011; Funke, F.
- Video enhanced web survey; 2011; Fuchs, M., Kunz, T., Gebhard, F.
- Scrolling or paging - it depends; 2011; Blanke, K.
- The German Access Panel and the Impact of Response Propensities; 2011; Amarov, B., Enderle, T., Muennich, R., Rendtel, U., Zins, S.
- A new approach to the analysis of survey drop-out. Results from Follow-up Surveys in the German Longitudinal...; 2011; Rossmann, J., Blumenstiel, J. E., Steinbrecher, M.
- Tracking the decision-making process – Findings from an Online Rolling Cross-Section Panel Study...; 2011; Faas, T.
- From "Web Questions" to "Propensity Score Weighting": An Evaluation of Topics and...; 2011; Welker, M., Taddicken, M.
- Rich Profiles – Or: What's the problem with self-disclosure data?; 2011; Tress, F.
- Mobile Research Apps – Adding New Capabilities to Market Research; 2011; Rieber, D.
- The influence of personality traits and motives for joining on participation behavior in online panels...; 2011; Keusch, F.
- Asking sensitive questions in a recruitment interview for an online panel: the income question; 2011; Schaurer, I., Struminskaya, B., Kaczmirek, L., Bandilla, W.
- Speeders in Online Value Research: Cross-checking results of fast and slow respondents in two separate...; 2011; Beckers, T., Siegers, P., Kuntz, A.
- Effects of survey question clarity on data quality; 2011; Lenzner, T.
- Lösungsansätze gegen den Allgemeinarztmangel auf dem Land - Ergebnisse einer Online-Befragung unter Ä...; 2011; Steinhäuser, J., Annan, N. F., Roos, M., Szecsenyi, J., Joos, S.
- Question Comprehensibility and Satisficing Behavior in Web Surveys; 2011; Lenzner, T.
- Agree-Disagree Response Format versus Importance Judgment; 2011; Krebs, D.
- Germans' segregation preferences and immigrant group size: A factorial survey approach; 2011; Schlueter, E., Ullrich, J., Schmidt, P.
- Benefits of Structured DDI Metadata across the Data Lifecycle: The STARDAT Project at the GESIS Data...; 2011; Linne, M., Brislinger, E., Zenk-Moeltgen, W.
- Microdata Information System MISSY; 2011; Bohr, J.,
- Explaining more variance with visual analogue scales: A Web experiment; 2011; Funke, F.
- Cognitive process in answering questions: Are verbal labels in rating scales attended to?; 2011; Menold, N., Kaczmirek, L., Lenzner, T.
- Experiments on the Design of the Left-Right Self-Assessment Scale; 2011; Zuell, C., Scholz, E., Behr, D.
- Separating selection from mode effects when switching from single (CATI) to mixed mode design (CATI /...; 2011; Carstensen, J., Kriwy, P., Krug, G., Lange, C.
- Asking Sensitive Questions: Do They Affect Participation In Follow-Up Surveys?; 2011; Schaurer, I., Struminskaya, B., Kaczmirek, L., Bandilla, W.
- Does social desirability compromise self-reports of physical activity in web-based research?; 2011; Crutzen, R., Goeritz, A.
- Testing for measurement equivalence of human values across online and paper-and-pencil surveys; 2011; Davidov, E., Depner, F.
- The use of paradata to monitor and manage survey data collection; 2010; Kreuter, F., Couper, M. P., Lyberg, L. E.
- Optimizing response rates in online surveys; 2010; Kaczmirek, L.
- There is an app for that! A review of smartphone apps for marketing research; 2010; Michelson, M.
- Innovative mobile research in developing countries; 2010; Bellity, E.
- Mobile location based research: Cross cultural examination of coffee culture; 2010; Morden, M., Ferneyhough, C., Grenville, A.
- Online research….and all that Jazz!; 2010; Gittelman, S. H., Trimarchi, E.
- Why are we trying to create new communities for market research purposes?; 2010; Pearson, C., Kateley, V.
- Internet-Based Measurement With Visual Analogue Scales: An Experimental Investigation; 2010; Funke, F.
- Managing the knowledge base - the DUVA system, from data entry to output tools; 2010; Then, R., Bangert, D.
- Recruiting Online Panel Members from a Mail survey in the General Population: Results from an Exploratory...; 2010; Reuband, K. H.
- Testing the Applicability of Respondent Driven Sampling as an Online Research Method to Sample Hidden...; 2010; Pajak, D.
- Seriousness Checks are Useful to Improve Data Validity in Online Research; 2010; Diedenhofen, D., Aust, F., Ullrich, S., Musch, J.
- Enrichment of Qualitative Research through Online Approaches: New Insights due to Online CoCreation...; 2010; Krischke-Ramaswamy, M., Knorr, H.
- Eye Tracking and Cognitive Interviewing: Steps to improve online questionnaires; 2010; Tries, S., Sattelberger, S.
- How new engagement techniques and question approaches are revolutionizing online research data gathering...; 2010; Puleston, J.
- Social Networking Sites: New approaches for Online-Panels?; 2010; Drosdow, M., Geißler, H.
- The Impact of Visual and Functional Design Elements in Online Survey Research; 2010; Hammen, K.