Web Survey Bibliography
The type of response scale may compromise the psychometric quality of a survey instrument even though its qualitative underpinning is excellent and the wording of the items is appropriate. A series of four experiments (one paper-and-pencil, two online, one eye-tracking experiment) were conducted to examine the functioning of response scales of different direction (agree-disagree versus disagree-agree). The effects of the direction of the response scale are exemplified by two different constructs, based on the Rasch measurement model. As a key result, the agree-disagree format performs better (in terms of fit and the unit of measurement) than the disagree-agree variant. Since a difference in the unit of measurement (a multiplicative bias) may lead to spurious mean differences, extreme caution has to be exercised when data sets based on different modes of collection or different response scales are merged. In search for a more detailed understanding of the differences found, an eye-tracking experiment was conducted. The eye-tracking experiment allowed to observe the answering process and to investigate the reading patterns of the respondents in a more direct way. In line with the results of the psychometric analysis, the two response formats also differed regarding reading patterns. For instance, fixation and observation length and counts of the defined areas of interest (e.g., scale-poles) were significantly higher for the disagree-agree format. These findings suggest the disagree-agree format implying more effort and difficulties for the respondents throughout the answering process.
Conference Homepage (abstract)
Web survey bibliography (95)
- Virtual reality meets sensory research; 2017; Depoortere, L.
- Methods for Evaluating Respondent Attrition in Web-Based Surveys; 2016; Hochheimer, C. J.; Sabo, R. T.; Krist, A. H.; Day, T.; Cyrus, J.; Woolf, S. H.
- Exploration of Methods for Blending Unconventional Samples with Traditional Probability Samples; 2016; Gellar, J.; Zhou, H.; D.; Sinclair, M. D.
- Ratio of Vector Lengths as an Indicator of Sample Representativeness ; 2016; Shin, H. C.
- Online and Social Media Data As an Imperfect Continuous Panel Survey; 2016; Diaz, F.; Garmon, F.; Hofman, J. K.; Kiciman, E.; Rothschild, D.
- Validating self-reported mobile phone use in adults using a newly developed smartphone application; 2015; Goedhart, G., Kromhout, H., Wiart, J., Vermeulen, R.
- Innovative Uses of Paradata Across Diverse Contexts ; 2015; Cheung, G.; Pennell, B.-E.
- Build your own social network laboratory with Social Lab: a tool for research in social media; 2014; Garaizar, P., Reips, U.-D.
- Picking up the Bread Crumbs: Holistic Insights from Social Media; 2014; Souda, P.
- Survey optimisation considerations for Android, Apple and Windows 8 mobile devices; 2013; Owen, R.
- Second Life as a Survey Lab: Exploring the Randomized Response Technique in a Virtual Setting; 2013; Richards, A., Dean, E.
- Virtual Cognitive Interviewing Using Skype and Second Life; 2013; Dean, E., Head, B., Swicegood, J. E.
- Assessing Nonresponse Bias in the Green Technologies and Practices Survey; 2013; Meekins, B., Sverchkov, M., Stang, S.
- Why Big Data is a Small Idea…and Why You Shouldn’t Worry So Much; 2013; Needel, S.
- Doing real time research: Opportunities and challenges; 2013; Back, L., Lury, C., Zimmer, R.
- Digital technology and data collection; 2013; Henriksen, B., Jewitt, C., Price, S., Sakr, M.
- Effects of Self-Awareness on Disclosure During Skype Survey Interviews; 2013; Feuer, S., Schober, M. F.
- Cognitive Interviewing in Online Modes: a Comparison of Data Collected in Second Life and Skype; 2013; Swicegood, J. E., Head, B., Dean, E., Keating, M.
- Effects of Displaying Videos on Measurement in a Web Survey; 2013; Mendelson, J., Gibson, J. L., Romano Bergstrom, J. C.
- Classifying Mouse Movements to Predict Respondent Difficulty; 2013; Horwitz, R.
- Are You Seeing What I am Seeing? Exploring Response Option Visual Design Effects With Eye-Tracking; 2013; Libman, A., Smyth, J. D., Olson, K.
- Survey Reminder Method Experiment: An Examination of Cost Efficiency and Reminder Mode Salience in the...; 2013; Anderson, M., Rogers, B., CyBulski, K., Hall, J. W., Alderks, C. E., Milazzo-Sayre, L.
- Virtual research assistants: Replacing human interviewers by automated avatars in virtual worlds; 2013; Hasler, B. S., Tuchman, P., Friedman, D.
- From mixed-mode to multiple devices. Web surveys, smartphone surveys and apps: has the respondent gone...; 2013; Callegaro, M.
- Internet-Mediated Technologies and Mixed Methods Research; Problems and Prospects; 2012; Hesse-Biber, S.; Griffin, A. J.
- An Introduction to Using Video for Research; 2012; Jewitt, C.
- Online Surveys Aren't Just for Computers Anymore! Exploring Potential Mode Effects between Smartphone...; 2012; Buskirk, T. D., Andrus, C.
- Smartphone Apps and User Engagement: Collecting Data in the Digital Era; 2012; Link, M. W.
- Specific mixed-mode methodology to reach sensory disabled people in quantitative surveys; 2012; Fontaine, S.
- Facing The Future Webcams as a survey tool in China; 2012; Gordon, A., Llewellyn, T., Gu, E.
- Comfortable in the new medium: How online qual can benefit from our share-happy culture ; 2012; Rubenstein, P.
- Using Collaborative Web Technology to Construct the Health Information National Trends Survey; 2012; Moser, R. P., Beckjord, E. B., Finney Rutten, L. J., Blake, K., Hesse, B. W.
- The Representativity of Web Surveys of the General Population compared to Traditional Modes and Mixed...; 2012; Klausch, L. T., Schouten, B., Hox, J.
- Time use data collection using Smartphones: Results of a pilot study among experienced and inexperienced...; 2012; Scherpenzeel, A., Sonck, N., Fernee, H., Morren, Me.
- Using Webinar Polls to Collect Online Survey Data: The Case of a Behavioral Finance Problem; 2012; Sahu, C.
- The Game Experiments: Researching how gaming techniques can be used to improve the quality of feedback...; 2011; Sleep, D., Puleston, J.
- The benefits and constraints of e-mail interviews and discussions as methods of accessing valid data; 2011; Roberts, An.
- Facial imaging: The new face of online survey research; 2011; Gordon, A., McCallum, D., Sorci, M., Llewellyn, T.
- On Affordances and Technological Intersubjectivity; 2011; Vatrapu, R.
- Building online communities; 2011; Mlačić, B., Milas, G., Mikloušić, I.
- Eye Tracking in testing questionnaires: What’s the added value?; 2011; Tries, S.
- Video enhanced web survey; 2011; Fuchs, M., Kunz, T., Gebhard, F.
- Engagement, Consistency, Reach – why the Technology Landscape Precludes All Three; 2011; Johnson, A., Rolfe, G.
- Twitter mood predicts the stock market.; 2011; Bollen, J., Mao, H., Zeng, X.-J.
- Web based CATI on Amazon Elastic Compute Cloud and VirtualBox using queXS; 2011; Zammit, A.
- Web/Cloud Based CATI Using queXS; 2011; Zammit, A.
- Partnership-Driven Resources to Improve and Enhance Research (PRIMER): A Survey of Community-Engaged...; 2011; Dolor, R. J., Greene, S. M., Thompson, E., Baldwin, L.-M., Neale, A. V.
- Weaving the Web into Personal Communication Networks: A Mobile Phone Based Study of Smartphone Users; 2011; Kobayashi, T., Boase, J.
- Different functioning of rating scale formats – results from psychometric and physiological experiments...; 2011; Koller, M., Salzberger, T.
- Measurement invariance in training evaluation: Old question, new context; 2011; P., Gissel, A., Stoughton, J. W., Whelan, T. J.Clark, A. P.