Web Survey Bibliography
Usability testing and user satisfaction are cornerstones of electronic survey development. There are nos clear recommendations regarding how self-reported user satisfaction questions (e.g., Chin, Diehl, & Norman, 1988) should be implemented. Human-computer interaction research and theory suggest that the method matters, and that overly-positive ratings are likely when they are collected on the same computer that previously ran the software being evaluated by the user (Nass & Moon, 2000). We use a 2x2 between-subjects randomized experiment to evaluate the effect of two components of social presence on user satisfaction ratings: data collection mode (paper-and-pencil v. computerized, where the computer used for satisfaction ratings is sometimes the computer that ran the software that was usability tested), and physical proximity of the respondent to the computer on which the usability test was conducted (i.e., in the same room as the usability test or a different room). We present data from about 200 respondents, across 5 usability tests of websites and web surveys. Our initial analyses do not support past research findings that participants rate satisfaction highest when answering on the computer that was used for the usability test. We find the lowest ratings when the satisfaction questionnaire was completed on paper in the same room as the usability test (social presence on the proximity dimension, but not the mode dimension). Individual satisfaction scale items asking about information arrangement, clarity, and navigation within the website or survey showed this difference. This difference was also found in a simple summative satisfaction scale. Our findings bring into question whether the robust social presence findings of Nass and colleagues and their theoretical implications (e.g., ethopoeia) are of concern in all human-computer interaction settings, and may provide guidance for usability testers‘ decisions about how and where to collect user satisfaction ratings.
Conference Homepage (abstract)
Web survey bibliography - Krosnick, J. A. (41)
- Assessing the Accuracy of 51 Nonprobability Online Panels and River Samples: A Study of the Advertising...; 2016; Yang,Y.;Callegaro,M.;Yang,Y.;Callegaro,M.;Chin,K.;Yang,Y.;Villar,A.;Callegaro, M.; Chin, K.; Krosnick...
- Establishing the accuracy of online panels for survey research; 2016; Bruggen, E.; van den Brakel, J.; Krosnick, J. A.
- Moderators of Candidate Name-Order Effects in Elections: An Experiment; 2016; Kim, Nu.; Krosnick, J. A.; Casasanto, D.
- Understanding Society Innovation Panel Wave 7: Results from Methodological Experiments; 2015; Blom, A. G.; Burton, J.; Booker, C. L.; Cernat, A.; Fairbrother, M.; Jaeckle, A.; Kaminska, O.; Keusch...
- Comparing response order experiments with probability and non-probability samples; 2015; Yeager, D. S.; Krosnick, J. A.; Silber, H.
- A critical review of studies investigating the quality of data obtained with online panels based on...; 2014; Callegaro, M., Villar, A., Yeager, D. S., Krosnick, J. A.
- Online panel research: History, concepts, applications and a look at the future; 2014; Callegaro, M., Baker, R., Bethlehem, J., Goeritz, A., Krosnick, J. A., Lavrakas, P. J.
- Improving ability measurement in surveys by following the principles of IRT: The Wordsum vocabulary...; 2012; Cor, K., Haertel, E., Krosnick, J. A., Malhotra, N.
- Improving Question Design to Maximize Reliability and Validity; 2012; Krosnick, J. A.
- How accurate are surveys of objective phenomena?; 2012; Chang, L. C., Krosnick, J. A.
- A Systematic Review of Studies Investigating the Quality of Data Obtained with Online Panels; 2012; Callegaro, M., Villar, A., Krosnick, J. A., Yeager, D. S.
- Measuring americans' issue priorities. A new version of the most important problem question reveals...; 2011; Yeager, D. S., Larson, S. B., Krosnick, J. A., Tompson, T.
- Experiments for evaluating survey questions; 2011; Krosnick, J. A.
- Does mentioning "some people" and "other people" in a survey question increase the...; 2011; Yeager, D. S., Krosnick, J. A.
- Measuring User Satisfaction in the Lab: Questionnaire Mode, Physical Location, and Social Presence Concerns...; 2011; Jans, M., Romano, J. C., Ashenfelter, K. T., Krosnick, J. A.
- Measuring Intent to Participate and Participation in the 2010 Census and Their Correlates and Trends...; 2010; Pasek, J., Krosnick, J. A.
- Assessing the Accuracy of the Face-to-Face Recruited Internet Survey Platform: A Comparison of Behavioral...; 2010; Villar, A., Malka, A., Krosnick, J. A.
- Study of Non-Probability Sample Internet Surveys' Estimates of Consumer Product Usage and Demographic...; 2010; Yeager, D. S., Carter, A., Tewoldemedhin, H., Krosnick, J. A.
- User’s Guide to the Advance Release of the 2008-2009 ANES Panel Study ; 2009; DeBell, M.; Krosnick, J. A.; Lupia, A.; Roberts, C.
- Question and Questionnaire Design; 2009; Krosnick, J. A., Presser, S.
- Attrition in a Face-to-Face Recruited Internet Panel with Substantial Incentives; 2009; Malka, A., Krosnick, J. A., Ackermann, A., Debell, M., Turakhia, C.
- Lessons Learned About How to Accomplish Effective In- Person Recruitment of a Web-Equipped Survey Panel...; 2009; Ackermann, A., Krosnick, J. A., Turakhia, C., Debell, M., Malka, A., Jarmon, R.
- Comparison Study of Probability and Non-Probability Sample Surveys Conducted by Internet and Face to...; 2009; Yeager, D. S., Krosnick, J. A.
- Does Weighting Improve the Accuracy of Data from Non- Probability Internet Survey Panels of People Who...; 2009; Yeager, D. S., Krosnick, J. A.
- National Surveys Via RDD Telephone Interviewing vs. the Internet: Comparing Sample Representativeness...; 2009; Chang, L. C., Krosnick, J. A.
- Scientific Survey Research: Sustainable in an Online World?; 2009; Krosnick, J. A.
- Money for Surveys: What about Data-Quality?; 2009; A.Krosnick, J. A.
- The accuracy of online surveys with non-probability samples; 2008; Krosnick, J. A.
- ‘For Example’: How Different Example Types in Online Surveys Influence Frequency ; 2008; Berent, M., Krosnick, J. A.
- Comparing the Results of Probability and Non-probability Telephone and Internet Survey Data; 2008; Wang, R., Krosnick, J. A.
- “For Example…,” How Different Example Types in Online Surveys Influence Frequency...; 2008; Berent, M., Krosnick, J. A.
- Response option ordering: Reconciliating meanings conveyed by rating scale position and label. Unpublished...; 2007; Garland, P., Krosnick, J. A.
- Vote Over-Reporting: Testing the Social Desirability Hypothesis in Telephone and Internet Surveys; 2005; Holbrook, A. L., Krosnick, J. A.
- The Economist/YouGov Internet Presidential poll.; 2004; Fiorina, M., Krosnick, J. A.
- Telephone versus face-to-face interviewing of national probability samples with long questionnaires:...; 2003; Holbrook, A. L.; C., Green, M. C., Krosnick, J. A.
- Can What We Don’t Know (about “Don’t Know”) Hurt Us?: Effects of Item Non-response...; 2003; Krosnick, J. A., Behnke, C. S., Lafond, C. R., Thomas, R. K.
- The threat of satisficing in surveys: The shortcuts respondents take in answering questions; 2000; Krosnick, J. A.
- Survey research; 1999; Krosnick, J. A.
- Designing rating scales for effective measurernent in surveys; 1997; Krosnick, J. A., Fabrigar, L. R.
- Mail Surveys for Election Forecasting? An Evaluation of the Columbus Dispatch Poll; 1996; Krosnick, J. A.
- Response strategies for coping with the cognitive demands of attitude measures in surveys; 1991; Krosnick, J. A.