Web Survey Bibliography
Title Mode test of an online and paper employee satisfaction survey: Comparison of respondents and non-respondents
Author Gesell, S. B., Burkholz, S. D., Standiford, M. J.
Year 2004
Access date 10.05.2004
Abstract Using a controlled experimental research design, we compared a 41-item self-administered employee satisfaction web survey to a self-administered paper and pencil survey.
This study was designed to avoid typical problems of web surveys:
1. We did not have a problematic sampling frame as all employees (n=644) of one U.S. hospital were included and had access to a computer.
2. We guarded against respondents answering more than once by assigning a unique PIN to each employee.
3. Technical support was made available to respondents.
4. We had demographic information for all respondents and non-respondents.
Response rates did not vary across mode, gender, or mode by gender.There were differences between respondents and non-respondents depending on mode. For both modes, respondents and non-respondents differed on employment status, supervisory role, and number of years employed at the hospital. For the paper survey, respondents and non-respondents differed in age. For the online survey, respondents and non-respondents differed in position. Respondents and non-respondents did not vary on education, race, sex, or work shift for either mode. In general, the questions with the most missing data had it missing in both conditions and revolved around sensitive issues such as grievance procedures, compensation, and benefits. Respondents skipped these sensitive items on paper more than online. There were no signs of respondent fatigue or skipping the final sections on the survey in either mode. Overall satisfaction with one's current work situation was computed as a composite score: Variability in responses was equivalent across modes; however, the online survey yielded a lower mean score. It is speculated that employees felt more anonymity online and less hesitation to critique their employer. When asked which mode employees would prefer to use to complete an employee satisfaction survey, respondents were equally distributed between preferring an online survey, preferring a paper survey, and having no preference for mode.Respondents gave accurate responses to questions asking for personal information. It does not appear that respondents tried to hide their identity.
This study was designed to avoid typical problems of web surveys:
1. We did not have a problematic sampling frame as all employees (n=644) of one U.S. hospital were included and had access to a computer.
2. We guarded against respondents answering more than once by assigning a unique PIN to each employee.
3. Technical support was made available to respondents.
4. We had demographic information for all respondents and non-respondents.
Response rates did not vary across mode, gender, or mode by gender.There were differences between respondents and non-respondents depending on mode. For both modes, respondents and non-respondents differed on employment status, supervisory role, and number of years employed at the hospital. For the paper survey, respondents and non-respondents differed in age. For the online survey, respondents and non-respondents differed in position. Respondents and non-respondents did not vary on education, race, sex, or work shift for either mode. In general, the questions with the most missing data had it missing in both conditions and revolved around sensitive issues such as grievance procedures, compensation, and benefits. Respondents skipped these sensitive items on paper more than online. There were no signs of respondent fatigue or skipping the final sections on the survey in either mode. Overall satisfaction with one's current work situation was computed as a composite score: Variability in responses was equivalent across modes; however, the online survey yielded a lower mean score. It is speculated that employees felt more anonymity online and less hesitation to critique their employer. When asked which mode employees would prefer to use to complete an employee satisfaction survey, respondents were equally distributed between preferring an online survey, preferring a paper survey, and having no preference for mode.Respondents gave accurate responses to questions asking for personal information. It does not appear that respondents tried to hide their identity.
Access/Direct link
Homepage - conference (abstract)
Year of publication2004
Bibliographic typeConferences, workshops, tutorials, presentations
Web Survey Bibliography - 2004 (510)
- HISBUS Online-Panel: A virtual student village as an instrument for providing political advice and flash...; 2004; Muessig Trapp, P.
- Comparison of laboratory and online experiments: A research program; 2004; Ollesch, H., Heineken, E., Schulte, F. P.
- Paper and pencil or online? Methodological Experiences from an employee survey; 2004; Poetschke, M.
- Employee surveys via Internet or paper? The influence of administration mode, anonymity, voluntariness...; 2004; Reips, U. -D., Franek, L.
- Using web surveys in mixed-mode approaches: an experimental comparison with traditional survey modes; 2004; Riek, S., Rietz, C., Kruger, T.
- Teaching online methods in higher education - the study module "Experimental Psychology" as an example...; 2004; Schulte, F. P., Heineken, E., Ollesch, H.
- "What the h... are they doing?" - What are respondents doing while filling in an online-questionnaire...; 2004; Stieger, S.
- An Overview of Capabilities and Methodological Research Conducted by the Government and Academic Area...; 2004; Dennis, J. M.
- Probability Samples vs. Volunteer Respondents in Internet Research: Defining Potential Effects on Data...; 2004; Pineau, V., Slotwiner, D.
- R U There? Using Text Messaging as a method of contact in Wireless; 2004; Buskirk, T. D., Steeh, C. G.
- Qualitative Discussion Groups: An Online Contribution to Research Methods; 2004; Dammer, I.
- Motivations underlying the intention to participate in Internet-based research; 2004; Deutskens, E., Wetzels, M., de Ruyter, K.
- Online Experiments in Commercial Market Research; 2004; Nordmeyer, C. -F., Geissler, H., Donath, T.
- Online- and offline-employee surveys in theory and practice; 2004; Escher, C., Hauser, F.
- Do access panels really yield representative results?; 2004; Faas, T.
- Mode test of an online and paper employee satisfaction survey: Comparison of respondents and non-respondents...; 2004; Gesell, S. B., Burkholz, S. D., Standiford, M. J.
- Online-Surveys: Effects of different display formats, response orders as well as progress indicators...; 2004; Glauer, R., Schneider, D.
- What They See Is What We Get: Response Options for Web Surveys; 2004; Tourangeau, R., Crawford, S. D., Conrad, F. G., Couper, M. P.
- Stereotypic Response Patterns within Matrix Questions in Web Surveys; 2004; Gockenbach, S., Bosnjak, M., Goeritz, A.
- Material Incentives in Web Surveys: Two Meta-Analyses; 2004; Goeritz, A.
- Salutation, Power and behaviour in on-line panels; 2004; Joinson, A. N., Reips, U. -D.
- Web-based surveys in market and social research - usage and needs of different user groups in the EU; 2004; Kaczmirek, L., Bosnjak, M., Bandilla, W., Auer, T.
- Conceptual brand m@pping - A web-based method to elicit conceptual networks of brand knowledge and to...; 2004; Klein-Reesink, T.
- Website optimisation & controlling by benchmarking; 2004; Knapp, F.
- Application of image-based conjoint analysis on the internet; 2004; Koch, T.
- E-voting: participation, turn out, and digital divide; 2004; Oostveen, A.-M., Besselaar, P.
- The Effect of Motivating Elements on Response Strategies in Online Surveys; 2004; Boehme, R.
- Online and other methods of data collection in employee surveys: A comparison; 2004; Borg, I., Faulbaum, F.
- Are online-offline differences in personality test scores due to increased self-disclosure?; 2004; Buchanan, T., Joinson, A. N.
- Telephone versus Internet samples for a national advisory referendum: are the underlying stated preferences...; 2004; Li, H., Berrens, R. P., Bohara, A. K., Silva, C. L., Weimer, D. L., Jenkins-Smith, H. C.
- Should We Trust Web-Based Studies? A Comparative Analysis of Six Preconceptions About Internet Questionnaires...; 2004; Gosling, S. D., Vazire, S., Srivastava, S., John, O. P.
- On the integration of the Internet into science communication; 2004; Barjak, F.
- Determinants of participitation in online access panels; 2004; Batinic, B.
- Spotting online influentials among business audiences. Using online research in viral marketing campaigns...; 2004; Cakim, I.
- Publishing survey reports with XML; 2004; MacKay, I.
- Fully automated quality monitoring; 2004; Lundgren, P., Nergard, A.
- Are the mobile phone users ready for MCASI - mobile computer assisted self interviewing?; 2004; Tjostheim, I., Thalberg, S.
- What you see is what you get: An international perspective in online survey design; 2004; Thomas, R. K.
- Website design is about understanding the user: How a modal theory of user experience can help develop...; 2004; Comley, P., Lang, J.
- “The full picture” – using eye tracking technology to make web site design more effective...; 2004; Barber, H., Janes, I.
- From e-surveys to e-interviews: how to use technology to interact more intelligently with respondents...; 2004; Loewe, G.
- Sampling: the next must-have for online market researchers; 2004; Luth, R.
- Online panels are the future!; 2004; Noyce, D.
- Internet-Usage within a Delphi-study; 2004; Balzer, L.
- User and Non-user Behaviour - Illustrated by the largest Internet Retailer; 2004; Ergenzinger, R., Bamert, T.
- The impact of web page text-background colour combinations on readability, retention, aesthetics and...; 2004; Hall R. H., Hanna, P.
- An Examination of the Equivalence of Web-Based Versus Paper-and-Pencil Upward Feedback Ratings: Rater...; 2004; Smither, J.W., Walker, A.G., Yap, M.K.T.
- The role of metadata in the Statistical Knowledge Network - An emerging research agenda; 2004; Denn, S., Hs, S.W., Hert, C. A.
- Response Rate and Response Quality of Internet-Based Surveys: An Experimental Study; 2004; Deutskens, E., de Ruyter, K., Wetzels, M., Oosterveld, P.
- The Influence of Visual Layout on Scalar Questions in Web Surveys; 2004; Christian, L. M.