Web Survey Bibliography
The seriousness check (Reips, 2000, 2008) consists of one or two questions asked at the very beginning of an Internet-based study (“Have you already participated in this study?”, “Are you going to fill in the questionnaire seriously?”). This gives participants the chance to state their motivation in taking part. Some are, for example, researchers or students and thus only interested in the study design and technology behind the online study. These people should have the possibility to mark their data as invalid. In the present study we empirically validate the seriousness technique approach.
In a large online questionnaire study, a total of 1,046 data sets were collected. In order to get a measure of data quality, we recorded user actions (clicks, mouse movements, activation of form elements like radio buttons, text input) with time stamps by using a Java script that was implemented on each HTML page. User actions (e.g., clicking through, excessive clicking, longer inactivities, or changes on already given answers) were inspected for suspicious and highly suspicious behaviors.
A correlation analysis showed that participants who stated having already participated in the study also had more highly suspicious behaviors (r = .113, p < .001) and participants who stated not going to fill in the questionnaire seriously also showed more suspicious (r = .080, p = .011) and highly suspicious behaviors (α = 10%; r = .051, p = .099). Sorting out previous and non-serious participants reduced the amount of highly suspicious behaviors (mean drop of 11.9%).
The approach used in this study made it possible to add to our knowledge about the effectiveness of the seriousness check. It turned out that asking filter questions indeed reduces problems with data quality in online questionnaires.
Conference homepage (abstract)
Web Survey Bibliography - General Online Research Conference (GOR) 2009 (66)
- Conducting Mobile Surveys: A Hands-on Introduction to an Innovative Research Mode; 2009; Pferdekämper, T., Melcher, T.
- Empirical Evaluation of Web Survey Software Tools: Powerful or Friendly?; 2009; Vehovar, V., Berzelak, N., Lozar Manfreda, K., Horvat, T., Debevc, M.
- Qualitative Research via Internet: Asynchronous Online Discussions and the Use of WebCT; 2009; Giatsi Clausen, M., Nicol, M., Gill, J.
- Representativeness of Mobile Internet Surveys - A comparative study of CAMI vs. CATI ; 2009; Maier, U., Neubarth, W., Grosser, A., Hombach, A.
- The use of online data-collection in financial services market measurement research : the FRS experience...; 2009; Cooke, M., Watkins, N.
- Using flash type questions – stroke of luck or curse for data quality?; 2009; Laufer, S., Klapproth, U., Noll, S.
- Pause Mechanism in Complex Online Surveys; 2009; Milewski, J.
- Sensitive Questions in Online Surveys: Experimental Results for the Randomized Response Technique (RRT...; 2009; Coutts, E., Jann, B.
- Response Formats in Cross-cultural Comparisons in Web-based Surveys; 2009; Thomas, R. K.l, Terhanian, G., Funke, F.
- Generic or Project-Specific Mail? – The Influence of Invitations on Response Behaviour in the...; 2009; Schroll, S.
- Results from 6 Independent Web Experiments Comparing Visual Analogue Scales with Categorical Scales; 2009; Funke, F., Reips, U. -D.
- Relevance Of Health-Related Online-Information In Offline- And Online-Samples; 2009; Stetina, B. U., McElheney, J., Lehenbauer, M., Hinterberger, E., Pintzinger, N., Kryspin-Exner, I.
- Data Collection online: Can do, must (not) do, Should do; 2009; Diesner, J., Gadeib, A., Lüttschwager, F., Sassinot-Uny, L.
- Three Different Designs of Type Ranking‐Questions; 2009; Sackl, A.
- Usability of Mobile Surveys; 2009; Tarkus, A.
- Gay and Lesbian People: The Use of Online Communication Services; 2009; Lehenbauer, M., Stetina, B. U., Kryspin-Exner, I.
- An Online Study on Coping with Anxiety and Disease-Specific Internet Use in Panic Attack Sufferers; 2009; König, D., Hiebler, C., Kryspin-Exner, I.
- Distortion of demographics through technically induced dropout in restricted online surveys; 2009; Voracek, M., Stieger, S., Goeritz, A.
- An Internet-based Study on Coping with Illness and Attitudes towards Online Health Care in Cancer Patients...; 2009; Setz, J., König, D., Kryspin-Exner, I.
- WebEXEC: A Short Self-Report Measure of Executive Function Suitable for Administration via the Internet...; 2009; Buchanan, T., Heffernan, T. M., Parrott, A. C., Ling, J., Rodgers, J., Scholey, A. B.
- Does the seriousness check really sieve out datasets with bad data quality?; 2009; Stieger, S., Reips, U. -D.
- Respondents’ Evaluations of Questionnaires: A Single-Item Open-Ended Instrument for Quality Control...; 2009; Baier, C., Kaczmirek, L., Züll, C.
- Let's go formative: Continuous student ratings with Web 2.0 application Twitter; 2009; Burger, C., Stieger, S.
- Don't know and no opinion responses in Web surveys; 2009; Mechling, J., Baker, R. P., Couper, M. P.
- Mixed Methods in Online Research; Conceptualisation and Future Research Agenda; 2009; Koller, M., Sinitsa, E.
- Market Research Online and Offline - Differences in output and processing of Qualitative Online and...; 2009; Knorr, H., Krischke-Ramaswamy, M.
- Informal Learning in Virtual Communities. Individual Practice between Information Retrieval, Observation...; 2009; Kahnwald, N.
- Self-Efficacy Of Online Health Seekers; 2009; Stetina, B. U., Schramel, C., Lehenbauer, M., Schawill, W., Kryspin-Exner, I.
- Investigating the High Hurdle Technique; 2009; Frauendorfer, D., Reips, U. -D.
- Volumetric Forecast based on Online Access Panels; 2009; Rodenhausen, T., Drewes, F.
- How representative are sentiments expressed in social media for the marketing target audience? A comparison...; 2009; Jarchow, C., Thomas, J.
- SNB - Social Network Barometer; 2009; Drosdow, M., Geißler, H.
- Online Reputation Systems: The Effects of Feedback Comments and Reactions on Building and Rebuilding...; 2009; Utz, S., Matzat, U., Snijders, C.
- Using Interactive Features to Motivate and Probe Responses to Open-Ended Questions; 2009; Christian, L. M., Oudejans, M.
- Payments via Paypal as an Incentive in Online Panels; 2009; Goeritz, A., Wolff, H.-G., Goldstein, D. G.
- Advertising Effects of Online Video Ads; 2009; Wolf, M., Schönfeldt, J.
- The impact of privacy concerns on data collection; 2009; Reips, U. -D., Joinson, A. N., Buchanan, T., Schofield Paine, C.
- Online election forecasts; 2009; Faas, T., Geißler, H.
- Propensity Score Adjustment for Web Survey of Voting Behavior: A Case in Japan; 2009; Kobayashi, T.
- Why Do I Use the Social Web?” Exploring the Motives of Active and Passive Users via Focus Groups...; 2009; Jers, C., Taddicken, M., Schenk, M.
- Diffusion of Mobile Services Adoption in Taiwan; 2009; Doong, H.-S., Wang, H.-C.
- Verbal Vs Visual Response Options: Reconciling Meanings Conveyed by a Computer Aided Visual Rating Scale...; 2009; Garland, P., Cape, P.
- Strategies for Increasing Response Rates in Web Surveys and their Differential Effects on Sample Composition...; 2009; Haas, I., Bosnjak, M., Bandilla, W., P., Galesic, M.Couper, M. P.
- Increasing response rates in list based samples; 2009; Keusch, F., Kurz, H., Penzkofer, P.
- GPCP: A German Version of the Scale for Online Privacy Concern and Protection for Use on the Internet...; 2009; Oostlander, J., Reips, U. -D., Buchanan, T.
- Resolving the Privacy Paradox? - How Privacy Concerns, Strategic Self-presentation, and Norms Influence...; 2009; Utz, S.
- AGOF internet facts – increasing the response rate for onsite-surveys; 2009; Förstel, H.
- Implementation of a reaction time tool for brand measurement at Swisscom; 2009; Paar, I., Urbahn, J.
- It’s all about customer satisfaction - Advantages and limitations of online surveys in applied...; 2009; Einhorn, M., Klein-Reesink, T., Löffler, M.
- Potential Of The Mobile Internet - What You Ask Is What You Get; 2009; Neubarth, W., Maier, U., Geißlitz, A.