Purpose of the study: Speeding and nondifferentiation as common satisficing behaviors in grid questions might be prevented by interactive feedback which can either be provided (1) after a respondent has already submitted the entire grid (delayed feedback), or (2) while a respondent is still in the process of answering the grid items (instant feedback).
Design/methodology/approach:In a randomized between-subjects experiment embedded in a Web survey among university freshmen, the effectiveness of providing instant feedback on speeding (Experiment 1) and nondifferentiation (Experiment 2) in grid questions was compared to providing delayed feedback or no feedback at all.
Findings: Providing instant feedback on nondifferentiation reduces both nondifferentiation and speeding to a larger extent as compared to delayed feedback. Also, delayed feedback on speeding results in a longer completion time which is not accompanied by a decrease in nondifferentiation. By contrast, with instant feedback the additional time used by respondents seems to be productive time since nondifferentiation is reduced. Results indicate that the timing of feedback is decisive for reducing satisficing behaviors because instant feedback is more effective in preventing speeding and nondifferentiation in grid questions than delayed feedback.
Originality/value: Although Web surveys have the technical capacities to implement a variety of interactive design features, little is known by now about the possible applications of interactive design features and their consequences for respondent behavior. While previous research has highlighted the importance of providing feedback to respondents after they have answered a question (delayed feedback), this study aims to assess the impact of feedback that is provided during the question-answer process (instant feedback).
Research limitations/implications: The experiments reported have been conducted in a student population. Whether results can be generalized to the general population has to be determined. Results are further limited by the fact that only few questions have been tested.
Practical implications: Instant feedback reduces satisficing behaviors without overstraining respondent effort. Thus, instant feedback represents a good compromise between enhancing data quality in grid questions while, at the same time, keeping respondent burden within a limit.