Notice: the WebSM website has not been updated since the beginning of 2018.

Web Survey Bibliography

Title Panel Conditioning in Web Surveys: A Comparison between Trained and Fresh Respondents
Year 2009
Access date 14.08.2009

Trained respondents may answer questions differently than those with little or no experience in a panel. This can result in different responses with regard to content (e.g. because of increasing knowledge on topics) as well as the procedure (question-answering process). Having two panels—a trained one and a completely fresh one —created a unique opportunity for analysing panel conditioning effects (both panels are recruited via a probability sample of the Dutch population). Due to procedural learning, trained respondents may react differently to web survey design choices than inexperienced respondents. Because of their experience they may be able to process more information on a screen, e.g. make fewer errors when more items are placed on a single screen. In addition, they may be less sensitive to social desirability bias and less reluctant to select a response category that seems unusual in the range of responses. They also may be used to a particular question layout so that changing that layout (e.g. from disagree-agree to agree-disagree) may not be noticed. We investigated whether there are differences in design effects between trained and fresh respondents using a questionnaire consisting of three experiments. In the experiments we varied the number of items on a screen, the choice of response categories, and the layout of a five point rating scale. We found that trained respondents were somewhat more sensitive to satisficing. Besides procedural knowledge, trained respondents can develop knowledge on topics addressed in previous surveys. To find out the relation between question type and panel conditioning, trained and fresh respondents answered questions with different question types. We used nine questions on two different subjects: food infection and old-age pensions. These subjects were presented to the trained panel multiple times, so we thought these would be the most sensitive to bias due to panel conditioning. The results show that panel conditioning only arises in knowledge questions. Questions on attitudes, actual behaviour, or facts were not sensitive to panel conditioning. Panel conditioning in knowledge questions was restricted to less-known subjects (more difficult questions), suggesting a relation between panel conditioning and cognition.

Access/Direct link

Conference homepage (abstract)

Year of publication2009
Bibliographic typeConferences, workshops, tutorials, presentations
Full text availabilityFurther details