Notice: the WebSM website has not been updated since the beginning of 2018.

Web Survey Bibliography

Title Mode Effects and Non-Response Bias in an Undergraduate Student Satisfaction Survey: Results from a Randomized Experiment Comparing Telephone and Web Administration
Source The American Association for (AAPOR) 63rd Annual Conference, 2008 & WAPOR 61th Annual Conference, 2008
Year 2008
Access date 02.06.2009

Student satisfaction surveys are a key source of input to university enrollment management and quality improvement efforts. Maximizing response rates and collecting high quality data in these surveys are priorities for university administrators. One of the key design decisions in student surveys is mode of administration, with many arguing that web surveys are particularly appropriate for this population. This paper presents results of a randomized experiment comparing telephone (sample n = 1,967) and web-based (n = 1,966) administrations of surveys to freshmen, sophomores, and juniors at the University of Pittsburgh in spring, 2007. Both modes involved multiple follow-up contacts to maximize response rates, and the survey instrument was parallel across modes. The survey asked about satisfaction and experiences in academic, social, and several other campus life domains. The overall response rate was slightly higher for the web (60.4%; n = 1,188 responses) than for the telephone (55.4%; n = 1,090), although this difference varied by sub-groups of students. Differences between responders and non-responders (obtained from administrative records) were also somewhat different for telephone versus web. Analyses of the effects of mode on survey responses showed fairly consistent differences in which students reported significantly lower satisfaction levels on the web compared to the telephone when reporting to an interviewer. Mode effects were found for evaluative/subjective survey items, but were less likely for survey items dealing with factual/objective behavior. Mode effects were especially pronounced in the domain of social integration and satisfaction. There were also interesting reversals of the mode effects (e.g., web greater than phone) on items related to the perceived contribution of their college education to various academic skills. Implications of the findings for the design of student surveys are discussed.

Access/Direct link

Homepage (abstract)

Year of publication2008
Bibliographic typeConferences, workshops, tutorials, presentations
Full text availabilityAvailable on request