In order to address concerns that changes in future collection modes could impact the consistency of Teacher Followup Survey (TFS) estimates over time, the authors conducted a mode effect analysis on the 2004-05 administration. Although predominately a paper collection, the 2004-05 sampling design incorporated a small web-collection component. Using the experiment data for secondary analysis, the authors tested the 2004-05 TFS estimates for the possibility of mode effects. The more recent 2008-09

TFS data collection used paper surveys mainly to convert nonrespondents and for a small group of teacher who did not provide an e-mail address and were not sent the web survey. This study aims at exploring potential differences in teachers’ responses between those who used the web questionnaire and those who opted for the paper questionnaire. The authors tested mode effects using six survey questions with different characteristics and levels of complexity. Selected questions with multiple items and multiple rating categories were transformed to be analyzed as estimated measures of differentiation.

A two-stage Heckman-type instrumental variable (IV) regression model was used for these analyses. The first stage models whether teachers with certain characteristics were more prone to choose the web instrument than the paper instrument. The second stage of the model determines whether having used the web or paper instrument affected the quality of the survey responses, using the IV web-choice estimated from stage one. Regression results indicate that using the web-based instrument does not lead to lower quality or different survey responses compared to paper-based responses. As a result of the findings that support the initial hypothesis that no mode effects are observed on the 2004-05 TFS, the authors conclude that changes to data collection methodologies in the recently collected web-based 2008-09 TFS are unlikely to create long-term inconsistency issues.