Web Survey Bibliography
Title Propensity score and calibration as bias reducing techniques in surveys based on Internet panels: application to the outcome of the Swedish Parliament
Author Johansson, C., Lorenc, B.
Year 2003
Access date 06.05.2004
Abstract When systematic differences in key background variables with respect to a population are present, unweighted results of surveys based on Internet panels are often misleading. An inventive application of propensity score weighting, otherwise an established technique for reducing bias in observational studies (Rosenbaum, Rubin, 1984), to the field of Internet surveys was suggested by George Terhanian of Harris Interactive (e.g. Terhanian et al. 2001). The method is though still a matter of some controversy, possibly owing somewhat to the fact that its formal presentation has not been given yet.
After a preparatory study that showed viability of the propensity score technique in a simulated population with known properties (Lorenc, in preparation), we report here on its application to reducing bias in responses on voting behaviour of an Internet panel with respect to the population in the recent Parliament election in Sweden. For the present purpose we assume that the reporting is correct and that the difference in the unweighted means of the response variables between the panel and the population as a whole is due to systematic differences in auxiliary variables between the groups.
In addition, we compare propensity score weighting with calibration (Deville, Särndal, 1992), a method that uses known univariate or cross-tabulated population totals of auxiliary variables. Thus, a contrast is established with the propensity score technique, which uses data on the individual level and can exploit more richly eventual covariation in the auxiliary information.
Deville, Särndal (1992). "Calibration estimators in survey sampling". JASA, 87:376-82.
Lorenc (in preparation). "A study of effectiveness of propensity score weighting in a simulated population". Statistics Department, Stockholm University.
Rosenbaum, Rubin (1984). "Reducing bias in observational studies using sub-classification on the propensity score". JASA, 79:516-24.
Terhanian, Marcus, Bremer, Smith (2001): "Reducing error associated with non-probability sampling through propensity scores". JSM, Atlanta, Georgia, USA.
After a preparatory study that showed viability of the propensity score technique in a simulated population with known properties (Lorenc, in preparation), we report here on its application to reducing bias in responses on voting behaviour of an Internet panel with respect to the population in the recent Parliament election in Sweden. For the present purpose we assume that the reporting is correct and that the difference in the unweighted means of the response variables between the panel and the population as a whole is due to systematic differences in auxiliary variables between the groups.
In addition, we compare propensity score weighting with calibration (Deville, Särndal, 1992), a method that uses known univariate or cross-tabulated population totals of auxiliary variables. Thus, a contrast is established with the propensity score technique, which uses data on the individual level and can exploit more richly eventual covariation in the auxiliary information.
Deville, Särndal (1992). "Calibration estimators in survey sampling". JASA, 87:376-82.
Lorenc (in preparation). "A study of effectiveness of propensity score weighting in a simulated population". Statistics Department, Stockholm University.
Rosenbaum, Rubin (1984). "Reducing bias in observational studies using sub-classification on the propensity score". JASA, 79:516-24.
Terhanian, Marcus, Bremer, Smith (2001): "Reducing error associated with non-probability sampling through propensity scores". JSM, Atlanta, Georgia, USA.
Year of publication2003
Bibliographic typeConferences, workshops, tutorials, presentations
Web Survey Bibliography (6374)
- Effects of Lotteries on Response Behavior in Online Panels; 2013; Goeritz, A., Luthe, S. C.
- Lotteries and study results in market research online panels; 2013; Goeritz, A., Luthe, S. C.
- The Effects of Errors in Paradata on Weighting Class Adjustments: A Simulation Study; 2013; West, B. T.
- Using Paradata to Study Response to Within-Survey Requests; 2013; Sakshaug, J. W.
- Paradata for Coverage Research ; 2013; Eckman, S.
- Improving Surveys with Paradata: Analytic Uses of Process Information; 2013; Kreuter, F.
- Theory of adaptation or survival of the fittest?; 2013; Cavallaro, K.
- Online Fundraising Essentials, Second Edition; 2013; Stevenson, S. C.
- Tips for Evaluating Online Effectiveness; 2013; Stevenson, S. C.
- The Digital Divide: The internet and social inequality in international perspective; 2013; Ragnedda, M., Muschert, G.
- Ten questions to ask your online survey provider; 2013; Williams, D.
- Survey quality prediction system 2.0; 2013
- Practical tools for designing and weighting survey samples; 2013; Valliant, R. L., Daver, J. A., Kreuter, F.
- Paradata in web surveys; 2013; Callegaro, M.
- Report Of The AAPOR Task Force On Non-probability sampling; 2013; Baker, R. P., Brick, J. M., Bates, N., Battaglia, M. P., Couper, M. P., Dever, J. A., Gile, K. J., Tourangeau...
- Incentive effects; 2013; Goeritz, A.
- A nationwide web-based freight data collection; 2013; Samimi, A., Mohammadian, A., Kawamura, K.
- Mode Matters: Evaluating Response Comparability in a Mixed-Mode Survey; 2013; Bowyer, B. T., Rogowski, J. C.
- Comparing Survey Results Obtained via Mobile Devices and Computers: An Experiment With a Mobile Web...; 2013; de Bruijne, M., Wijnant, A.
- Cognitive Probes in Web Surveys: On the Effect of Different Text Box Size and Probing Exposure on Response...; 2013; Behr, D., Bandilla, W., Kaczmirek, L., Braun, M.
- The E-Interview in Qualitative Research; 2013; Bampton, R., Cowton, C., Downs, Y.
- Methodological Considerations of Qualitative Email Interviews; 2013; Nehls, K.
- Best Practice in Online Survey Research with Sensitive Topics; 2013; Kays, K., Keith, T. L., Broughal, M. T.
- Research Intentions are Nothing without Technology: Mixed-Method Web Surveys and the Coberen Wall of...; 2013; Ganassali, S., Rodriguez-Santos, C.
- Reducing Response Burden for Enterprises Combining Methods for Data Collection on the Internet; 2013; Vik, T.
- Measuring Wages Worldwide: Exploring the Potentials and Constraints of Volunteer Web Surveys; 2013; Steinmetz, S., Raess, D., Tijdens, K., de Pedraza, P.
- Using Web Surveys for Psychology Experiments: A Case Study in New Media Technology for Research; 2013; Peden, B. F., Tiry , A. M.
- The Distinctiveness of Online Research: Descriptive Assemblages, Unobtrusiveness, and Novel Kinds of...; 2013; Lanfrey, D.
- Sampling, Channels, and Contact Strategies in Internet Survey; 2013; Macrì, E., Tessitore, C.
- Advancing Research Methods with New Technologies; 2013; Sappleton, N.
- Data Quality in PC and Mobile Web Surveys; 2013; Mavletova, A. M.
- PDAs in socio-economic surveys: instrument bias, surveyor bias or both?; 2013; Escobal, J., Benites, S.
- Virtual research assistants: Replacing human interviewers by automated avatars in virtual worlds; 2013; Hasler, B. S., Tuchman, P., Friedman, D.
- Compared to a small, supervised lab experiment, a large, unsupervised web-based experiment on a previously...; 2013; Ryan, R. S., Wilde, M., Crist, S.
- From mixed-mode to multiple devices. Web surveys, smartphone surveys and apps: has the respondent gone...; 2013; Callegaro, M.
- Moving an established survey online – or not?; 2013; Barber, T., Chilvers, D., Kaul, S.
- Using mobile devices to access the realities of youth: How identification with society influences political...; 2013; Smith, M.
- On the Use of Latent Variable Models to Detect Differences in the Interpretation of Vague Quantifiers...; 2013; Griffin, J.
- Managing mobile research: How it's different and why it matters; 2013; Kachhi-Jiwani, D., Tucker, J., Wilding-Brown, L.
- An approach to selecting online respondents; 2013; Terhanian, G.
- By the Numbers: Theory of adaptation or survival of the fittest?; 2013; Cavallaro, K.
- Designing and conducting business surveys; 2013; Snijkers, G.,Araldsen, G., , Willimack, D. K.Jones, J.
- Battle of the Scales: Understanding Respondent Scale Usage in the US and Abroad; 2013; Courtright, M., Pashupati, K., Pettit, F. A.
- Modular Survey Design: A Bite Size Proposal; 2013; Kelly, F., Stevens, S., Johnson, A.
- Cyborgs vs. Monsters: Assembling Modular Surveys to Create Complete Datasets; 2013; Johnson, E. P., Siluk, L., Tarraf, S.
- Do I Have Your Full Attention?; 2013; Cape, P. J.
- Does Sample Size Still Matter?; 2013; Bakken, D. G., Bond, M.
- Optimizing Surveys for Smartphones: Maximizing Response Rates While Minimizing Bias; 2013; Lattery, K., Park Bartolone, G., Saunders, T.
- Shorter Isn't Always Better; 2013; Burdein, I.
- Solving the Unintentional Mobile Challenge; 2013; Peterson, G., Mechling, J., LaFrance, J., Ham, G.
