Relevance & Research Question: The use of web-based data collection methods is still a growing trend in the Social Sciences. However, up to now, only very little is known about which features online researchers are actually using in their online questionnaires. The present study aims to fill this knowledge gap by examining all available active online questionnaires from five major online recruitment portals.
Methods & Data: After identifying online portal sites that are widely used by researchers to recruit participants for their online studies, we extracted objectively identifiable information regarding the use of technical features from all available online questionnaires on these portal sites (N = 720 online questionnaires). Online questionnaires were evaluated according to quality standards recommended in the methodological literature.
Results: Online questionnaires complied well with the avoidance of disadvantageous features (e.g., using pre-answered item options, using non-standard technologies), but failed to comply with standards pertaining to the implementation of advantageous features (e.g., using progress indicators, keeping expected duration short).
Added Value: Regarding the compliance with methodological standards of online questionnaires, online researchers are quite successful in avoiding the use of disadvantageous features in their questionnaires. However, online researchers should also consider the implementation of advantageous features that are beneficial for both participants' motivation and data quality.