Web Survey Bibliography
Relevance & research questions: The increased flexibility users experience through different devices and mobile Internet is changing the way they consume content from the Internet. In 2015, a representative study of the German population found that users do not only go online to fulfill specific goals such as finding a piece of information (76 % of the participants), but they also browse the Internet for recreational purposes such as for reading articles (59%) or watching videos (53%) (ARD/ZDF-Onlinestudie, 2015). While searching for information can be seen as a goal-directed task, Internet visits for the purpose of recreation do not require the user to undertake specific tasks. For instance, when killing time users may visit the Internet with no specific goal in mind. They explore and browse. In research this differentiation between goal-directed and exploratory online behavior has been widely recognized (e.g. Hoffmann & Novak, 1996; Hassenzahl, 2005; Wang, Wang, & Farn, 2007). Although it is often assumed that during website interaction the type of task may change users’ perception (e.g. Van Schaik & Ling, 2009), very few studies experimentally manipulate the given tasks to assess possible interactional effects. This thesis aimed to fill this gap by investigating how goal-directed (searching) or exploratory (browsing) interactions performed on a website moderate the impact of key website attributes on users’ judgements. Such key elements, namely content, perceived usability, and aesthetics have already been identified by various researchers (e.g. Thielsch, Blotenberg, & Jaron, 2014). Furthermore, the current study distinguished between three post-use judgements. Besides the overall impression of a website, the intention to revisit or recommend a website was assessed. Based on psychological models following moderating effects were expected. The influences of content and usability perception were assumed to increase in searching tasks, while aesthetics was hypothesized to be of greater importance when browsing.
Methods & data: The survey was conducted online in English and German. In total, 481 participants (21.8 % female) took part, of whom 319 were German and 163 were English speaking. Ages ranged from 14 to 69 years with a mean age of 34.84 (SD = 10.68). Two different websites of competing brands in the premium and luxury fashion segment were used as stimuli. Therefore, familiarity to the online shopping context was given. Participants were randomly assigned to one of the two conditions: free exploration (browsing) or fulfillment of specific tasks (searching). In the searching condition participants were instructed to compare prices of two products and to answer questions about product materials. Both tasks were followed by items measuring the predictor variables: Perceived website usability was measured using the English questionnaire PWU (Perceived Website Usability; Flavián et al., 2006) and its German version WWU (Moshagen, Musch, & Göritz, 2009; Thielsch, 2008). For measurement of aesthetics the VisAWI (Visual Aesthetics of Websites Inventory; Moshagen & Thielsch, 2010), was employed. The Web-CLIC served to measure the subjective perception of content facets (Thielsch & Hirschfeld, under review). Finally, participants rated their intention to revisit or recommend the website and provided their overall impression.
Results: To assess the predictive strength of content, perceived usability, and aesthetics, hierarchical and stepwise multiple linear regressions were conducted using overall impression, recommendation, and revisit intention as outcome variables. Overall, content and aesthetics contributed significantly to revisit and recommendation intention as well as overall impression. Content had the greatest impact on revisit and recommendation intention, whereas aesthetics was the most influential for overall impression. Perceived usability only contributed to overall impression. Moderation analyses further analyzed the strength of those relationships for searching and browsing tasks. Interestingly, perceived usability was not influenced by the type of task in any of the calculated models. However, contrary to the hypotheses the predictive strength of aesthetics was stronger when searching. Yet, this change did not show significance in the moderation analysis. Only the influence of content was, in some models, significantly stronger when browsing. Taken all together, browsing or searching did not consistently change the way users were influenced by content, aesthetics or usability.
In additional analyses, differences in main effects were found for English and German participants, and for a company’s registered and non-registered customers.
Added value: Three main conclusions for researchers and professionals can be drawn from this study. First, the proposed assumptions regarding the influence of browsing and searching tasks cannot be supported: The perception of neither usability nor content showed stronger influences on users’ judgements in searching tasks. Aesthetics was not of greater importance when browsing. In fact, to some degree, opposite effects were found. For professionals, this may lead to important conclusions that contrast intuition. For instance, when dealing with highly goal-directed users (e.g. searching for information, comparing different products) perception of usability does not become overly important. Contrary it might be as important to aim for high aesthetical appeal as communicating product information. In sum, results imply that the importance of each construct may not strongly depend on the type of task, but rather differs for spontaneous and more complex evaluations. Content seems to be the strongest driver for building more complex decisions such as revisit or recommendation intention whereas aesthetics is highly influential for spontaneous and overall evaluations of a website. This results in a second implication. Being valued for its simplicity a great number of companies use a measurement of customer satisfaction called the Net Promotor Score (NPS), which consists of one question asking for users' intention to recommend a website (e.g. Reichheld, 2003). However, the current survey suggests that the intention to recommend is highly influenced by content, but not as much by website facets like usability and aesthetics. The two latter constructs seem to be important rather for building an overall impression. Thus, professionals should not only focus on intention to recommend a website but also consider overall impression evaluations such as giving the website a specific grade when measuring success of websites. Third and last, the reported individual differences require further investigations. Website designers should consider adapting their content or aesthetics levels in order to enhance user experience within different cultural backgrounds or for loyal customers.
Web survey bibliography - Germany (361)
- Interviewer effects on onliner and offliner participation in the German Internet Panel; 2017; Herzing, J. M. E.; Blom, A. G.; Meuleman, B.
- Comparing the same Questionnaire between five Online Panels: A Study of the Effect of Recruitment Strategy...; 2017; Schnell, R.; Panreck, L.
- Push2web or less is more? Experimental evidence from a mixed-mode population survey at the community...; 2017; Neumann, R.; Haeder, M.; Brust, O.; Dittrich, E.; von Hermanni, H.
- Social Desirability and Undesirability Effects on Survey Response latencies; 2017; Andersen, H.; Mayerl, J.
- Comparison of response patterns in different survey designs: a longitudinal panel with mixed-mode and...; 2017; Ruebsamen, N.; Akmatov, M. K.; Castell, S.; Karch, A.; Mikolajczyk, R. T.
- Mobile Research im Kontext der digitalen Transformation; 2017; Friedrich-Freksa, M.
- Kognitives Pretesting; 2017; Neuert, C.
- Grundzüge des Datenschutzrechts und aktuelle Datenschutzprobleme in der Markt- und Sozialforschung; 2017; Schweizer, A.
- Article Establishing an Open Probability-Based Mixed-Mode Panel of the General Population in Germany...; 2017; Bosnjak, M.; Dannwolf, T.; Enderle, T.; Schaurer, I.; Struminskaya, B.; Tanner, A.; Weyandt, K.
- Socially Desirable Responding in Web-Based Questionnaires: A Meta-Analytic Review of the Candor Hypothesis...; 2016; Gnambs, T.; Kaspar, K.
- Methodological Aspects of Central Left-Right Scale Placement in a Cross-national Perspective; 2016; Scholz, E.; Zuell, C.
- Predicting and Preventing Break-Offs in Web Surveys; 2016; Mittereder, F.
- Incorporating eye tracking into cognitive interviewing to pretest survey questions; 2016; Neuert, C.; Lenzner, T.
- Geht’s auch mit der Maus? – Eine Methodenstudie zu Online-Befragungen in der Jugendforschung...; 2016; Heim, R.; Konowalczyk, S.; Grgic, M.; Seyda, M.; Burrmann, U.; Rauschenbach, T.
- Comparing Cognitive Interviewing and Online Probing: Do They Find Similar Results?; 2016; Meitinger, K., Behr, D.
- Device Effects - How different screen sizes affect answers in online surveys; 2016; Fisher, B.; Bernet, F.
- Effects of motivating question types with graphical support in multi channel design studies; 2016; Luetters, H.; Friedrich-Freksa, M.; Vitt, SGoldstein, D. G.
- Analyzing Cognitive Burden of Survey Questions with Paradata: A Web Survey Experiment; 2016; Hoehne, J. K.; Schlosser, S.; Krebs, D.
- Secondary Respondent Consent in the German Family Panel; 2016; Schmiedeberg, C.; Castiglioni, L.; Schroeder, J.
- Does Changing Monetary Incentive Schemes in Panel Studies Affect Cooperation? A Quasi-experiment on...; 2016; Schaurer, I.; Bosnjak, M.
- Using Cash Incentives to Help Recruitment in a Probability Based Web Panel: The Effects on Sign Up Rates...; 2016; Krieger, U.
- The Mobile Web Only Population: Socio-demographic Characteristics and Potential Bias ; 2016; Fuchs, M.; Metzler, A.
- The Impact of Scale Direction, Alignment and Length on Responses to Rating Scale Questions in a Web...; 2016; Keusch, F.; Liu, M.; Yan, T.
- Web Surveys Versus Other Survey Modes: An Updated Meta-analysis Comparing Response Rates ; 2016; Wengrzik, J.; Bosnjak, M.; Lozar Manfreda, K.
- Retrospective Measurement of Students’ Extracurricular Activities with a Self-administered Calendar...; 2016; Furthmueller, P.
- Privacy Concerns in Responses to Sensitive Questions. A Survey Experiment on the Influence of Numeric...; 2016; Bader, F., Bauer, J., Kroher, M., Riordan, P.
- Ballpoint Pens as Incentives with Mail Questionnaires – Results of a Survey Experiment; 2016; Heise, M.
- Does survey mode matter for studying electoral behaviour? Evidence from the 2009 German Longitudinal...; 2016; Bytzek, E.; Bieber, I. E.
- Forecasting proportional representation elections from non-representative expectation surveys; 2016; Graefe, A.
- Setting Up an Online Panel Representative of the General Population The German Internet Panel; 2016; Blom, A. G.; Gathmann, C.; Krieger, U.
- Online Surveys are Mixed-Device Surveys. Issues Associated with the Use of Different (Mobile) Devices...; 2016; Toepoel, V.; Lugtig, P. J.
- Stable Relationships, Stable Participation? The Effects of Partnership Dissolution and Changes in Relationship...; 2016; Mueller, B.; Castiglioni, L.
- Will They Stay or Will They Go? Personality Predictors of Dropout in Online Study; 2016; Nestler, S.; Thielsch, M.; Vasilev, E.; Back, M.
- Respondent Conditioning in Online Panel Surveys: Results of Two Field Experiments; 2016; Struminskaya, B.
- A Privacy-Friendly Method to Reward Participants of Online-Surveys; 2015; Herfert, M.; Lange, B.; Selzer, A.; Waldmann, U.
- The impact of frequency rating scale formats on the measurement of latent variables in web surveys -...; 2015; Menold, N.; Kemper, C. J.
- Investigating response order effects in web surveys using eye tracking; 2015; Karem Hoehne, J.; Lenzner, T.
- Implementation of the forced answering option within online surveys: Do higher item response rates come...; 2015; Decieux, J. P.; Mergener, A.; Neufang, K.; Sischka, P.
- Translating Answers to Open-ended Survey Questions in Cross-cultural Research: A Case Study on the Interplay...; 2015; Behr, D.
- The Effects of Questionnaire Completion Using Mobile Devices on Data Quality. Evidence from a Probability...; 2015; Bosnjak, M.; Struminskaya, B.; Weyandt, K.
- Are they willing to use the web? First results of a possible switch from PAPI to CAPI/CAWI in an establishment...; 2015; Ellguth, P.; Kohaut, S.
- Measuring Political Knowledge in Web-Based Surveys: An Experimental Validation of Visual Versus Verbal...; 2015; Munzert, S.; Selb, P.
- Changing from CAPI to CAWI in an ongoing household panel - experiences from the German Socio-Economic...; 2015; Schupp, J.; Sassenroth, D.
- Rating Scales in Web Surveys: A Test of New Drag-and-Drop Rating Procedures; 2015; Kunz, T.
- Mode System Effects in an Online Panel Study: Comparing a Probability-based Online Panel with two Face...; 2015; Struminskaya, B.; De Leeuw, E. D.; Kaczmirek, L.
- Higher response rates at the expense of validity? Consequences of the implementation of the ‘forced...; 2015; Decieux, J. P.; Mergener, A.; Neufang, K.; Sischka, P.
- A quasi-experiment on effects of prepaid versus promised incentives on participation in a probability...; 2015; Schaurer, I.; Bosnjak, M.
- Response Effects of Prenotification, Prepaid Cash, Prepaid Vouchers, and Postpaid Vouchers: An Experimental...; 2015; van Veen, F.; Goeritz, A.; Sattler, S.
- Recruiting Respondents for a Mobile Phone Panel: The Impact of Recruitment Question Wording on Cooperation...; 2015; Busse, B.; Fuchs, M.
- The Influence of the Answer Box Size on Item Nonresponse to Open-Ended Questions in a Web Survey ; 2015; Zuell, C.; Menold, N.; Koerber, S.