Relevance & Research Question:

Previous research found that the current motivational state of subjects represents an important predictor for the quality of online surveys (e.g. Harper, Raban, Rafaeli & Konstan, 2008). Thus, measuring the motivation in real-time while subjects work on a survey could enable us to provide adaptive motivational cues and improve answer quality. Moreover, automated labelling of answers with motivation measures could help analysing data quality. However, previous methods of measuring current motivation are limited to self-reports or indirect measures like cognitive association tasks (see Touré-Tillery & Fishbach, 2014 for a review) – those methods tend to be biased and time-consuming. As surveys often contain open text answers, we propose to use indices of typing behaviour (e.g. speed, pauses, corrections) as a source of information about current motivation. This study investigates whether analysing the typing behaviour in surveys can help to explain variance of the motivational state and the quality of open text answers.

Methods & Data:

Data acquisition is still in progress. 50 subjects will take part in a correlation study. As a cover story, subjects are asked to evaluate an online learning environment about website programming for about one hour. This includes four open answer tasks (e.g. “How would you change the page design?”) at different time intervals. Before each task, the depending variable current motivation is assessed with 5 items (e.g. “I think, this task is fun”). The answer quality is rated manually including length, content variance and number of propositions. Typing behaviour is recorded by a JavaScript framework. Multiple regression models are used to check for relations between typing indices and motivational states.

Results:

As data acquisition is still in progress, results are not yet available. However, an explorative pre-study revealed medium correlations between typing speed and answer quality (r = -.369, p<.01) as well as between the number of corrections and motivational state (r = .540, p<.01).

Added Value:

Analysing typing behaviour is an unobtrusive, non-reactive, low-cost and objective method that could help predicting motivation and labelling answers in online surveys. This study investigates the practical applicability and theoretical validity of this idea.