Relevance & Research Question:

When measuring concepts cross-culturally, measurement equivalence is essential in yielding meaningful results. As part of the questionnaire design processes, cross-cultural cognitive interviewing (CCCI) is commonly used to identify possible threats to measurement equivalence. CCCI is a version of standard cognitive interviewing used for assessing the cognitive processes behind the response process in personal interviews. For pragmatic reasons, CCCI is usually carried out with small sample sizes and involves the use of different probing techniques. The relatively new tool of online probing (OP) combines features of CCCI with the advantages of an online survey, achieving a greater sample size and broader coverage of concepts. This study investigates whether OP can be an efficient alternative to CCCI in developing cross-cultural questionnaires by comparing response quality and substantial results.

Methods & Data:

In this study both CCCI and OP are applied in the questionnaire design process of developing a cross-cultural questionnaire on justice attitudes. Existing items that measure justice attitudes will be presented to respondents in CCCI and OP followed by comprehension and category-selection probes. A convenience sample of university students and employees with an international background will be recruited focusing on the languages German, English and Russian. The response quality of both methods will be evaluated by comparing nonresponse as well as response length. In a second step we will analyze whether both methods identify similar threats to measurement equivalence.

Results:

The study is work in progress; preliminary results will be available for the conference.

Added Value:

While CCCI as a method yields a higher level of interactivity and is assumed to produce higher quality data, OP can be implemented in web surveys which allow for larger sample sizes in the evaluation of threats to cross-cultural equivalence. In comparing results of both methods we study the relative benefits of both methods for the assessment of cross-cultural equivalence. We expect to find OP as a useful additional technique in the development of questionnaires, especially in cross-cultural settings.