Traditional testing of survey instruments involves drafting an instrument and pretesting it with respondents who fill it out, if possible. New instruments often misfire, as the researcher learns during initial testing that the instrument does not use appropriate terms for the respondent or suffers from a naive view of the subject matter. We highlight our work on a national web survey on R&D activity of 423 state agencies. In this paper, we describe the early stage scoping interviews we used with a "sliding scale approach." This technique allowed efficient and cheap questionnaire development using a total of six one-day visits to five states, supplemented with phone calls to additional states and to earlier respondents.

To assist survey designers in understanding the sliding scale approach, we outline three principles of early stage interviewing. The first principle is "Start fresh." This principle is implemented with a very small number of exploratory interviews to understand the substantive issues from the respondent's point of view. The second principle is to "Learn the respondent's language." During a second small group of interviews, a paper prototype is used to check the language in a draft of the instrument. The third principle is to "Know thy user, for it is not you." The final set of interviews covers usability and navigation issues

for a nearly-complete instrument. The sliding scale refers to the changes across pretest interviews -- the portion of the interview devoted to substantive issues starts near 100 percent and diminishes to near zero across the timeline of interviews, while the portion of the interview dedicated to administrative and navigation issues starts near zero and rises to nearly 100 percent. The paper shows how early stage scoping interviews can utilize these three principles to minimize the resources needed to maximize measurement quality while minimizing respondent burden.