Surveys routinely include rating scales that vary considerably in length, from as short as 2 points (e.g., agree/disagree) to moderate lengths (e.g., strongly agree to strongly disagree) to scales as long as 101 points (from 0 to 100 to measure liking or probability). Because the same type of construct is measured in different surveys by scales of different lengths, there appears to be no consensus among investigators about the optimal length of a scale, defined in terms of measurement accuracy (as indicated by reliability and validity), by administration practicality (e.g., how long it takes a respondent to answer the question), and the ratio of the two (measurement accuracy per unit of administration time, in case a large gain in measurement accuracy comes at a small price in terms of administration practicality). Optimal scale length might vary depending on the familiarity of the topic (perhaps people can effectively use longer rating scales to report more refined views on topics about which they have thought a great deal in the past), the nature of verbal labeling of scale points (perhaps full verbal labeling with optimally-chosen labels allows people to use longer scales more effectively, because the meanings of scale points are clearer) and whether the underlying construct is bipolar or unipolar (since a bipolar scale is essentially two unipolar scales joined at the middle). To explore these issues, we conducted a 3-wave web-based panel survey experiment, in which members of a representative sample of American adults (N = 6,055) were randomly assigned to receive various different versions of 20 rating scales. They were also asked theoretically-relevant criteria questions to assess concurrent validity. This paper will report the results of validity assessments to identify the design of rating scales that maximize validity while also maximizing administration practicality (a separate paper will examine reliability).