There have been many response formats developed to measure issue of feature importance. Some have used checklists ("select all that are important"), others have used yes-no formats, and still others have used bipolar and unipolar rating scales. Using a web-based survey and across o number of different topic areas, we explored how a number of these different response formats functioned. The multiple response format required less time to complete, but the unipolar importance measure appeared to yield higher criterion-related validity.