Web surveys are commonly used for data collection in empirical social research because they are cheaper, faster, and simpler to conduct than other survey modes. Furthermore, they enable researchers to capture a variety of additional data (so-called paradata) during the survey process such as response times. Measuring response times has by now a long tradition in social psychological research as well as survey research to investigate response behavior and response processes. One key problem, however, is the determination of appropriate thresholds to define outliers; to a certain degree researchers determine them arbitrarily. Until now, there is no scientific consensus with respect to the definition of outliers. In our study we developed an (innovative) two-stage outlier definition procedure for web surveys using paradata. This approach is based on the activity of the web survey while processing, accompanied by an outlier definition that is based on the distribution of the response times. Our web survey (n = 1899) is based on an onomastic sampling approach and contained individual questions as well as grid questions. Moreover, we tested different procedures for dealing with outliers based on the response time distributions. Our analyses show that common outlier definition procedures, which are based on the distributions of response times, provide insufficient results. This implies that they are frequently unable to capture respondents who leave the web survey for a short time so that the response times are biased upwards. Particularly, this circumstance can be observed for grid questions. Altogether, our findings suggest that the two-stage outlier definition procedure is superior to common methods for dealing with outliers that are only based on response time distributions.