Using Response Times to Detect Aberrant Responses in Computerized Adaptive Testing (CT-01-02)
Wim J. van der Linden, Edith M. L. A. van Krimpen-Stoop, University of Twente, Enschede, The Netherlands
In computerized testing, the time between keystrokes can be recorded automatically. The assumption that these records reflect the time the examinee actually needed to process the items and produce a response seems tenable for high-stakes testing. In this paper, response times are used to detect possible aberrancies in the examinees’ behavior during the test. Two simple examples of such behavior are: (1) examinees who know some of the items prior to the test and therefore respond to them more quickly than expected and (2) examinees who show a series of unexpectedly short response times towards the end of the test because they run out of time. The former indicates fraudulent examinee behavior whereas the latter may point at a flaw in the design of the test. Two types of statistical tests were developed to test such hypotheses. Both types are based on the size of the difference between the actual and expected response times by the examinee. Actual response times are recorded during the test. Expected response times are calculated using a statistical model with a parameter for the slowness of the examinee and the time required by the item. The former is estimated from the full set of response times by the examinees during the test, the latter from response times by the population of examinees when the items are calibrated prior to the test. Both types of statistical tests were shown to be effective in an extensive computer simulation study.