Making Use of Response Times in Standardized Tests: Are Accuracy and Speed Measuring the Same Thing (CT-97-04)
by David J. Scrams, Johns Hopkins University and Deborah L. Schnipke, Law School Admission Council
Response accuracy and response speed provide very different measures of performance. Psychometricians have tended to focus on accuracy with the goal of characterizing test takers on the basis of their ability to respond correctly to items from a given content domain. With the advent of computerized testing, response times can now be recorded unobtrusively during operational tests, and this new source of data may provide additional information about test takers, including, possibly, law school admission test takers in the future.
Some analyses show positive relationships between response accuracy and response time, but most analyses suggest no relationship at all. These results are interpreted as evidence that response accuracy and response speed measure different test taker characteristics.
Test speededness is discussed in light of the present findings. Test speededness is the extent to which some test takers run out of time and receive lower scores than they would have received had more time been available. Because response accuracy and response speed seem to measure different test taker characteristics, test speededness may be a serious problem if only accuracy is of interest. For example, because speed and accuracy seem to be unrelated, strict time limits may cause some test takers to run out of time and receive lower scores than they should.
The possibility of incorporating measures of response speed into ability estimation is also discussed. Increasing the importance of response speed may have unknown effects on test takers' strategy selection, and we know very little about the relationship between measures of response speed and outcome measures such as success in future education. For these reasons, we are not recommending that response speed be incorporated into ability estimation at this time.