by David J. Scrams, Johns Hopkins University and Deborah L. Schnipke, Law School Admission Council

Executive Summary

Response accuracy and response speed provide very different measures of performance. Psychometricians have tended to focus on accuracy with the goal of characterizing test takers on the basis of their ability to respond correctly to items from a given content domain. With the advent of computerized testing, response times can now be recorded unobtrusively during operational tests, and this new source of data may provide additional information about test takers, including, possibly, law school admission test takers in the future.

The present work addresses how response accuracy and response speed are related. First, a theoretical model of item processing is introduced. This model assumes that test takers select a particular way of solving the problem at hand and decide how much time to spend on an item. These choices (along with test taker, item, and strategy characteristics) determine response accuracy. Next, a series of analyses addressing the relationship between observed response time and observed response accuracy is conducted using data from operational computerized tests of verbal, quantitative, and reasoning skills.

Some analyses show positive relationships between response accuracy and response time, but most analyses suggest no relationship at all. These results are interpreted as evidence that response accuracy and response speed measure different test taker characteristics.

Test speededness is discussed in light of the present findings. Test speededness is the extent to which some test takers run out of time and receive lower scores than they would have received had more time been available. Because response accuracy and response speed seem to measure different test taker characteristics, test speededness may be a serious problem if only accuracy is of interest. For example, because speed and accuracy seem to be unrelated, strict time limits may cause some test takers to run out of time and receive lower scores than they should.

The possibility of incorporating measures of response speed into ability estimation is also discussed. Increasing the importance of response speed may have unknown effects on test takers' strategy selection, and we know very little about the relationship between measures of response speed and outcome measures such as success in future education. For these reasons, we are not recommending that response speed be incorporated into ability estimation at this time.
 

Making Use of Response Times in Standardized Tests: Are Accuracy and Speed Measuring the Same Thing (CT-97-04)

Research Report Index