Test Design and Speededness (RR-07-01)
by Wim J. van der Linden, University of Twente, Enschede, The Netherlands
Test speededness may be defined as the degree to which test takers do not have sufficient time to respond to all test items (questions). So far, test speededness has been addressed in test design by means of a rather intuitive approach. The main reason for this was that the degree of speededness of a test could only be assessed after it had been administered operationally. Typical measures for the degree of speededness of a test were based on counts of the items the test takers omitted or randomly guessed at toward the end of the test.
In this research, we explored a model-based definition of the degree of speededness as the probability of not completing the test while answering the items at an acceptable level of speed. This definition allows us to calculate the degree of speededness of any test before its operational use, provided the items have been calibrated using a response-time model, such as the one developed in an earlier project for the Law School Admission Council (LSAC).
We show how the new measure of speededness can be used to design tests with a preestablished degree of speededness. The procedure is demonstrated for a variety of test design problems, including the assembly of a new test form that is equally speeded as an old form, the assembly of a new test to fit an existing time slot, the design of a test with a change in the content specifications but a previous degree of speededness, and the design of test accommodations with extended time. The common step in the solutions presented for each of these problems is the introduction of a constraint on the time parameters during item selection that controls the degree of speededness of the test.