A Formal Characterization of and Some Alternatives to Sympson-Hetter Item-Exposure Control in Computerized Adaptive Testing (CT-02-05)
by Wim J. van der Linden, University of Twente, Enschede, The Netherlands

Executive Summary

In computerized adaptive testing (CAT), it is necessary to implement item-exposure control procedures to assure that the test items with the statistical characteristics that are most attractive to the item selection algorithm are not overused. The overuse of such items may compromise their security. Currently, the Sympson-Hetter (SH) method is the most popular method of item-exposure control in CAT. The method is based on a probabilistic experiment that is used to determine if an item that is selected by the CAT algorithm should actually be administered.

The implementation of the SH item-exposure control method requires that a large number of computer simulations be run in order to determine what the values of the required control parameters should be. For example, it is not uncommon to have to run 100–200 simulation studies before the values of the control parameters are determined.

In this paper, several formal properties of the SH method are presented that help us explain why the adjustment process can be slow and does not guarantee that acceptable values of the control parameters will be obtained. These properties are then used to derive several methods of item-exposure control that can serve as an alternative to the SH method. The behavior of these alternatives was studied for adaptive tests simulated from an item pool from the Law School Admission Test (LSAT). One of the alternatives showed particularly attractive behavior and resulted in acceptable values for the control parameters for all items in a fraction of the number of computer simulations required for the SH method.


A Formal Characterization of and Some Alternatives to Sympson-Hetter Item-Exposure Control in Computerized Adaptive Testing (CT-02-05)

Research Report Index