Simultaneous Assembly of Multiple Test Forms (CT-97-13)
by Wim J. van der Linden, University of Twente, Enschede, The Netherlands, Jos J. Adema, PTT Telecom

Executive Summary

In educational testing, often multiple test forms have to be assembled from the same item pool at the same time. A well-known example is a testing organization assembling several forms for administration at different time slots. In computerized testing, the problem of multiple-form assembly occurs if a set of units for a multistage testing system has to be assembled or an item pool has to be reorganized into a pool of testlets for use in a testlet-based adaptive test. These units or testlets have to be informative at different intervals of the ability variable measured by the item pool. In addition, they have to be assembled such that each examinee gets a set of items meeting the same set of specifications.

The problem of multiple-form assembly can be solved in a sequential (one form after the other) or a simultaneous way (all forms at the same time.) Sequential algorithms are fast but have difficulty balancing the item content between forms; simultaneous algorithms balance optimally between forms but generally are too slow to be of practical value. The present paper formulates a heuristic in which a simultaneous assembly problem is reformulated as a series of computationally less intensive two-form problems that can be solved as fast as in a sequential approach. At each step, one of these forms is a form needed. The other is a specially designed dummy or shadow form that is returned to the item pool and whose only task is to create a balance between earlier and later forms in the series. It is shown how the heuristic can be implemented using the technique of 0-1 linear programming.

The papers contain two empirical examples, both based on a former pool of 753 items from the Law School Admission Test (LSAT). In one example, a set of parallel forms was assembled to meet the same target derived from the current specifications for the LSAT. In the other example, the forms were required to meet the same set of content specifications in use for the LSAT but to differ systematically in difficulty. In both examples, all forms met their targets perfectly.

Simultaneous Assembly of Multiple Test Forms (CT-97-13)

Research Report Index