Equating an Adaptive Test to a Linear Test (CT-04-01)
by Wim J. van der Linden, University of Twente, Enschede, The Netherlands

Executive Summary

A popular method of reporting scores on a computerized adaptive test (CAT) is to convert each test taker’s CAT score to a number-correct score on a linear reference test; for example, a previous paper-and-pencil form or a special form assembled to the same content specifications as the CAT. This conversion is accomplished through a statistical process called test equating. Traditionally, two methods are used to carry out the equating that this score reporting method requires: (1) the equipercentile equating method, which determines the equated score by matching the percentiles for the CAT scores to those for the number-correct scores on the reference test, and (2) the test characteristic curve transformation equating method, which applies item response theory (IRT) to determine the reference test score that would correspond to each test taker’s CAT score. (Note that IRT is a mathematical model that is commonly used to analyze test data.)

In this paper, it is argued that these two methods are necessarily biased because they use a single equating transformation for an entire population of test takers and therefore have to compromise between the observed-score distributions of individual test takers. Two new methods for the equating of an adaptive test to a linear test are presented, which allow for the differences between these individual observed-score distributions because they condition on the ability level of the individual test takers.

The four methods were evaluated empirically in a study with the difference between the distributions of the equated score and the actual observed score on the reference test as the success criterion. The four methods showed comparable degrees of precision but, as predicted, the two traditional methods were strongly biased while the two new methods were unbiased.

Equating an Adaptive Test to a Linear Test (CT-0-4-01)

Research Report Index