Routing Rules for Multiple-Form Structures (CT-02-08)
by Ronald D. Armstrong, Rutgers Business School, Rutgers University

Executive Summary

The last two decades have seen paper-and-pencil (P&P) tests being replaced by computerized adaptive tests (CATs) for many standardized test administrations. CATs have several advantages over conventional P&P tests. As a CAT proceeds, items for administration to an individual test taker are chosen in real-time based on the test taker’s response to previous items; thus, each test is adapted to the test taker’s skill level. As a result of its adaptive nature, CAT can acquire more information about a test taker’s ability while administering fewer items.

As a variation on the standard CAT that adapts the difficulty level of the test after each individual item is administered, researchers have explored testing models where the adaptation takes place after the administration of a pre-assembled bundle of items, commonly referred to as testlets. In the multiple forms structure (MFS) approach to computerized testing, an ordered collection of testlets is assembled. The term “structure” is used to describe this collection of testlets. The structure is divided into stages, and one testlet is administered to the test taker at each stage. The testlet administered to an individual test taker at a given testing stage is based on his or her performance on earlier testlets. One major advantage of this approach is that there are a predetermined number of paths that the test taker may follow through the structure. A path is analogous to a test form in P&P testing. In this way, the MFS approach is a hybrid between the conventional P&P test and a CAT.

This paper concentrates on the routing aspect of the MFS approach. At specified stages of an MFS administration, a decision has to be made as to the next testlet to be included in the test taker’s form. Formal procedures are introduced for matching a test taker’s ability to future items. Routing rules are presented and evaluated with simulation results.
 

Routing Rules for Multiple-Form Structures (CT-02-08)

Research Report Index