Outlier Detection in High-Stakes College Entrance Testing (CT-01-08)
Rob R. Meijer, University of Twente, Enschede, The Netherlands
Though the development of computerized adaptive testing (CAT) has resulted in more efficient educational and psychological measurement, it has also generated new practical and theoretical problems. One theoretical problem that arises is the identification of item score patterns (correct and incorrect responses) for particular test takers that do not conform to what would be expected based on the mathematical model being applied. An example of such an aberrant item score pattern would be that of a test taker who answered many easy items incorrectly and many difficult items correctly. If a test taker has an item score pattern that does not fit, the pattern is unlikely to give valuable information about the test taker's ability, but may point toward other behavior during the test. Explanations of aberrant item score patterns in CAT differ from those in paper-and-pencil testing. For example, in paper-and-pencil testing, answer copying may result in unexpected item scores, whereas in a CAT, direct answer copying is impossible because different test takers are administered different tests.