The Impact of Local Dependencies on Some LSAT Outcomes (SR-95-02)
by Lynda M. Reese, Law School Admission Council

Executive Summary

This study explored the impact of various degrees of violations of the item response theory (IRT)-local independence assumption on the Law School Admission Test (LSAT) calibration and score distribution estimates. Initially, results from the LSAT and two other tests were investigated to determine the approximate state of local item dependence (LID) found in actual test data. Yen’s Q3 statistic was employed for this purpose. Based on these analyses, four levels of LID were defined and associated data sets generated. Estimates from the simulated data were compared to their corresponding LSAT generating values in order to analyze the effects of the LID on IRT calibration and score distribution estimates.

The results indicated that LID causes low scores to be underestimated and high scores to be overestimated. Effects on item and ability parameter and test characteristic curve estimation were clearly demonstrated. Score distributions were also markedly changed by the introduction of LID. The effects observed were mainly problematic for high LID levels. Deficiencies in Yen’s Q3 statistic were also observed.

The Impact of Local Dependencies on Some LSAT Outcomes (SR-95-02)

Research Report Index