The Validity of Law School Admission Test Scores For Repeaters: A Replication (RR-90-02)
by Linda F. Wightman

Executive Summary

Fair and accurate treatment of multiple test scores for law-school applicants who take the Law School Admission Test more than one time was the subject of a number of research studies during the mid 1970s. The consistent conclusion from those studies is that the simple arithmetic average of multiple scores results in the most valid score and provides the best prediction of subsequent law school performance for repeat test takers. More than a decade has passed since the last repeater study was completed. During the intervening years, the content of the LSAT has changed substantially, test disclosure was introduced, coaching courses have increased in number and visibility, and many cohorts of law-school students have come and gone. The current study is a partial replication of an earlier study (Pitcher, 1977) so as to reexamine the validity and predictive accuracy of the different scores that are presented by repeat test takers. In particular, the study examined the validity of using (1) initial, (2) most recent, (3) highest, and (4) average score for repeaters.

The study includes only those schools that enrolled 50 or more first-year students who had taken the LSAT on more than one occasion, in order to assure stability in the validity estimates. Forty-six schools are included in the study sample. In addition to validity data, the study also provides descriptive data comparing one-time test takers with repeat test takers. Consistent with the earlier studies, repeat test takers tend to earn lower LSAT scores than one-time takers regardless of whether initial, most recent, highest, or average score is considered. Repeaters and one-time test takers tend to perform comparably in their undergraduate academic work, but one-time takers tend to earn higher first-year averages in law school than do repeaters.

Prediction equations developed from data from repeaters only are compared with prediction equations developed from data from first-time takers only. The least amount of difference between the equations is found when the average score is used for repeaters, while the largest number of differences is found as a result of using either highest or most recent score for repeaters. That is, when highest or most recent score is used, a prediction equation based on repeaters and one timers combined tends to slightly overpredict future law school performance for repeaters by about one fifth of the combined group standard deviation, or approximately one to 1.3 points on the standardized first-year grade scale.

Validity coefficients are presented for repeaters alone, for one-time takers alone, and for the total group, using first-year average in law school as the criterion variable and UGPA alone, LSAT alone, and UGPA and LSAT in combination as predictors. There is no evidence that including repeat test takers in the group results in lower validity estimates. The data show that using average score for repeaters tends to result in validity coefficients that are equal to or higher than the coefficients obtained using any of the other score options, but the differences are small. The data also show that using the combination of LSAT and UGPA results in higher validity coefficients than using either predictor alone. These results hold for each separate group as well as for the total combined group.

A primary practical concern for score users is "Which of the scores presented by repeat test takers will most accurately predict subsequent performance in law school?" Comparison of the predicted first-year average with the actual first-year average supported the advice that has historically been given. That is, in general, the arithmetic average of LSAT scores is the best predictor of performance in law school for repeat test takers. However, the data in this study, as in the previous study, demonstrate that this is not true for every applicant and that the differences obtained from using alternative score options are not dramatic. The data in the present study demonstrate that across law schools, the average score is the best predictor for the majority of schools and the initial score is the next best predictor.

As in previous repeater studies, the data support the use of the simple average score for law-school applicants who present multiple test scores. A primary advantage of the average score is that it makes use of all the data that are available about the applicant. Further, no other score has been found to be superior to it. The data in this and previous studies also underscore the need to consider individual circumstances when evaluating scores for repeat test takers. That is, although the aggregate statistics confirm that, overall, using the average score for repeaters provides higher validity coefficients and more accurate prediction of first-year grades, there are individual test takers for whom this is not the case. As important, there are examples in which one of the other score options would provide more accurate information about an individual applicant. In some instances, the initial score provides the best prediction, and intervening preparation results in a higher score that overpredicts subsequent law school performance. In others, the initial score does not accurately reflect the ability of the test taker and the test taker self-selected to repeat the test in order to obtain a more accurate reflection of his or her ability.

If a general rule that will be most fair to the majority of law-school applicants is to be applied, the data continue to support the recommendation of the average score for general use. Regardless, score users need to be sensitive to individual differences among test takers and evaluate multiple scores in the context of additional information.

The Validity of Law School Admission Test Scores For Repeaters: A Replication (RR-90-02)

Research Report Index