Beyond FYA: Analysis of the Utility of LSAT Scores and UPGA for Predicting Academic Success in Law School (RR9905)
Linda F. Wightman, University of North Carolina at Greensboro
Executive Summary
This study was designed to examine questions about the validity and utility of two commonly used predictors of academic success in law school, LSAT score and UGPA, when the criterion measure is grade point average at the completion of law school (cumulative LGPA). The study also examines the multiple correlation of LSAT scores and undergraduate gradepoint average (UGPA) with cumulative LGPA. The question of interest in these analyses is whether there are differences in the strength of the relationship between the two predictors and cumulative LGPA, on the one hand, and between them and firstyear LGPA, on the other. The data are examined for individual schools as well as within six clusters of law schools that are similar to one another on a variety of school and student body characteristics.
A second set of issues examined in the study concerns differential prediction. The study focuses on two questions:
 Do LSAT scores and UGPA predict cumulative LGPA as accurately for nonwhite students as for white students, and for women as for men?
 Are there patterns of over or underprediction among those groups that are different when cumulative LGPA is the criterion than those observed when firstyear LGPA is the criterion?
Methods. Although the preferred method for addressing the first question would be to estimate separate regression equations for each group of interest and then compare slopes and intercepts and proportion of variance explained to evaluate their similarity, the number of nonwhite students at individual law schools was too small to support this approach. Instead, regression systems were constructed using data for white students and the question of whether the equations predicted the law school performance of nonwhite students equally well was investigated. In a similar fashion, regression systems were constructed using data for men and the question of whether the equations predicted the law school performance equally well was investigated. The mean residuals and the correlations between actual and predicted LGPA were used to assess the accuracy of the predictions.
The study is based on longitudinal data collected as part of the LSAC National Longitudinal Bar Passage Study (Wightman, 1998). In order to study prediction patterns, differences in the magnitude and direction of residuals were compared for different levels of LGPA within each group of interest. Specifically, each group was divided into quartiles based on predicted LGPA and the magnitudes of the residuals across quartiles within the group were compared. The studied groups included ethnic groups for which there were sufficient data, resulting in separate analyses using data from Asian American, black, other Hispanic, and white law school students, and from women and men.
The data were collected from 142 law schools, those that reported firstyear grades and had participation rates in the LSAC National Longitudinal Bar Passage Study of at least 20 percent. Only students for whom complete data were available were included in the various analyses, meaning that only students for whom both firstyear and cumulative LGPA were available are reflected in the analyses. Because other data about students for whom cumulative LGPA were missing suggested that these students were not missing at random, LGPA scores were standardized—and the results reported—in two different ways. One method was to standardize using firstyear LGPAs for all students who earned firstyear grades, separately for each school, allowing comparisons between students who graduated and those who did not. The second was to standardize using grades for only those students who persisted and graduated, to enable direct comparisons between standardized firstyear and standardized cumulative LGPAs. Both approaches were included because each provides important information.
All analyses were conducted separately within each law school but, for the most part, the data in this report are summarized by law school cluster. The main reason for using law school cluster as the unit of analysis is to account better for possible differences in the meaning of law school grades among students who attend different law schools. It is also the case, however, that when data are examined within cluster rather than by individual law school, the sample size increases.
The regression models constructed to assess predictive validity focused on the following questions
 Do the regression equations estimated using data from white students predict law school academic performance for nonwhite students about the same as they predict for white students?
 If they do not, are there identifiable patterns of under or overprediction?
 Are the patterns observed between white and nonwhite students when cumulative LGPAs are predicted the same as the patterns observed when firstyear LGPAs are predicted?
Results. The descriptive data for the study reflect the variability that exists in ethnic distribution among the different law school clusters. Schools grouped in Cluster 6 are characterized by a high proportion (nearly half) of nonwhite attendees, compared with other clusters, in which white students make up between 80 and 90 percent of the student body. Other differences exist as well, including, by virtue of the fact that selectivity was one of the variables that contributed to the clustering process, average LSAT scores. Within clusters, white students earned the highest test scores, undergraduate grades, and LGPAs, both firstyear and cumulative. Moreover, among white students who graduated, average standardized cumulative grades where virtually identical to their average standardized firstyear LGPAs. In contrast, a pattern of slightly decreasing law school grades was found among students from other ethnic groups, although the magnitude of the observed differences is small.
The major finding of the study is that LSAT score and UGPA, in combination, were related to cumulative LGPA at approximately the same level as they were related to firstyear LGPA. Correlational data comparing the magnitude of the coefficients obtained when cumulative LGPA is the criterion with the magnitude when firstyear LGPA is the criterion showed that median correlations between predictors and criterion were virtually the same for each predictor. The predictors included LSAT score alone and in combination with UGPA. The data also show that the median correlation coefficients are fairly consistent across the six law school clusters; only in Cluster 6 were the validity coefficients slightly higher for cumulative LGPA than for firstyear LGPA for each predictor.
A second important finding is that the patterns of predictive validity for different ethnic and sex groups do not seem to change regardless of whether the criterion is firstyear LGPA or cumulative LGPA. However, there is an overall tendency for test scores and undergraduate grades to overpredict law school performance for nonwhite law school students. The overprediction was greater when LSAT score was used alone than when it was used in combination with UGPA. And, when used in combination, the two predictors tended to overpredict for nonwhite students with higher predicted LGPAs to a greater extent than it overpredicted for those with lower predicted LGPAs.
The data from the study demonstrate the utility of LSAT scores and UGPAs in the law school admission process beyond the prediction of firstyear academic performance in law school, laying to rest a common criticism of their use. The study shows that the predictive power of these measures extends to law school performance as measured by cumulative law school grades. It does not, however, address the prediction of achievements beyond law school. Moreover, the modest size of the correlations suggests that a substantial amount of the variance in outcomes is left unexplained by the two measures. While law school grades are an important outcome in selecting law school students, they are not the only outcome of interest, although they were the only one examined in this study.
Beyond FYA: Analysis of the Utility of LSAT Scores and UPGA for Predicting Academic Success in Law School (RR9905)
Research Report Index
