A Review of the LSAT Using Literature on Legal Reasoning (CT-97-08)
by Gilbert E. Plumer
In 1995 the Law School Admission Council implemented a five-year research agenda to examine the feasibility and advisability of computerizing the Law School Admission Test (LSAT). Part of this agenda is the Skills Analysis study, the general purpose of which is to systematically identify skills important for success in law school. As one component of the Skills Analysis study, research using current literature on legal reasoning was conducted with the goals of (a) determining what skills are most important in good legal reasoning according to such literature, (b) determining the extent to which existing LSAT item types and subtypes are designed to assess those skills, and (c) suggesting test specifications or new or refined item types and formats that could be developed in the future to assess any important skills that appear [by (a) and (b)] to be measured in a limited or minimal way by the current LSAT. So far as can be determined, such systematic research using legal reasoning literature has never been previously conducted.
Broadly speaking, the findings validate the current LSAT, with its item types and subtypes and their mix. The skills identified as important in recent literature on legal reasoning are primarily ones of reasoning and text comprehension and analysis, which constitute the focus of the current LSAT. However, at the level of specific skill categories, at least as they can be derived from this literature, there are areas relative to which changes in the LSAT should be considered. These possibilities, which are not conclusive although they are informed and educated, can be summarized in tabular form as follows.
LSAT skills assessment
A fair number of potential new test specifications and item types and formats are suggested in this report as possible ways of more closely aligning the skills that the LSAT is designed to assess with skills identified as important in legal reasoning literature. For example, a specification for Logical Reasoning inference, assumption, and technique subtypes could perhaps be developed whereby the item's credited response is always a conclusion, assumption, or analysis of an analogical argument. However, it should be noted that literature on legal reasoning is only one of a number of possible sources to use in identifying skills that the LSAT might assess. Indeed, other parts of the Skills Analysis study are focused on law school teaching materials and methods and on task analyses of law school courses, and this research may reveal other skills. Moreover, identifying skills to be assessed is only one of a number of factors that determine the makeup of the LSAT; cost of producing items, for instance, is another factor. Therefore, before any changes in the LSAT are actually instituted, an investigation that takes all such considerations into account would have to be undertaken.