LSAT Listening Assessment: Theoretical Background and Specifications (RR-03-02)
by Kenneth Olson
It is clear that listening is one of the skills that is important for success in law school, a conclusion that has recently been bolstered by the Law School Admission Council (LSAC) Skills Analysis Study. Since the Law School Admission Test (LSAT) seeks to test such skills, a prima facie case can be made that the test should contain a measure of listening. A first step in constructing such a measure is determining what it is going to test, which is the subject of this report.
A perusal of the literature reveals that there is no generally accepted definition of listening despite numerous proposals. We argue that the attempt to formulate a definition is at best premature and, fortunately, not required for our purposes. What we need is rather to identify a particular “construct” (a word that makes no attempt to hide its essentially factitious nature), in this case, a set of skills relevant to listening in the law school context. Useful information for this purpose is provided by the Skills Analysis Study. For example, it emerges from this study that dialogue in one form or another plays a larger role in legal education than does the traditional lecture. Hence, the skills involved in following dialogues would be expected to assume some prominence in our construct.
It is also helpful, and in some cases essential, to have in mind a basic model of the listening process, and here we turn to cognitive psychology. Much of the research related to listening in cognitive science focuses on the role of memory. This focus is a consequence of the information processing model of the mind employed by most cognitive scientists, in which information is viewed as being held in one or more memory systems at different stages of processing. The two most important memory systems for our purposes are short-term memory and long-term memory. Short-term memory, also called working memory, contains what is present to the mind at a given time. It is severely limited in capacity, holding roughly seven items at a time. Long-term memory is what we normally think of as memory, a deep repository of information that we can call up more or less at will. Information given in one form may need to be transformed into another in order to be stored in the appropriate memory system. The critical point here is that the actual words we hear are generally held in short-term memory just long enough to decode them to get their meaning. It is this meaning that is stored in long-term memory, while the actual words are discarded.
This account depends on a distinction between words and their meanings, and it is to this that we turn next. Most people probably think of the meaning of a word or longer expression as its dictionary definition, that is, as more words. But a little reflection should suffice to show that this only leads us around in a circle. A better way of thinking about meaning is to view it as whatever it is that enables us to connect words to their references. We have grasped the meaning of the word “tree,” for example, when we are able to identify objects around us as trees. Semanticists typically deploy an arsenal of abstract objects to account for meaning. For example, a sentence is held to express a proposition, which is often identified in turn with a set of “possible worlds,” the set of worlds in which it is true. It is clear that, in general, a sentence in no way resembles the proposition it expresses.
Exactly how meanings are represented in the mind is not fully understood. But there is good reason to believe that they are stored in a form quite different from that in which they were originally received. It is highly doubtful that even the integrity of propositions is always preserved. Johnson-Laird theorizes that the information we obtain linguistically is ultimately used to build up “mental models” that resemble structures in the world rather than structures in language. For example, a description of a house is given to us in a series of sentences, each of which has a certain syntactic structure, but we use these sentences, and the propositions they express, to build up a model of the house that is more or less isomorphic to the house itself.
A model of listening must also take into account the pragmatic aspects of communication. Normal listening takes place in a context which includes the speaker and hearer. Language is designed to exploit this context through the use of so-called indexical expressions, words like “I,” “you,” “here,” and “now,” that denote different things on different occasions of use, depending, for example, on who is speaking, to whom, where, and when. Oral communication makes greater use of such features than does written communication, because it is “situated,” whereas written communication is more or less disembodied.
This leads to a more general discussion of the differences between listening and reading. Both require us to build up a kind of theory or model of the content. In neither case is the process merely passive. It requires inference and hypothesis formation. But reading and listening place different demands on memory. We can read at our own pace and, if we start to lose our way, we can always look back to see where we went wrong. It also helps to know where the author is heading, and we can look ahead to find this out. None of these features is present in listening. The speaker sets the pace, and we have to keep up as best we can. Moreover, oral language differs from written language in important ways. Not only does it make greater use of indexicals, but it is typically less complex, eschewing sentences with many subordinate clauses in favor of simpler ones. Anyone who has listened to a paper read aloud knows how hard it is to follow compared to an off-the-cuff talk. At least part of the difference comes from the fact that written language is meant to be seen and not heard.
We draw a number of lessons from this discussion. One of the most important is that a listening test should focus more on the ability to keep up with a monologue or dialogue without getting lost, and less on verbal recall. Stimuli should be challenging enough to require inference and hypothesis formation. Asking about exact wording is pointless, because a good listener will usually discard it. Stimuli should be couched as much as possible in oral, as opposed to written language. To ensure this, stimuli should be generated orally, perhaps from notes, rather than read from a prepared text.
We conclude by describing a tentative set of specifications for a listening test, which were used to make up the sections used for field testing in 2002–2003. These specifications are not intended to be final, and it is likely they will be amended in light of field test data and further experience.