Testing for Structure Recognition (RR-98-02)
by Gilbert E. Plumer

Executive Summary

The nontechnical ability to identify or match argumentative structure is considered by many to be an important reasoning skill. Instruments that have questions designed to measure this skill include major standardized tests for graduate school admission, for example, the Law School Admission Test (LSAT), the Graduate Record Examinations (GRE), and the Graduate Management Admission Test (GMAT). Writers and reviewers of such tests need an appropriate foundation for developing such questions—they need a proper representation of phenomenological argumentative structure—for legitimacy, and because these tests affect people’s lives. This paper attempts to construct an adequate and appropriate representation of such structure, that is, the logical structure that an argument is perceived to have by mature reasoners, albeit ones who are untrained in logic.

Typical questions on the standardized tests mentioned that ask the test taker to identify or match structure consist of a short argumentative passage, a question stem on the order of either (I) “The argument’s method of reasoning is” or (M) “The pattern of [flawed] reasoning in the argument above is most similar to that in which one of the following arguments?” and five answer choices. Since all answer choices must be cast in ordinary nontechnical prose, questions of type (I) generally concern only the grosser features of an argument’s structure. Questions of type (M), however, can pertain to much more subtle features (since the test taker is not asked to explicitly identify them), and it is this type that constitutes our focus.

Regarding formal or deductive structure, the principle that is argued for is this: In the construction and defense of questions of type (M), when a question stem emphasizes reasoning structure by the use of a phrase such as “pattern of reasoning” or “parallel reasoning,” more weight can legitimately be assigned to reasoning structure than to surface logical structure and the structure of the argument’s terms. Yet these latter must still be taken into account in determining overall (phenomenological) argumentative structure. In this way we adopt the principle that the proper or “best formal representation will be the one that exhibits the most structure.”

It is undeniable that in ordinary life we routinely evaluate arguments as invalid or fallacious. But if a certain theoretical view that alleges a strong asymmetry between showing validity and showing invalidity were right, many, if not all, of these judgments would be illegitimate. A case is developed against this view, and this helps to legitimate questions on an exam like the LSAT that ask test takers to match flawed patterns of reasoning.

Regarding the informal or nondeductive structure of an argument, my thesis is that it can include any of the argument’s general elements that figure in the purported cogency of (that function in purporting to advance the conclusion in) any pattern of reasoning. The proper representation of a given argument’s phenomenological argumentative structure will include these elements whether or not the given argument exhibits the pattern of reasoning in question. This point regarding informal structure corresponds to the point before regarding formal structure that such features as surface logical structure and the structure of the argument’s terms need to be taken into account. But also as before, more weight can legitimately be assigned to the general elements that actually figure in the purported cogency of the given argument. It is shown how this approach has more substance to it than might be evident. In the first place, the approach rules out purely syntactical features, such as the location of the argument’s conclusion, as immaterial. Secondly, it coheres well with the established tradition in informal logic that the cogency of a nondeductive argument is largely a matter of its form. Finally, it is shown how the approach can be used to help explain both the statistical success and failure of nondeductive test items of type (M).

Testing for Structure Recognition (RR-98-02)

Research Report Index