Analytical Reasoning Skills Involved in Graduate Study

Perceptions of Faculty in Six Fields
Despite the complexity of human cognitive abilities, standardized
admissions tests have tended to focus almost exclusively on the
measurement of broadly applicable verbal and quantitatice aptitudes.
One criticism of such omnibus verbal and quantitative ability measures
is that they provide only limited descriptions of students' academic
strengths and weaknesses, and that they do not therefore adequately
reflect test takers' differential development in other important
cognitive areas.
In 1974 the GRE Board approved a plan to restructure the GRE
Aptitude Test in order to allow examinees to demonstrate a broader
range of academic skills (Altman, Carlson, & Donlon, 1975). A survey
of constituents revealed that, of several possible new areas of
measurement (e.g., abstract reasoning, scientific thinking, and study
skills), graduate faculty, administrators, and students were most
receptive to assessing analytical or abstract reasoning skills
(Miller & wild, 1979). Developmental activities then followed and,
after careful psychometric study of several alternative analytical
item types, four distinct kinds of items were selected for the new
analytical section of the GRE Aptitude Test, which was introduced
operationally in the 1977-78 testing year. Graduate institutions were
cautioned against using the scores from the new analytical section
until further evidence could be generated on the validity of the new
measure. Subsequently, the administration of the new measure to large
numbers of examinees under operational conditions enabled the further
collection of information about the new measure.
Some research strongly suggested the promise of the analytical
section: it appeared to measure an ability that was distinguishable
from the verbal and quantitative abilities measured by the test
(Powers & Swinton, 1981), and the score derived from it was related to
successful performance in graduate school (Wilson, 1982).
Unfortunately, however, further research suggested serious problems
with the two item types (analysis of explanations and logical
diagrams) that comprised the bulk of the analytical section.
Performance on these item types was shown to be extremely susceptible
to special test preparation (Swinton & Powers, 1983; Powers & Swinton,
1984) and to within-test practice (Swinton, wild, & Wallmark, 1983).
Consequently, in 1981 the two problematic item types were deleted from
the test, and additional numbers of analytical reasoning and logical
reasoning items, which constituted a very small part of the original
analytical measure, were inserted.
The most recent research on the General Test (Stricker & Rock,
1985; Wilson, 1984) has given us some reason to question both the
convergent and the discriminant validity of the two remaining item
types. Two currently used GRE analytical item types reflect only a limited
portion of the reasoning skills that are required of graduate
students. The most notable omission is the assessment of inductive
reasoning skills, i.e., reasoning from incomplete knowledge, where the
purpose is to learn new subject matter, to develop hypotheses, or to
integrate previously learned materials into a more useful and
comprehensive body of information. Thus, it seemed, the analytical
ability measure of the GPE General Test might be improved through
further effort.
The objective of the study reported here was to generate
information that might guide the development of future versions of the
GRE analytical measure. More specifically, the intention was to gain
a better understanding of what reasoning (or analytical) skills are
involved in successful academic performance at the graduate level, and
to determine the relative importance of these skills or abilities both
within and across academic disciplines. It was thought that this
information might be especially useful for developing additional
analytical item types.
As mentioned earlier, the initial version of the GRE analytical
ability measure was developed after a survey had suggested the
importance of abstract reasoning to success in graduate education.
This survey, however, was not designed to provide any detailed
information on the importance of specific analytical skills, as was
the intention here.