Factor Analytic Results

To condense the many questions into a more manageable form,
factor analyses were computed for each section of the questionnaire.

For the section on reasoning skills, only the importance ratings were
analyzed because they were so highly correlated with frequency
ratings. Because frequency ratings were slightly less correlated with
ratings of seriousness and criticality in the other two sections, they
too were analyzed for the questionnaire sections on reasoning errors
and critical incidents.
The reader should bear in mind that the factors resulting from
this analysis should not be construed as representing dimensions of
analytical ability, but rather only as reflecting the dimensions that
underlie faculty perceptions of analytical abilities. These
dimensions merely reflect the extent to which graduate faculty tended
to rate certain skills as about equally important (or equally
unimportant ) , not the degree to which these dimensions represent
“factors of the mind.” Thus, the results presented below are intended
to provide a parsimonious representation of faculty perceptions rather
than a basis for postulating distinct analytical abilities.
Reasoning skills. For the ratings of importance of reasoning
skills, the largest eigenvalues were 16.4, 3.9, 2.4, 1.6, and 1.1, and
the application of a scree test (Cattell, 1966) suggested the
appropriateness of a five-factor solution, which was then rotated
according to the varimax criterion (Kaiser, 1958). The five-factor
varimax rotation accounted for 80% of the common variance. The factor
loadings and communalities are given in Appendix B. Table 7
summarizes the variables that were most instrumental in defining each
factor.
Factor I, which accounted for about a third of the common
variance, was characterized by highest loadings, generally, from
skills involving arguments. Thus, Factor I seems to involve a kind of
critical thinking related to argumentation.
Factor II accounted for about 29% of the common variance, and was
defined primarily by variables related to the drawing of conclusions,
e.g. I generating valid explanations, supporting conclusions with
sufficient data, and drawing sound inferences from observations. The
conclusion-oriented skills that define this second critical thinking
factor would seem to be of a more active or productive nature,
involving the construction of inferences or conclusions, rather than
evaluating the soundness of arguments or inferences, as is the case
for Factor I.
Factors III-V each accounted for a somewhat smaller proportion of
common variance (10% - 15%) than did Factors I and II. Factor III is
best defined by skills related to defining and setting up problems or
analyzing their components as a prelude to solving them. Factor IV is
best characterized by inductive reasoning skills, i.e., the drawing of
conclusions that have some evidential support, but not enough to
indicate logical necessity. Factor V is somewhat difficult to define,
but, by virtue of its two highest loadings, it seems to reflect an
ability to generate alternatives.

Reasoning errors. For the ratings of seriousness of reasoning
errors, the largest eigenvalues were 6.5 and 1.1, and the two factors
accounted for 96% of the common variance. (Frequency ratings were
also factor analyzed and are presented in Appendix B. Because the
results were so similar to the analysis of seriousness ratings, they
are not discussed here.) As shown in Table 8, Factor I, which
explained about 52% of the common variance, was characterized by
loadings from errors involved in the evaluation of evidence, e.g.,
offering irrelevant evidence to support a point. Factor II, on the
other hand, seemed to involve more formal logical errors, particularly
as related to reasoning with more statistically oriented material--for
example, failing to take account of a base rate, failing to recognize
differences between populations and samples, and confusing correlation
with causation.

Critical incidents. As for reasoning errors, only ratings of
the effects of critical incidents, not their frequencies, were factor
analyzed. This analysis yielded eigenvalues of 8.2, 1.6, and 0.9 for
the largest factors, and this three-factor solution accounted for 89%
of the common variance. Table 9 summarizes the results, and the
complete set of loadings is given in Appendix B.
Factor I, explaining about 38% of the conanon variance, was best
defined by highest loadings from such incidents as accepting/
supporting arguments based more on emotional appeal than evidence,
offering nonconstructive or unsound criticism of other students’
presentations, and confusing anecdote and/or opinion with “hard data.”
This factor appears to involve critical facility.
Factor II, accounting for about 34% of the common variance,
appears to involve the ability to consider or to generate
alternatives, being defined primarily by high loadings from such
incidents as accepting conclusions without critically evaluating them,
being able to criticize but unable to suggest better alternatives,
being unable to integrate ideas from various sources, and being unable
to generate hypotheses.
Factor III, defined by such incidents as applying a formula or
rule without sufficient justification, and searching for a complicated
solution when a simpler one is obvious, is difficult to interpret.
One possible characterization might be a kind of rationality or
critical facility that is sometimes referred to as practical judgment
or perhaps “common sense. I’
Scales based on reasoning skill factors. Table 10 gives the
average scores for each discipline on scales composed of the
questionnaire items that best defined the reasoning skill factors
discussed earlier. As is clear, there are substantial differences
among disciplines on these scales. Skills involved in analyzing/
evaluating arguments (Scale 1) were rated as extremely important in
English (m = 4.53), quite important in education (m = 3.83) and
psychology (m = 3.73), and somewhat less important in the other three
disciplines, particularly computer science (m = 2.97).

Critical thinking skills involved in developing or otherwise
dealing with conclusions (Scale 2) were viewed as very important
(means greater than 4.0) in all disciplines except computer science.
Abilities involved in analyzing and defining problems (Scale 3)
were rated as extremely important in computer science (m = 4.05) and
engineering (m = 4.00), but less important in other disciplines,
especially English (m = 2.70).
The inductive reasoning skills reflected on Scale 4 were rated as
moderately important on each of the six disciplines. The skills
composing Scale 5, generating alternatives/hypotheses, were rated very
high in psychology (m = 4.21) and in education (m = 3.93), and as
somewhat less important in other disciplines, particularly computer
science (m = 2.90).