THE EFFECTS OF EVIDENCE IN PERSUASIVE COMMUNICATION
James C. McCroskey*
In recent years the importance of evidence in persuasive communication has come into serious question as a result of experimental research. While Cathcart1 and Bettinghaus2 found that evidence significantly improved the persuasiveness of speeches, at least four other studies have found that clearly identified evidence was no more persuasive than assertion or material presented without citation of source.3
In the most extensive experimental study on evidence to date, Dresser found that "satisfactory evidence" was no more persuasive than evidence attributed to unreliable sources, irrelevant evidence, or internally inconsistent evidence.4 In addition, Dresser found that the college students used in his experiment were not even able to perceive irrelevance or internal inconsistency in evidence.5
After reviewing this body of experimental research, Gregg concluded that "the audience reaction to an argument may have little or nothing to do with whether the argument includes fully documented or completely undocumented evidence, relevant or irrelevant evidence, weak or strong evidence or any evidence at all."6 Thus, the usually assumed psychological relationship of evidence to conclusion is questionable. This led Gregg to advance several additional psychological relationships which may better explain the role of evidence in persuasion.7 Three of these are relevant to the present study.
1. Conclusion-to-evidence. This is the opposite of the usual evidence to conclusion theory. It suggests we draw conclusions and then judge the quality of evidence used to support them. If we agree with a proposition we view evidence which supports it as good or sound, while if we were to oppose the proposition we would view that the same evidence as poor or unsound. This theory has not received much attention in previous experimental research.
2. Advocate-to-evidence. This theory suggests that we view evidence presented by those we like and respect as good or sound, while if that same evidence were presented by someone we dislike or do not respect, we would consider it poor or unsound. This theory is supported by the experimental research on ethos.8
3. Evidence-to-advocate. This is the opposite of the above. It
suggests that we look at an advocate's use of evidence and then draw conclusions
about his competence. Like the concept of "conclusion-to-evidence"
reasoning this theory has received little specific attention in previous
Hypotheses based on these theories and the traditional evidence-to-conclusion
theory were tested in the present study. Specifically, the following hypotheses
1. Auditors can perceive qualitative differences in use of evidence in persuasive communication.
2. Auditors initially favorable to a speaker's position perceive evidence usage to be of higher quality than those who are initially unfavorable.
3. Good use of evidence increases perceived authoritativeness of a speaker.
4. Good use of evidence increases perceived character of a speaker.
5. Speeches which include good use of evidence produce a greater attitude
shift than those which do not.
Speeches on two topics were developed, one advocating abolition of capital
punishment and the other advocating federal control of education. Two versions
of each speech were developed, one including "evidence," and
the other including "no evidence." For the purposes of this study
"evidence" was operationally defined as: Factual statements
originating from a source other than the speaker, objects not created by
the speaker, and opinions of persons other than the speaker which are offered
in support of the speaker's claims. In the "evidence" versions
of the speeches sources of facts and opinions were cited and qualified.
In the "no evidence" speeches no sources were cited for any material
included. Authoritative opinions used in the "evidence" speeches
were also used in the "no evidence" speeches, but were worked
into the speeches so that they appeared to be merely statements made by
the speaker. Statistical evidence used in the "evidence" speeches
was "generalized" in the "no evidence" speeches. For
example, the quotation:
According to Bulletin 1957-13 of the U.S. Office of Education,
entitled "Provisions Governing Membership on Local Boards of Education,"
the U.S.O.E. national survey of school boards determined that 23.8% of
the nation's school boards include people who are not even high school
graduates. In the South the figure is 41%.
was generalized to "many school boards in the United States include people who are not even high school graduates."
It should be emphasized that the above distinction between "evidence" and "no evidence" speeches is an operational one. The construction of a speech including nothing that anyone would consider evidence would be extremely difficult, if not impossible, and would not resemble speaking that would be found outside the experimental setting. The distinction made in this study is essentially the same as that made in the previous studies on evidence. Vague, general references to supposed facts were not considered evidence either in this study or in previous studies by other researchers. Therefore, some may prefer to think of the distinction between the two versions of the speeches as that of "good use of evidence" as opposed to "very poor use of evidence."
The speeches were delivered from a manuscript and recorded on magnetic tape. Two graduate students in speech were the speakers. Pretesting indicated that the speakers' voices gave no reliable cues about age and credibility. The two "evidence" speeches were approximately twenty minutes in length. The "no evidence" speech on capital punishment was approximately sixteen minutes in length, the "no evidence" speech on education approximately seventeen minutes.
Each speech was presented to an audience composed of sixty males and sixty females randomly selected from students enrolled in sixteen sections of Speech 200 at Pennsylvania State University.9 All subjects completed preliminary attitude scales of the Likert type four weeks before the experiment. Each group of subjects heard an "evidence" speech on one topic and a "no evidence" speech on the other topic. Immediately after each speech one-half of each group completed an attitude scale on the topic. The other half of each group completed two ethos scales, an authoritativeness scale, and a character scale. All subjects in both groups also completed on a six-item speech evaluation questionnaire following each speech. A control group (N = 69) completed the attitude scales four weeks before the experiment and again 24 hours before the experiment.
It should be noted that every effort was made to keep the above procedures consistent, as far as was desirable, with previous research on evidence. Although the education speech was wholly written by the experimenter, the capital punishment speech was a revised and updated version of the speech used by Cathcart.10
One major difference in procedure was desirable. There has been no consistent
method of measurement used by previous researchers. In this study, five-choice
Likert scales were used to measure attitudes on the two topics and four
the two dimensions of ethos. The speech evaluation scale was composed of
six seven-choice items, one of which concerned evidence usage. All of the
scales except the speech evaluation scale were pretested two more times.
The reliability estimates for these scales are reported in Table I.
Reliability of Scales
|No. of Items||Split-Halves * (Odd-Even)||
|Hoyt Internal Consistency*||
*At least four separate split-halves and Hoyt estimates, derived from
pre-testing and the final experiment, were made for each scale.
To test hypotheses 1, 3, and 4 two factor analyses of variance were
computed. The dependent variables were evidence ratings, authoritativeness
scale scores, and character scale scores respectively. The two factors
analyzed were sex and evidence usage.11 To test hypothesis 2,
Pearson product moment correlations were computed between initial attitude
and evidence rating and post-communication attitude and evidence rating.12
To test hypothesis 5 two factor analyses of covariance were computed. The
covariate was initial attitude and the department variable was post-communication
attitude; the two factors were sex and evidence usage.13 The
usual .05 criterion for significance was used except for instances where
the "F" test was used to test a directional hypothesis. In these
instances "t" tests were run if the "F" test met the
.1 criterion. The .05 criterion was used for these "t" tests.
In the following report of results the highest significance level met is
reported for each phase of the study.14
Auditors can perceive qualitative differences in use of evidence
in persuasive communication. Hypothesis confirmed. Analysis of variance
of evidence ratings for both topics indicated significant differences attributable
to evidence usage. (Capital punishment topic, F = 107.92, p <
.0001; education topic, F = 69.73, p < .0001.) Effects attributable
to sex were not significant. The evidence ratings for the "evidence"
speeches were much higher than those for the "no evidence" speeches.
These auditors clearly perceived the qualitative differences in use of
evidence in these speeches.
Auditors initially favorable to a speaker's position perceive evidence
usage to be of higher quality than those who are initially unfavorable.
Hypothesis not confirmed. The Pearson correlation between initial attitude
and evidence rating for the auditors hearing the capital punishment speeches
was .0058. The correlations between initial attitude and evidence rating
for the auditors hearing the education speech was .0294. Both of these
correlations are non-significant. However, the correlations between post-attitude
and evidence rating are much higher. For the capital punishment speeches
the correlation was .321. For the education speeches it was .447. Both
of these correlations are significant at the .001 level. This suggests
that not only was the "conclusion-to-evidence" pattern not operating,
but the reverse pattern, "evidence-to-conclusion," was most likely
in operation. In other words, these auditors were not judging the evidence
on the basis of their initial attitude toward the topic, but rather were
more likely judging evidence on the basis of their attitude as modified
by the speech which included that evidence.
Good use of evidence increases perceived authoritativeness of a speaker.
Hypothesis confirmed. Analysis of variance of the authoritativeness
scale scores for both topics indicated significant differences attributable
to evidence usage. (Capital punishment topic, F = 3.70, p < .1;
education topic, F = 5.75, p < .05.) Since a directional hypothesis
had been made, a one-tailed significance test was appropriate. Therefore
a "t" test run on the capital punishment topic scores. The result
was a "t" of 1.922 (p < .05 one tailed). There were
no significant differences attributable to sex. (Table II includes the
authoritativeness ratings for speakers in all treatments.)
Scores on Authoritativeness Scale1
1The hypothetical neutral point on this scale is 66. The lower the score the higher the perceived authoritativeness of the speaker.
*p < .025 level (one-tailed).
p < .005 (one-tailed).
p < .0005 Level (one-tailed).
We can conclude that the use of evidence in the speeches in this experiment
caused the auditors to perceive the speaker to be more authoritative than
the speaker who did not use evidence.
Good use of evidence increases perceived character of a speaker.
Hypothesis not confirmed. Analysis of variance of character scale scores
indicated no significant differences attributable to evidence for either
topic. Although use of evidence increases perceived authoritativeness,
it does not appear to have an effect on perceived character. It should
be noted, however, that there was a significant difference attributable
to sex on the capital punishment topic (F = 6.97, p < .01). Females
in both experimental treatments considered the speaker on this topic to
be of significantly higher character than did the males. This finding was
not replicated with the education topic. In fact, on this topic the males
rated the speaker's character higher than the females, although this difference
was non-significant. (Table III includes the character ratings for speakers
in all treatments.)
Scores on Character Scale1
1The hypothetical neutral point on this scale is 60. The lower the score the higher the perceived character of the speaker.
*p < .025 level (one-tailed).
p < .01 level (one-tailed).
p < .005 level (one-tailed).
§p < .0005 level (one-tailed).
Speeches which include good use of evidence produce a greater attitude shift than those which do not. Hypothesis confirmed. Analysis of covariance with the covariate of initial attitude and the dependent variable post-test attitude scores indicated significant differences attributable to evidence usage for both topics. (Capital punishment topic, F = 7.815, p < .01; education topic, F = 3.036, p < .1.) Since a directional hypothesis had been made, a one-tailed significance test was appropriate. Therefore, a "t" test was run on the education topic scores. The result was a "t" of 1.724 (p < .05 one tailed). In this experiment speeches which included "no evidence" produced less attitude change than did the speech using "evidence," although the subjects hearing the "no evidence" speech on both topics shifted significantly more than the control group. The control group was found not to shift significantly on either topic.
Significant effects attributable to sex and interaction of sex and evidence
usage were also indicated by the analysis of covariance for the capital
punishment topic. Females were significantly more opposed to capital punishment
both before and after the experimental treatment than the males and they
shifted their opinion against capital punishment significantly more than
the males. The significant interaction is explained by the fact that the
females who heard evidence shifted significantly more than any other group.
From this, one may speculate that evidence is uniquely effective when speaking
to females. However, since this finding was not replicated on the education
topic, such speculation has little support from this experiment.
In order further to examine the data available to be more certain that evidence produced the differences discussed above, the other five items on the speech evaluation scale were submitted to analysis of variance. There were no significant differences for the items on "organization and clarity of the speech," "voice usage," or "originality of thought" on either topic. A significant difference attributable to sex was found in "quality of reasoning" on the capital punishment topic. This is explained by the fact that the females were significantly more opposed to capital punishment both before and after hearing the speeches. It would be expected that "reasoning" with which one agrees would be rated higher than that with which one is not in agreement.
Significant differences in "quality of reasoning" and "general quality of the speech" were observed on the education topic. Since a highly significant difference in evidence usage was perceived, this could be expected. The only surprising thing is that similar differences were not found on the capital punishment topic.
The results on the three items unrelated to evidence lend justification to the conclusion that the "evidence" and "no evidence" speeches were comparable in all important aspects except evidence usage. It appears that the other elements of the speeches were adequately controlled to justify the conclusion that the results attributable to the difference between speeches were produced by the variations in evidence usage.
Other personal factors of the experimental subjects were checked to
be certain that they did not confound the results of this study. Age, term
in school, and grade point average were all found not to correlate significantly
with any of the dependent variables. We can conclude that personal factors
other than sex and initial attitude probably did not confound the results
of this study.
1. The traditional theory of the importance of evidence in persuasive communication is supported by these results. The speeches including evidence were significantly more effective in producing attitude shift than those which included no evidence. In this respect, these results are in agreement with those of Cathcart and Bettinghaus.15
2. The Gregg hypothesis of "conclusion to evidence" is not supported by the results. There was apparently no relationship between the auditors' initial attitude on the topics and their perception of the quality of evidence usage.
3. The Gregg hypothesis of "evidence to advocate" is supported. The speeches with evidence resulted in significantly higher scores on authoritativeness. However, no such result was found for character scores. It would appear that the "evidence to advocate" pattern operates only on the authoritativeness dimension.
4. Unlike the subjects in the Dresser study, the subjects in this study were able to perceive qualitative differences in evidence usage. It should be noted, however, that the differences in the treatments in this study were much more pronounced than those in the Dresser study. Therefore, the findings of this study should not be interpreted as contradicting those of Dresser's study. Rather, they suggest that conclusions based on Dresser's findings, which suggest that people cannot perceive qualitative differences in evidence usage, should be limited to small qualitative differences.
5. The importance suggested by Scheidel16 of controlling
the sex factor in experimental studies involving attitude shift is reinforced
by the results of this study. As noted above, sex was a significant factor
in some cases and not in others. The effects of sex must be considered
important but, as yet, unpredictable.
Some caution is advised in interpreting the results reported above. As indicated in Tables II and III, the unseen, unknown, tape-recorded speakers into his experiment were rated significantly above neutral in both authoritativeness and character after the subjects had heard them speak. Whether this represents a significant shift from their initial image of the speaker is unknown. It is quite possible that in this experimental setting the subjects initially perceived the speakers as high ethos sources. If this were the case, the results herein reported may not be generalizable beyond speakers who are initially high ethos sources. It should be noted that the finding of high post-communication credibility is consistent with the findings of both Cathcart and Dresser. The possibility that the ethos factor has confounded previous research on evidence and is responsible for the inconsistent findings reported should be seriously considered by future researchers concerned with the effects of evidence in persuasive communication.
However, it would seem reasonable to hypothesize that evidence should make a greater contribution to the effectiveness of a low ethos source than it would to a high ethos source. Since high ethos by itself may produce the maximum possible persuasion, evidence may have no effect when used by high ethos sources. If this hypothesis is correct, the results of the experiment reported above are even more significant than might appear at first consideration. Since significant effects attributable to evidence usage were found when the source was considered significantly higher than neutral on both authoritativeness and character measures, it would be reasonable to speculate that the effect of evidence would be even greater for neutral or low ethos sources.
A more precise estimation of the importance of evidence in persuasive
communication awaits further research which systematically manipulates
the ethos of the communicator. Until such time as this much needed research
is completed, it would appear that the study reported above provides some
justification for the stress that proper evidence usage receives in our
public speaking and argumentation instruction.
1. Robert S. Cathcart, "An Experimental Study of the Relative Effectiveness of Four Methods of Presenting Evidence," Speech Monographs, XXII (August, 1955), 227-233.
2. Erwin Paul Bettinghaus, Jr., "The Relative Effect of the use of Testimony in a Persuasive Speech upon the Attitudes of Listeners," unpublished thesis (Bradley, 1953).
3. Delmar C. Anderson, "The Effect of Various Uses of Authoritative Testimony in Persuasive Speaking," unpublished thesis (Ohio State, 1958); Dan L. Costley, "An Experimental Study of the Effectiveness of Quantitative Evidence in Speeches of Advocacy," unpublished thesis (Oklahoma, 1958); Howard Gilkinson, Stanley F. Paulson, and Donald E. Sikkink, "Effects of Order and Authority in an Argumentative Speech," Quarterly Journal of Speech, XL (April, 1954), 1983-192; and Gerard A. Wagner, "An Experimental Study of the Relative Effectiveness of Varying Amounts of Evidence in a Persuasive Communication," unpublished thesis (Mississippi Southern, 1958).
4. William R. Dresser, "Effects of 'Satisfactory' and 'Unsatisfactory' Evidence in a Speech of Advocacy," Speech Monographs, XXX (August, 1963), 302-306; and "Studies of the Effects of Evidence: Implications for Forensics," AFA Register, X (No. 3, 1962), 14-19.
6. Richard B. Gregg, "Some Hypotheses for the Study of the Psychology of Evidence," paper read at the SAA convention, Chicago, Illinois, (Dec. 28, 1964).
8. Kenneth Andersen and Theodore Clevenger, Jr., " A Summary of Experimental Research in Ethos," Speech Monographs, XXX (June, 1963), 59-78.
9. This is a course in speech fundamentals required of all students.
10. Robert S. Cathcart, "An Experimental Study of the Relative Effectiveness of Four Methods of Presenting Evidence," unpublished dissertation (Northwestern, 1953).
11. Quinn McNemar, Psychological Statistics (New York, 1962), Ch. 16.
12. Ibid., Chs. 8-10.
13. Ibid., Ch. 18.
14. All statistical computations were completed with the aid of the Pennsylvania State University Computer Center.
15. Op. cit.
Thomas M. Scheidel, "Sex and Personality," Speech Monographs, XXX (Nov. 1963), 353-358.
Click Here To Go Back To PERIODICALS