Over the past decade the importance of evidence in persuasive communication has been the subject of several experimental studies, with confusing results. Some researchers have found evidence a significant factor in influencing audience attitudes;¹ others have found no significant effect which could be attributed to evidence.² The authors of this essay have secured equally contradictory results with experiments of essentially similar design, using identical measuring instruments. We suspect some factor or combination of factors biased our own findings and may have similarly confounded earlier studies of evidence in persuasion.

Re-examination of the designs of our experiments and the designs of the experiments conducted by previous researchers concerned with the role of evidence suggests two elements that could have accounted for the divergent findings: ethos and experimenter bias. Because these two elements could affect many kinds of experiments in persuasive communication it seemed to us worthwhile to reanalyze our data with a view to discovering whether ethos and experimenter bias could, in fact, account for discrepant findings. The issues to be resolved in judging the confounding potentialities of ethos and experimenter bias in modern experimental research on persuasion are two.

(1) What is the initial ethos level of an unseen, unknown, tape-recorded speaker in an experimental setting? As in many previous studies the unidentified, tape-recorded speakers in our experiments were assumed to be neutral-ethos sources. The possibility that this assumption is untenable has been suggested previously by Dresser.³

(2) What effect does the experimenter's ethos have on an unseen, unknown, tape-recorded speaker's initial ethos? Numerous studies reported in psychological journals indicate that the impact of the experimenter on his subjects can confound his study;⁴ however, no reported research has dealt directly with the possibility that the experiment's ethos may bias the perceived ethos of a source used as an experimental stimulus.

The main difficulty in attempting to answer the first question is measurement. No measure of initial ethos of the tape-recorded speaker was taken in either of our experiments because it seemed doubtful that subjects could, or would, give meaningful responses to a scale concerning their opinions of an unseen, unknown, as yet unheard speaker. In such circumstances only indirect estimates can be made of the subjects' initial image of the speaker.

Several modern psychological theories of behavior, based on tension-reduction models, provide for predicting the direction of an audience member's attitude change toward a source if the audience member's attitudes toward the source and the source's proposition prior to message stimulus are known.⁵ These theories suggest that if the audience member is favorable to the source's proposition, his attitude toward the source will improve as a result of being exposed to the source's message. If he is neutral toward the source's proposition, his attitude toward the source will remain unchanged. If he is hostile to the source's proposition, his attitude toward the source will become less favorable as a result of exposure to the source's message.

In our first experiment measures of audience attitude were taken prior to message stimuli and measures of ethos were taken after message stimuli. By alternately assuming that the unseen, unknown, tape-recorded speaker was a neutral-ethos source and a high-ethos source prior to message stimuli, predictions of post-communication ethos were made. Thus, it was possible to test the first hypothesis with which our analysis was concerned--An unseen, unknown, tape-recorded speaker in an experimental setting is a neutral-ethos source--and the alternate hypothesis--An unseen, unknown, tape-recorded speaker in an experimental setting is a high-ethos source.

Measurement is also the main difficulty in attempting to estimate an experimenter's influence on the perception of the ethos of an unseen, unknown, tape-recorded speaker. The problem of measuring the effect of an experimenter's ethos on a speaker's ethos is similar to that of measuring the unseen, unknown, tape-recorded speaker's ethos. It cannot be measured directly, which is probably the reason it has been so seldom taken into account in previous analyses of research data. The method we have used to estimate the effect of the experimenter's ethos is based on the assumption that if all important factors except the experimenter's ethos are held constant between two groups, any differences in perceived ethos as measured at the completion of an experimental speech should be attributable to the experimenter's ethos. Using this procedure the following hypothesis was tested: An experimenter's ethos has no effect on the ethos of an unseen, unknown, tape-recorded speaker.⁶

It is important to stress two things at this point. First, we are not primarily concerned here with the results of our two experiments; rather, our concern is with why our results were contradictory. Secondly, the reason for attempting to explain why the results of our experiments were contradictory is that the procedures used in these experiments were very similar to procedures used by other experimenters concerned with other communication variables. The question, then, is whether a procedure or combination of procedures used by us and other researchers conducting experimental studies of persuasive communication can be identified as introducing a potentially confounding element in experimental design.

Two versions of speeches on two topics, capital punishment and federal control of education, were developed. In one version on each topic there was extensive use of documented and qualified evidence. In the other version no sources were cited or qualified and all specific, factual material was generalized. For example, a documented passage read:

According to Bulletin 1957-13 of the U.S. Office of Education, entitled "Provisions Governing Membership on Local Boards of Education," the U.S.O.E. national survey of school boards determined that 23.8% of the nation's school boards include people who are not even high school graduates. In the South the figure is 41%.

This was generalized to "Many of the school boards in the United States include members who are not even high school graduates" in the speech lacking documentation. The speeches were recorded by two graduate students in speech.

Experiment I included the speeches on both topics. Subjects used were randomly selected from sixteen sections of Speech 200 at The Pennsylvania State University.⁷ Subjects' attitudes toward the two topics were measured four weeks prior to the experiment. Measures of perceived authoritativeness and perceived character of the speaker and of post-communication attitude were administered immediately following each speech. In addition, each subject completed a speech rating scale which included an item concerning the speaker's use of evidence. Attitude and ethos measures were five-choice Likert-type scales. The experimenter was completely unknown to the subjects. He was not introduced or identified in any way. The subjects met in the evening outside of class, and no one was present during the experiment except the experimenter and the subjects.

The second experiment used only the speeches on capital punishment. Subjects were students participating in the Summer High School Speech Institute at The Pennsylvania State University.⁸ The attitude measures and speech rating scale used in Experiment I were administered immediately after completion of the experimental speech. Two experimenters, the director of the institute and one of the teachers in the institute, were present.⁹ Although there was no direct measure of the ethos of these individuals, an institute-evaluation questionnaire administered the following day indicated overwhelming enthusiasm for the quality of teaching the subjects believed they had received and for the general quality of the institute. It therefore may be assumed that the experimenters were perceived as at least moderately high-ethos sources. The experimenters made no direct attempt to imply sponsorship of the tape-recorded speaker, but no attempt was made to avoid such a sense of sponsorship. The subjects were merely told, "We would like to have you listen to this speech and then we will ask you to complete some questionnaires.

Hypothesis I. An unseen, unknown, tape-recorded speaker in an experimental setting is a neutral-ethos source. As we have noted previously, when a source is initially neutral, subjects favorable to the source's proposition should shift favorably toward the source, subjects neutral on the source's proposition should not shift at all, and subjects hostile to the source's proposition should shift unfavorably toward the source. (See Table 1.) In reanalyzing our data the direction of shift toward the source was computed by comparing the subjects' post-communication attitudes toward the source with the hypothetical neutral area on the ethos scales. An eleven-point area was arbitrarily designated as "neutral" on the post-communication measures of attitude toward the source. All scores within five points of the hypothetical neutral point on the scale were considered "neutral." On the basis of chance we could expect that one-third of our predictions would be accurate. Table 2 reports the predictions for the subjects which were confirmed, those which were un-confirmed and the x2 tests for significance of the accuracy of the predictions compared to chance.

In only one of the eight cases was the number of confirmed predictions significantly more than could be expected from chance. Even in this case less than 50 percent of the predictions were confirmed. On the basis of these findings we may reject the hypothesis that the unseen, unknown, tape-recorded speaker in this experimental setting was a neutral source. In all eight tests of this hypothesis the obtained results correspond to predictions appropriate when the source is presumed significantly above neutral. Therefore, the alternate hypothesis is supported by this analysis.

Hypothesis 2. An experimenter's ethos has no effect on the ethos of an unseen, unknown, tape-recorded speaker. For the purposes of analyzing our data in such a manner as to test the hypothesis concerning the experimenter's effect on initial ethos, it was assumed that the subjects in experiment I and experiment II were comparable. Experiment I involved a random sample of college students while experiment II involved a group of superior high school students. We recognize that any findings based on the comparability of these subjects must be tentative and that replication is necessary before firm conclusions can be drawn. Our concern in reanalysis of our data was to discover whether the presence of known and highly regarded experimenters--or some other as yet unisolated difference--could confound experimental data drawn from a particular population. We believe further that if data from the high school students indicated that the tape-recorded speaker had significantly higher ethos in this experiment than in our first experiment, there were, at the very least, grounds for suspecting that our own research and the research of others had been colored by forces not at all associated with either the character of the stimulus message or its alleged source.

*Significant at the .02 level.

On these bases the results of our two experiments were compared. Table 3 reports the mean authoritativeness and character scores observed in the "evidence" and "no evidence" conditions on the capital-punishment topic for both experiments. The only significant difference in ethos scores within an experiment was between the evidence and no-evidence versions on the authoritativeness dimension in experiment I. However, all of the obtained "t's" for differences in ethos scores between experiments were significant at the .05 level or above. (Table 4 reports the results of t-tests of the mean differences between experiments.) In other words, the subjects in experiment II considered the experimental speaker to be significantly higher in both authoritativeness and character than the subjects in experiment I.

*The hypothetical neutral point for the authoritativeness scale is 66; for the character scale, 60. The lower the score, the higher the perceived ethos.

Of particular note is the fact that the "no-evidence" speech in experiment II generated higher ethos than the "evidence" speech did in experiment I. Because of this, one might suspect that the differences in evidence usage didn't "take" in experiment II. This was found not be the case. Ratings of evidence usage were obtained on the speech rating scale in both experiments. In both experiments the ratings in the "evidence" condition were much higher than in the "no-evidence" condition and the difference was statistically significant (p < .001). Additionally, there were no significant differences observed on evidence ratings between experiments, a fact suggesting that our high school students did not differ from our college students in their perceptions of the messages themselves.

If one assumes the comparability of subjects in our experiments, one may reject the hypothesis that the experimenter's ethos had no effect on the ethos of the unseen, unknown, tape-recorded speaker. To assume that no essential comparability existed between randomly selected college students enrolled in a speech course and able high school students especially interested in oral communication is to question whether findings concerning communication are ever generalizable to populations not exactly comparable on all demographic dimensions. Given the fact that our younger subjects were at most only one, two, or three years from eligibility for inclusion in our college population, we believe there is a worrisome probability that our discrepant results in experiments I and II were attributable to the higher ethos of the experimenters in the second experimental situation. This interpretation seems all the more credible in view of the already mentioned fact that there were no significant differences observed on evidence ratings between experiments.

How one shall interpret the difference in results between experiments on the primary dependent variable presumably being studied is, of course, the basic problem in all research. In our experiments significant differences in attitude toward the topic were apparently induced by the evidence and no-evidence speeches in experiment I. No significant difference was observed between the two versions in experiment II.

Comparisons of mean terminal attitudes of subjects in the two treatments between experiments show marked differences. For both treatments the subjects in experiment II were significantly more opposed to capital punishment. ("Evidence" treatment, "t" = 2.880, p < .01; "no evidence" treatment, "t" = 2.926, p < .01.)

One must, we think, posit marked but elusive differences between the perceptions of our high school and college populations or suppose that the ethos of the experimenter made a significant difference in one condition and not in the other. There is, indeed, a line of reasoning concerning the known differences between our subjects which, if extended, leads to the conclusion that if differences of age and level of education were confounding elements in our findings, the differences observed experimentally should have been in the opposite direction from what was actually observed.

* Significant at the .05 level.

** Significant at the .02 level.

*** Significant at the .01 level.

****Significant at the .001 level.

The younger subjects (experiment II) were found to be significantly more opposed to capital punishment than the older subjects. Such opposition is generally considered a liberal attitude. Previous research indicates that as students proceed through school their attitudes tend to become more liberal.¹⁰ On this line of reasoning it would appear unlikely that differences in initial attitude toward the topic produced the observed differences in terminal attitudes. This is an admittedly tenuous analysis of our data. We offer it only to emphasize that insofar as data and reasoning allow for systematic explanation of our discrepant results, the presence of a known and respected experimenter is the experimental factor most suspect.

Whether one adopts assumptions of one of the tension-reduction theories of behavior or the guidance of common sense, there are theoretical grounds for supposing an unseen, unknown, tape-recorded speaker in an experimental setting can be perceived as a higher-than-neutral source. Yet much of the research reported in speech and psychology operates on the assumption that such a speaker is always a neutral source. The entire orientation of academia would lead a college or high school student to expect a high source. Student subjects are not ordinarily subjected to any other kind; and, when they are, they are usually explicitly informed that this is the case. Particularly if an experiment is conducted in a classroom with the subjects' teacher present, it seems unreasonable to assume that the experimental source will be considered neutral.¹¹ The findings of our experiment II support this belief.

There are equally good grounds for supposing that much the same would be the case with known and/or live speakers. There is certainly adequate experimental evidence on ethos to show that the introductions given speakers can modify their ethos.¹² Since these introductions are presented by the experimenter in most cases, the acceptability of the content of the introduction itself is partially dependent on the ethos of the experimenter.

The research and analyses reported here lend support to the theory that ethos is a very important factor in persuasive communication. The disturbing fact is that had only one of our experiments been conducted and had ethos not been measured we, like other experimenters, would have felt justified in describing inclusion of evidence in a speech as "valuable" or of "no value" depending on which study was conducted. Now, it appears the best explanation of the value of evidence must be given in terms of the ethos level of the speaker. A single research study involving evidence, which does not measure or report the perceived ethos level of the source, is probably of little or no value.

We do not wish to over-generalize our findings, but their implications are obvious and serious. If ethos can confound research on evidence, why can't it also confound research on other variables? And is there any theoretical ground for supporting it would not do so in all cases where ethos of any sort could operate? The conflicting findings in studies on primacy-recency, one-sided vs. two-sided presentations, and the like may be as directly attributable to differential ethos levels as to the controlled variables. On the basis of our own experience with discrepant findings, we would be very hesitant to accept any findings of research in persuasive communication unless the ethos level were measured. To conduct a study of a speech element on only one ethos level is, of course, of value. However, if we do not know precisely what that level is, it seems unsafe to generalize beyond the narrow limits of the study itself. We would argue that future researchers must be more concerned than they, or we, have been with the confounding factor of ethos and the possibility of bias as a result of experimenters' ethos. Appropriate allowances in research design can and badly need to be made, and ethos levels need to be clearly specified if we are to generalize safely from the one experimental situation to another, much less from the laboratory to the public platform.

1. Erwin Paul Bettinghaus, Jr., "The Relative Effect of the Use of Testimony in a Persuasive Speech upon the Attitudes of Listeners," unpublished thesis (Bradley, 1953) and Robert S. Cathcart, "An experimental Study of the Relative Effectiveness of Four Methods of Presenting Evidence," Speech Monographs, XXII (August, 1955), 227-233.

2. Delmar C. Anderson, "The Effect of Various Uses of Authoritative Testimony in Persuasive Speaking," unpublished thesis (Ohio State, 1958); Dan L. Costley, "An Experimental Study of the Effectiveness of Quantitative Evidence in Speeches of Advocacy," unpublished thesis (Oklahoma, 1958); Howard Gilkinson, Stanley F. Paulson, and Donald E. Sikkink, "Effects of Order and Authority in an Argumentative Speech," Quarterly Journal of Speech, XL (April, 1954), 183-192; Gerard A. Wagner, "An Experimental Study of the Relative Effectiveness of Varying Amounts of Evidence in a Persuasive Communication," unpublished thesis (Mississippi Southern, 1958); and William R. Dresser, "Effects of 'Satisfactory' and 'Unsatisfactory' Evidence in a Speech of Advocacy," Speech Monographs, XXX (August, 1963), 302-306.

3. William R. Dresser, "Studies of the Effects of Satisfactory and Unsatisfactory Evidence in a Speech of Advocacy," unpublished dissertation (Northwestern, 1962).

4. See for example, F. J. McGuigan, "The Experimenter: A Neglected Stimulus Object," Psychological Bulletin, LX (November, 1963), 421-428; Robert Rosenthal, "Experimenter Attributes as Determinants of Subjects' Responses," Journal of Projective Technique and Personality Assessment, XXVII (September, 1963), 324-331; Robert Rosenthal and Kermit L. Fode, "Psychology of the Scientist: V. Three Experiments in Experimenter Bias," Psychol. Rep., XII (April, 1963), 491-511; Harold W. Stevenson and Sara Allen, "Adult Performance as a Function of Sex of Experimenter and Sex of Subject," Journal of Abnormal and Social Psychology, LXVIII (1964), 214-216; and Robert Rosenthal, Ray C. Mulry, Mardell Grothe, Gordon W. Persinger, and Linda Vikun-Kline, "Emphasis on Experimental Procedure, Sex of Subjects and the Biasing Effects of Experimental Hypotheses," Journal of Projective Technique and Personality Assessment, XXVIII (December, 1964), 470-473.

5. See S. E. Asch, "The Doctrine of Suggestion, Prestige, and Imitation in Social Psychology," Psychological Review, LV (July, 1948), 250-276; Leon Festinger, A Theory of Cognitive Dissonance (Evanston, 1957); Fritz Heider, "Attitudes and Cognitive Organization," Journal of Psychology, XXI (January, 1946), 107-112; Nathan Maccoby and Eleanor E. Maccoby, "Homeostatic Theory in Attitude Change," Public Opinion Quarterly, XXV (Winter, 1961), 538-545; Theodore M. Newcomb, "An Approach to the Study of Communicative Arts," Psychological Review, LX (November, 1953), 393-404; and Charles E. Osgood and Percy H. Tannenbaum, "The Principle of Congruity in the Prediction of Attitude Change," Psychological Review, LXII (January, 1955), 42-55.

6. It should be stressed that the tests of these hypotheses and the hypotheses themselves are based on ex post facto analysis. The rigor which should be present for sound experimental tests of hypotheses was unattainable; therefore, these tests should be viewed more as theoretical tests than as experimental tests.

7. This is a required course in oral communication. Students from all colleges in the University are enrolled and the classes include students at all undergraduate levels.

8. This institute is conducted for superior high school students. The course of study includes debate, extemporaneous speaking, and oral interpretation. Students from Pennsylvania and surrounding states participated.

9. One of the experimenters was the same individual who conducted experiment I.

10. For a report of an extensive study involving this question, see T. M. Newcomb, Personality and Social Change (New York, 1943).

11. For a discussion of the possible "sponsorship effects" on experimental studies in classroom settings, see Carl I. Hovland, "Reconciling Conflicting Results Derived from Experimental and Survey Studies of Attitude Change," American Psychologist, XIV (January, 1959), 8-17.

12. Kenneth Andersen and Theodore Clevenger, Jr., "A Summary of Experimental Research in Ethos," Speech Monographs, XXX (June, 1963), 59-78.

Click Here To Go Back To PERIODICALS