Evaluation can be regarded as a distinct branch of social science research and, as such, presents evaluators with specific ethical problems not encountered by other researchers. This paper examines ethical difficulties arising from an evaluation study and puts forward for discussion a code of ethics designed to resolve these.
I had come to evaluate the work of an educational planning unit. That was my purpose. But what transpired as the real purpose of my study? I learned in time that I ... had come to assist in marketing a product: to bottle up the dreams, hopes frustrations and expectations of the members of a planning unit and label all that neatly - "of Value: Not To Be Disbanded".The above quotation epitomises the problem I wish to examine in this paper. The hapless and perhaps somewhat naive evaluator, one who moves from educational research into the field of evaluation, can very quickly become embroiled in the politics of evaluation. That is, there often is the urge for clients to view evaluation as a means for marketing their 'product', rather than as an equitable means of assessing its merit.
Clear identification of the audience for an evaluation could serve to alleviate such a problem as the one discussed above. Scriven (1981, p.63) has identified formative evaluation as a process conducted, "during the development or improvement of a program.... It is an evaluation which is conducted for the in-house staff of the program and normally remains in-house". This is somewhat different from summative evaluation which Scriven (1981, p.150) has characterised as being conducted for the benefit of an external audience. This distinction allows for the evaluation audience to be identified prior to commencement of the study and so those involved will know beforehand the nature and possible consequences of the undertaking. In theory, therefore, this distinction between formative and summative evaluation is important because it allows participants to provide information, knowing in advance who their audience will be. In practice, however, political and personal differences amopng program staff can make even formative evaluation a task fraught with ethical difficulties related to the collection and disclosure of data.
In this paper I will examine the idiosyncrasies of evaluation research as compared to other modes of social science research, and argue the case for a code of ethics related specifically to evaluation studies, one that is designed to protect the rights of all those involved in evaluation research: evaluator, evaluated and audience.
Community involvement in the program meant that part of the evaluation brief was to monitor and evaluate the activities of community members participating in implementation. This was necessary because 'stakeholders' from the community were viewed as having the potential to change the program significantly through their input. Thus, the source, nature and effect of these changes needed to be recorded and evaluated. To this end, the roles, actions and motives of individuals and groups within the community who participated in the program were clearly identified as frames of reference for evaluation. It is in this part of the evaluation that ethical problems were encountered.
Smith (1980, p.193), in discussing the ethics of fieldwork in ease study research, has stated: "The two most important operating principles in maintaining respect for the persons involved in one's study are informed consent and anonymity of participants." In addition to this, Simons (1984, p.88) has made the following statement in discussing procedures for the conduct of an independent evaluation:
Confidentiality is necessary to protect individuals from inappropriate use of information which is private to them. Rules of access and consultation give individuals opportunities to decide what to share, to reflect on what they have shared, to edit or comment upon their information in context: to control, in other words, the use of their own information.Thus informed, I committed myself to adhere to a code of ethical conduct based on the principles of informed consent and privacy.
Prior to beginning data collection, I gave and explained to participants a document outlining the purpose, usefulness, methods and expected benefits of the evaluation. Also contained in this document was a detailed description of individuals' rights under the ethical code I had adopted. In summary, these rights were:
At its simplest, evaluation leads to a settled opinion that something is the case. It does not necessarily lead to a decision to act in a certain way, though today it is often intended for that purpose.... Evaluation leads to a judgment about the worth of something.These characteristics of evaluation - that it "leads to a settled opinion that something is the case" and to a judgment about the worth of something - place it squarely into the political arena. In the case of the program I evaluated, the execution of this role required me to judge the actions of individuals in the context of their vested interests and motivations for becoming involved in the program. Thus, as evaluator I became involved in the "politics of personal aspirations".
And, in attempting to identify and judge the effect of these aspirations on the implementation of the program, I had to walk the fine line between what was public and what was private.
Simons (1987) also has discussed the role of evaluation, and has illustrated its distinguishing features when compared to other modes of research, one of which is that: "Immediate utility may not be a requirement for research but it is defining for evaluation" (p. 17). This feature of evaluation emphasises the difficulties the evaluator faces in trying to maintain the balance between protecting the rights of individuals and remaining independent and unbiased. The fact that evaluation focuses on the particular (Simons, 1987, p.9), rather than being disposed toward making generalisations (as is usually the case in other forms of research), means that the evaluator is often required to report intimate, and sometimes controversial, data to an audience that is closely involved in that which is being evaluated. The requirement that evaluation be useful in making value judgments also means that the evaluator must weigh the merit of the actions of individuals and then report to an audience which, if it is not made up of the same individuals, will include people associated with them.
These characteristics of evaluation make it almost impossible for the evaluator to guarantee the anonymity of participants. The usual devices, such as the suppression of names and specific locations, do not guarantee anonymity because the audience's knowledge of the scene would lead, just by inference, to their recognition of people and places no matter how they were disguised. Thus, the evaluator is faced with a number of problems: maintaining his/her independence and ensuring that the evaluation is free from bias; protecting the rights of participants; and reporting in such a way that the private lives of participants are not exposed to public scrutiny and judged as part of the evaluation. This brings us back again to the distinction between what is private and what is public.
On this subject, Pring (1984, pp.15-16) has stated:
The distinction between the private life and the public life of an individual is a difficult and blurred one.... The balance .. between the private and the public must relate to the political significance of the authorities and institutions being evaluated.To illustrate the problematic nature of attempting to make the distinction between private and public life, I will use an example from the evaluation of the program to which I have been referring.
My first interview with the chairperson of the program's executive committee took place on a Monday morning. The previous weekend there had been a major casts in the implementation of the program. Because of this, I questioned the committee chairperson about the crisis and his actions in dealing with it. The answers to my questions revealed some very interesting data which I considered to be highly relevant to the evaluation for the following reasons:
I have ammended [sic] the transcript to concentrate on items relative to the project and have deleted negative or personal comments on purpose. I would like your final report to be available for public reading and I don't think these comments would allow that.The amendments to the transcript constituted deletion of data related to his efforts to solve the crisis and his political differences with the other committee member. Through these changes it was made clear to me that, on my behalf, the committee chairperson had decided what was and was not relevant to the evaluation. It also showed that this person was willing to use his rights as a subject of the evaluation to censor data and to ensure that the evaluation was neither independent nor unbiased.
The committee chairperson's action in amending the transcript highlights a dispute over whether the data he gave me should remain private or should be made public. As the evaluator, I have no doubt that the data should be seen as public, because they affected the implementation of the program and thus they dealt with actions carried out in the public arena. This person, however, wanted his actions to remain private, because their revelation in the public arena would have been damaging to him in terms of his standing as a public figure in the community.
In any other form of educational research it is clear that the individual's right to privacy must be respected at all times. Any competent researcher would guarantee these rights to participants explicitly in negotiating to pursue his/her research. Moreover, in the case of disagreements over the relevance of data to the research, such disputes could often be solved by reassuring the individuals that their identities will not be revealed and that the audience for the research report will be at least once removed from the research site. If necessary, the researcher could discontinue the research and pursue the same problem at another site, although this would only be a consideration in extreme circumstances.
It is at this point, however, that the weakness in guaranteeing participants' rights to privacy in an evaluation becomes most apparent. The distinctive characteristics of evaluation bring into question the usual guarantees of privacy and anonymity the evaluator is able to offer participants. Ultimately, it is not possible to shift the focus of an evaluation to another site, because it is particular instances or programs that are of interest. Furthermore, offering participants in an evaluation the right to privacy also has the potential to cause disputes over what is private and what is relevant to the evaluation. This is clearly illustrated in the committee chairperson's censorship of his interview transcript.
I do not want to suggest, however, that the subjects of evaluation should place their reputations entirely in the hand of the evaluator. The line of demarcation between what is private and what is public should be drawn through a process of careful negotiation. In carrying out this negotiation, I would like to suggest the following guidelines:
Referring back to the quotation with which I began, this would mean that the "dreams, hopes, frustrations and expectations" of those who were participants in the evaluation would remain in their own hands. It would be the clients who would be involved in the 'marketing' of their product and, in this endeavour, they would be free to use the evaluation report as they saw fit. Moreover, the evaluator would be permitted to evaluate without becoming subject to political manipulation and unencumbered by a code of ethics which is not appropriate for his/her role.
House, E. R. (1980). Evaluating with validity. Beverley Hills: Sage Publications.
Pring, R. (1984). The ethics of evaluation. In C. Adelman (Ed.), The politics and ethics of evaluation, pp.8-18. London: Croom Helm.
Scriven, M. (1981). Evaluation thesaurus (3rd ed.). Inverness, CA: Edgepress.
Simons, H. (1984). Principles and procedures for the conduct of an independent evaluation. In C. Adelman (Ed.), The politics and ethics of evaluation, pp.87-92. London: Croom Helm.
Simons, H. (1987). Getting to know schools in a democracy: The politics and process of evaluation. London: Falmer Press.
Smith, L. (1980). Some not so random thoughts on doing field work: The interplay of values. In H. Simons (Ed.), Towards a science of the singular: Essays about case study in educational research and evaluation, pp.179- 205. Norwich: Centre for Applied Research in Education, University of East Anglia.
Sungalia, H. M. (1987). On doing an organisational evaluation. In R. J. S. MacPherson (Ed.), Ways and meanings of research in educational administration, pp.287-304. Armidale: University of New England.
|Author: Andrew Thompson is currently teaching at Wanneroo Primary School. He has recently completed a Master of Education at Edith Cowan University. His research interests include curriculum development, implementation and evaluation and issues associated with the process of change in education.
Please cite as: Thompson, A. (1992). The ethics and politics of evaluation. Issues In Educational Research, 2(1), 35-44. http://education.curtin.edu.au/iier/iier2/thompson.html
© 1992 Issues in Educational Research
Last revision: 9 Aug 2001. URL: http://education.curtin.edu.au/iier/iier2/thompson.html
Previous URL from 17 June 1997 to 7 Aug 2001: http://cleo.murdoch.edu.edu.au/gen/iier/iier2/92p35.htm
HTML : Clare McBeath, Curtin University [firstname.lastname@example.org] and Roger Atkinson [email@example.com]