|
It should be a reasonable expectation that all children entering school do so with equal opportunity of success which will be reflected in essentially equivalent outcomes of assessment of a diversity of groups irrespective of class, ethnicity or gender. This is, however not the case. This paper offers a brief overview of evidence of inequitable outcomes. It then examines some contributing factors including historical facets of test development and utilisation, and interrelated sociological elements such as language, socioeconomic status and ideological bases. Discussion then turns to possibilities for fair assessment in a culturally diverse population. It is proposed that regardless of technical or practical difficulties equal outcomes for diverse groups must be the goal, and that this goal cannot be well-served by traditional, standardised and criterion-referenced tests, nor by wide-scale assessments aimed at differentiating for accountability purposes. Situated performance assessments are suggested as a potential, though not guaranteed, means of catering for diversity, due to their multidimensional and localised perspectives.
While equal outcomes may be an unrealistic and debatable goal in assessment, there must be concerns if the differentials consistently favour particular groups at the expense of others. Evidence from the United Kingdom, the USA and Australia clearly suggests that this is the case in relation to some ethnic minorities.
Trends studied over two decades in the USA reveal that performance gaps between white and black students are closing in the areas of reading, mathematics and science according to tests administered by the National Assessment of Educational Progress (Jones, 1993). Nonetheless black and Hispanic students still score lower than others on standardised tests (O'Connor, 1989; Gipps & Murphy, 1994).
With the introduction of National curriculum assessment in the UK efforts to ensure fairness to seven year old bilingual students included presenting the tasks in English and other languages and by the screening out of biased items (Gipps & Murphy, 1994). Yet, as Gipps and Murphy point out, although the results for bilingual children exceeded teacher expectations, they still fell below other groups. The general patterns of teacher assessment and Standard Tests were parallel.
In Australia about forty-five percent of Aboriginal and Torres Strait Islander primary school children are deemed to have lower than average literacy and numeracy performance levels, compared with sixteen percent of non-indigenous Australians, a disadvantage carried over into their secondary education (Department of Employment, Education and Training, 1994).
Although these instances hardly provide exhaustive evidence of inequity, they are indicative of an ethnically-based differentiation which warrants much closer investigation than simplistic ascription to biased test items or acceptance that such differential outcomes are 'normal' reflections of cultural difference. In the interests of fairness and justice, a comprehensive range of factors needs to be explored and accounted for, in educational assessments, for a diverse society such as Australia.
Western presuppositions of intelligence as a fixed, measurable, innate capacity, underlaid notions of fair competition but in effect categorised students and located within individuals (Broadfoot, 1996) and consequently certain groups (Wolf, Bixby, Glenn & Gardner, 1991) the causes of their success or failure. These misapprehensions were exacerbated by hierarchies of types of knowledge and a view of learning as a linear sequence of knowledge acquisition performed by individuals in isolation (Wolf et al., 1991). Such notions are unrealistic and ethnocentric. They produce an excuse for accepting failure as inevitable for some, and as a basis for ranking and labelling students, with lifelong consequences (Wolf et al., 1991). As Broadfoot (1996) notes, these individualised attributions contrast with the Japanese approach which focuses on the relationship between effort and achievement and which presupposes achievable success for all.
Attempts by Western test developers to produce unbiased instruments through culture-free, culture-fair or culture-specific tests have either failed to deliver equity in outcomes or fallen foul of reliability requirements (Garcia & Pearson, 1994). Gipps and Murphy (1994) question the possibility of devising genuinely culture-independent tests.
Adverse consequences may apply beyond overtly high-stakes or exit-placement assessments. For instance, low scores may impact on motivation and consequent future performance (Gipps, 1994). However, there are more direct effects, such as differential access to resources of curriculum, teacher time and hardware, which often accompany test-based ability grouping (Gipps & Stobart, 1993). Although 'tracking' may have been formally eradicated in Australian primary schools, through policies of inclusion, there is still a great deal of informal sorting, the most adverse effects being felt by children of low socioeconomic groups, non-English speaking immigrants and indigenous Australians (Cope & Poynting, 1989).
Given that most school assessments are heavily language-dependent, it is obvious that limited proficiency in English will be a disadvantage not only in processing the semantic and syntactic structures, but also in unfamiliarity with what O'Connor (1989) terms 'discourse genre' - culturally specific patterns of discourse by which interactions are configured and recognised. Socialisation in these patterns, such as question-answer routines and other verbal interactions, is advantageous to children in interpreting a literacy task (O'Connor, 1989). The advantages of preparation for mainstream language use and interactions have been clearly demonstrated in Heath's (1983) ethnographic study, the implications of which are evident for indigenous Australian children. These are issues of access to background knowledge rather than specifically a problem of test instrument validity. International surveys indicate that 'when differences in curriculum background are controlled or out-of school experiences taken into account, group differences are reduced' (Gipps & Murphy, 1994, p. 264).
Beyond cultural differences are legitimating ideologies of universality and neutrality of mainstream cultural capital, as a result of which the judgements made by assessments appear to normalise differential outcomes (Apple, 1982; Broadfoot, 1996; Nespor, 1991). Assessment also situates students 'in a field of surveillance' (Foucault, 1979, p. 189). Thus assessment becomes a mediator between school and society (Broadfoot, 1996) and an instrument of social control (Wexler, 1982). It is, however, not necessarily a one-way process. Ideologies of resistance characterised by non-participation, indicate the struggle of minorities for a voice (Apple, 1982; Bourdieu & Passeron, 1990; Donald, 1991). Such resistance is evident among Aboriginal and Torres Strait Islander students and is implicated in continued school failure with long-term adverse consequences (Folds, 1987; Keeffe, 1992).
In the face of these complexities how are Australian educators to define the principle of equity and devise assessments which advance that principle?
Technical issues such as validity remain significant in maximising equity (Messick 1989; Nuttall, 1989), but fairness goes beyond statistical validations and elimination of culturally biased items (Gipps, 1994; O'Connor, 1989; Moss, 1992; Wiggins, 1993). In a psychometric model products are evaluated independently of other information about students or contexts. Wiggins (1993, p. 125), however, asserts that equity demands 'responsive dialogue', while Moss (1994) suggests a cycle of interpretation-validation-interpretation applied to multiple varied products as an approach which can account for social, contextual and ideological factors.
In a climate of cultural diversity these broader notions of equity rule out dependence on wide-scale, externally-imposed, psychometrically based tests (Wiggins, 1989 & 1993; Wolf et al., 1991) which traditionally presuppose differential performances (Goldstein, 1993). Educators must look toward some form of locally-defined performance assessment which, although not guaranteeing equity, holds promise for enabling culturally diverse students to show what they can do (Gardner, 1992).
Rather than being driven by instruction, whereby mastery of a fixed-sequence program is tested, performance assessment allows for the integration of instruction and assessment. That integration ideally incorporates a collaborative dynamic of performance --> feedback --> reflection --> higher level performance, operating among individuals, groups and teachers in mutually empowering learning partnerships (Wolf et al., 1991).
Terminology such as authentic, situated or localised (used interchangeably in this paper) refer to the 'on-site' nature of performance assessments where assessment and instruction operate in a synergistic relationship. There is a necessity in this teaching-learning cycle that performance goals and criteria for judgment be made explicit throughout the process (Garcia & Pearson, 1994). This form of assessment is compatible with current constructivist theories of learning which emphasise the variability and social construction of learning within and across cultures (Wolf et al., 1991).
Two factors in communication which may unfairly disadvantage indigenous students are resistance to direct questioning, based on traditional interactional patterns (Christie, 1985), and the use of Aboriginal English, which may be considered by assessors as inferior to standard Australian English. Clearly, prejudices against culturally-shaped patterns of communication may operate just as much in performance assessment contexts as in standardised or externally-mandated tests (Baker & O'Neil, 1994) due to stereotyping and ignorance of sociocultural differences (Filer, 1995). Nonetheless the localised nature of situated performance assessment presents the possibility of adapting questioning routines, altering response times and recognising non-standard English as a valid form of expression (West, 1994), so that students have a means for balancing 'their need to learn "the power code", standard English, with their personal need to honor their own dialect and language identity' (Garcia & Pearson, 1994, p. 371). As Kale and Luke (1991) point out, Aboriginal children enter school with considerable diglossic skills. If identified and valued these skills may be incorporated into locally-developed assessment tasks thus alleviating the disadvantages caused by the tendency of tests to rely on mainstream verbal skills.
Aboriginal children have been described as sharing a degree of equality with adults uncharacteristic of mainstream Australia (West, 1994). The accompanying autonomy is highly regarded in all areas of life (Keeffe, 1992; Malin, 1990), including education which is traditionally seen as a 'collective responsibility' (Folds, 1987, p. 20). Accordingly, assessments which promote ownership of, and partnership in, learning based on respect are more likely than externally-devised tasks to engage the students' interest in demonstrating their full scope of abilities. Responsibility for selecting, reflecting on and revising work samples promotes a sense of equality and autonomy. The framework or performance assessments includes such values (Garcia & Pearson, 1994) and encourages a trusting, co-operative relationship between teacher and students. Opportunity for parental and community involvement and negotiation can also be built into such a model, compensating for the reversal of the traditional 'elders down' educative process to the 'trickle-up' direction of Anglo-Australian schooling (Sykes, 1986, p. 61).
Traditional Aboriginal learning emphasised learning of real-life skills in an holistic sense by observation and imitation (Folds, 1987). Consequently there was little likelihood of making significant errors. West (1994) comments that Aboriginal children do not like making mistakes, which may at least partly explain the affiliative nature of their peer interactions in relation to attentiveness and nurturance (Malin, 1990). The implications for culturally sensitive assessments are clear. Assessment should be embedded in whole, real-life tasks, not imposed as isolated, decontextualised tests which are likely to be seen as both irrelevant and threatening. Collaborative effort should also be valued in recognition that knowledge acquisition is a social process - a fact in any culture, but one which is often overlooked in Western assessment paradigms (Wolf et al., 1991). It can be recognised by performance assessments.
Use of instructionally integrated assessment also reduces the demands of becoming test-wise, while engagement of children in collaborative scaffolded learning and assessment allows them to experience complex, meaningful activities rather than being restricted to acquiring isolated skills (Garcia & Pearson, 1994). This is particularly important since low-achievers, among whom cultural minorities are disproportionately represented, have generally been denied access to challenging, highly-valued areas of curricula and are thereby doubly disadvantaged (Baker & O'Neil, 1994). Indigenous Australian children are among those who have suffered such disadvantage (Malin, 1990); Walton, 1993). The possibilities of situated performance assessments to redress these aspects of inequity must therefore command attention.
At the site of assessment there is no guarantee that the processes will be used well. Students may not be made adequately aware of the conditions under which assessment is carried out - the criteria, approaches to elaborated tasks, the acceptability of collaboration and the requirement to think beyond rote-learned material (Wolf et al., 1991). At this stage there is also no certainty about whether performance assessments can, or will, adapt to recognise and include the linguistic strengths of minorities. If the verbal component remains high and culturally irrelevant, reformed assessment modes may be of little benefit.
A great deal then depends upon the expertise and awareness of teachers. Teacher biases, lowered expectations and lack of knowledge about culturally-defined interactions such as discourse patterns may well lead to failure to tap into the considerable knowledge resources of minority children just as surely as externally-developed standardised tests (Gipps & Murphy, 1994; Garcia & Pearson, 1994; McCann, 1994). Additionally, without confidence in their own professional competence and/or commitment to fundamental philosophies and practices teachers are likely to slot reformed assessments into old instructional design, thereby undermining such goals as empowerment and equity. Walton (1993), for instance, identifies the need for more explicit, interactive, socially responsive pedagogies in schools with Aboriginal students.
There are continuing controversies over the technical validity of performance assessments and other non-psychometrically centred systems. Further research and developmental projects are necessary to achieve reasonable assurance of validity and generalisability of interpretive, qualitative approaches. As Moss (1992) points out, however, there is a wealth of literature about qualitative assessment methodology on which to base validation, even for accountability and high-stakes purposes. The constraint would appear to depend on whether there is the social and political will to replace comparative, quantitative concepts of assessment with an instructionally informative, qualitative view.
Maintaining a formative focus is vital in maximising the success of performance assessments. Judging from the British experiences of National Curriculum assessment, where summative and accountability purposes begin to take precedence, there is a strong tendency for centralised/standardised assessments to re-emerge, accompanied by competitiveness, which leads to a narrowing of assessment tasks and of the curriculum itself (Gipps & Stobart, 1993). Given that performance assessments can be time-consuming, efficiency measures can easily undermine their flexibility and breadth. Such developments work against the valuing of diversity of knowledge and thinking which is basic to the concept of equity for cultural minorities (Wolf et al., 1991).
Currently there appears to be considerable emphasis on assessment as a key to ensuring improved literacy and numeracy standards. Yet assessment is but one fact of the educational landscape. Equity in relation to assessment must be placed in the broader field of conditions necessary to promote equitable participation and outcomes for Aboriginal children. It is reprehensible, for instance, that there are blatant inequities in access to schooling and resources which, in the Northern Territory at least, lock out remotely-located Aboriginal children from the education afforded to similarly located Anglo-Australian children (Walton, 1993). Without the systemic support of quality instruction, curriculum and resources, and equal access to that support, the effectiveness of the very best assessments will be severely constrained. Indeed, as Garcia and Pearson (1994, p. 377) assert: 'Investment in assessment without investment in resources... will do nothing but exacerbate current inequalities'.
While the aforementioned constraints are substantial, they by no means outweigh the possibilities of classroom-based performance assessment for promoting fairer assessments for Aboriginal and Torres Strait Islander children. The task of educators is to make the most of the possibilities while minimising the effect of constraints.
The issue here is to protect the primacy of assessment as a tool to advance children's learning. Though important, accountability is a secondary aspect. What must be made clear is that more, and higher-stakes, assessment will not improve standards. Standards only improve in relation to the quality of teaching and to learner engagement in the process (Gipps, 1994), the latter being a particular priority in improving educational outcomes of Aboriginal and Torres Strait Islander children. Perhaps one of the most practical means of providing opportunity for learner engagement is through the keeping of portfolios. Kept well, these embody the advantages of localised performance assessments in a concrete, manageable form (Sakrzewski, 1997).
Commitment to performance assessment should include a long-term perspective, since inconsistency in education practices are disadvantageous to cultural minorities especially where English is not the home language. Familiarity with the form and expectations of assessment is important for optimum development (Garcia & Pearson, 1994), hence the importance of target skills being made explicit and modelled, and assessment of those skills occurring in the context of normal utilisation (Brown, Campione, Webber & McGilly, 1992). In addition these instructionally-integrated forms of assessment should be begun in the P-4 years since this is where educational foundations are laid (Brown et al., 1992). If changes are indicated, the flexibility of performance assessment permits ongoing adaptations without the disruptions and alienation which may be precipitated by major system changes.
In relation to professional development, recommendations would include inservice training in: observational skills; target description and interpretation; the concept of construct validity (Gipps & Murphy, 1994); awareness of the cultural distinctives of teachers' characteristics as assessors; and attempting to view from the perspective of the 'other' (Kochman, 1989). Teachers routinely make running assessments which guide both short-term and longer-term instructional decisions. However, to formalise these processes, especially for accountability purposes, is not necessarily a simple transition. Delivery of assessment can be no better than the support that teachers are given in terms of time and resources expended on professional development.
While teachers may, or should, be already cognisant of the subjectivity and fallibility of all forms of assessment, these realities should also be made transparent to parents and the general public. Such transparency would provide a framework of realistic expectations of assessment systems and also reduce the appeal of calls for resurgence of standardised testing, with the awareness that equity is more assured with a localised performance-based system (Wiggins, 1989).
It is strongly recommended, therefore, that the involvement and input of Aboriginal and Torres Strait Islander parents be welcomed and encouraged, not expecting them to adapt to mainstream ways (Folds, 1992) but to foster a genuine sense of ownership, in conjunction with respect for the teacher's expertise and accountability. Without a co-operative approach there is a very real possibility that the indigenous community will perceive new forms of assessment as patronising or even as a deliberate ploy to maintain their subordinate position (Baker & O'Neil, 1994). In a nutshell: 'Minority communities must not once again become unwilling recipients of innovations which others believe are good for them.' (Baker & O'Neil, 1994 p. 14).
In a climate where 'accountability' and 'standards' tend to dominate the vocabulary of policy-makers, equity for indigenous students may appear to be a neglected goal. 'Accountability', 'standards' and 'equity' are, however, not incompatible concepts. All are best served by equipping children with strategies for rigorous intellectual activity (Wiggins, 19993) in culturally relevant contexts and using assessments which, rather than focusing on differentiation, reflect and commend each child's progress.
In supporting current learning theory, honouring diversity and encouraging equitable assessment, situated performance assessments appear to show the way forward to educational justice for all children.
Baker, E. L. & O'Neil, H. F. (1994). Performance assessment and equity: a view from the USA. Assessment in Education, 1(1), 11-26.
Bourdieu, P. (1994). The school as a conservative force: Scholastic and cultural inequalities (J. C. Whitehouse, Trans.). In J. Eggleston (Ed.), Contemporary research in the sociology of education (pp. 32-34). London: Methuen.
Bourdieu, P. & Passeron, J. C. (1990). Reproduction in education, society and culture. London: Sage.
Broadfoot, P. (1996). Education, assessment and society. Buckingham & Philadelphia: Open University Press.
Brown, A. L., Campione, J. C., Webber, L. S. & McGilly, K. (1992). Interactive learning environments: A new look at assessment and instruction. In B. R. Gifford and M. C. O'Connor (Eds), Changing assessments: Alternative view of aptitude, achievement and instruction (pp. 121-211). Boston: Kluwer.
Christie, M. J. (1985). Aboriginal perspectives on experience and learning: The role of language in Aboriginal education. Victoria: Deakin University.
Cope, B. & Poynting, S. (1989). Class, gender and ethnicity as influences on Australian schooling: An overview. In M. Cole (Ed.), The social context of schooling (pp. 217-237). London: Falmer Press.
Department of Employment, Education and Training (DEET) (1994). National review of education for Aboriginal and Torres Strait Islander people: A discussion paper. Canberra: Australian Government Publishing Service.
Donald, J. (1991). How illiteracy became a problem (and literacy stopped being one). In C. Mitchell & K. Weiler (Eds), Rewriting literacy: Culture and the discourse of the other (pp. 211-227). New York: Bergin & Garvey.
Filer, A. (1995). Teacher assessment: Social process and social product. Assessment in Education, 2(1), 23-38.
Folds, R. (1987). Whitefella school: Education and Aboriginal Resistance. Sydney: Allen & Unwin.
Folds, R. (1992). Yellow brick roads in Aboriginal education: A Central Australian critique of theories of two-way Aboriginal schooling. The Aboriginal Child at School, 20(2), 3-11.
Foucault, M. (1979). Discipline and punish. (A. Sheridan, Trans.). London: Penguin.
Garcia, G. E. & Pearson, P. D. (1994). Assessment and diversity. Review of Research in Education, 20, 337-391.
Gardner, H. (1992). Assessment in context: The alternative to standardised testing. In B. R. Gifford and M. C. O'Connor (Eds), Changing assessments: Alternative view of aptitude, achievement and instruction (pp. 77-119). Boston: Kluwer.
Gipps, C. V. (1994). Beyond testing: Towards a theory of educational assessment. London: Falmer.
Gipps, C. V. & Murphy, P. (1994). A fair test? Assessment, achievement and equity. Buckingham and Philadelphia: Open University Press.
Gipps, C. V. & Stobart, G. (1993). Assessment: A teacher's guide to the issues. London: Hodder & Stoughton.
Goldstein, H. (1993). Assessing group differences. Oxford Review of Education, 19(2), 141-150.
Heath, S. B. (1983). Ways with words: Language, life and work in communities and classrooms. New York: Cambridge University Press.
Jones, L. V. (1993). School achievement trends for black students. In B. R. Gifford (Ed.), Policy perspective on educational testing (pp. 19-73). Boston: Kluwer.
Kale, J. & Luke, A. (1991). Doing things with words: Early language socialisation. In E. Furniss & P. Green (Eds), The literacy agenda: Issues for the nineties. (pp. 1-16). Portsmouth: Heinemann.
Keefe, K. (1992). From the centre to the city. Canberra: Aboriginal Studies Press.
Kochman, T. (1989). Black and white cultural styles in pluralistic perspective. In B. R. Gifford (Ed.), Test policy and test performance: Education, language and culture. Boston: Kluwer Academic Publishers.
Lanhupuy, W. (1987). Balanda education: A mixed blessing for aborigines. The Aboriginal Child at School, 15(3), 31-36.
Malin, M. (1990). The visibility and invisibility of Aboriginal students in an urban classroom. Australian Journal of Education, 34(3), 312-329.
McCann, H. (1995). Towards a dynamic cultural model in Aboriginal education. Social Alternatives, 14(3), 45-49.
Messick, S. (1989). Meaning and values in test validation: The science and ethics of assessment. Educational Researcher, 18(2), 5-11.
Moss, P. (1992). Shifting conceptions of validity in educational measurement: Implications of performance assessment. Review of Educational Research, 62(3), 229-258.
Moss, P. (1994). Can there be validity without reliability? Educational Researcher, 23(2), 5-12.
Nespor, J. (1991). The construction of school knowledge: A case study. In C. Mitchell & L. Weiler (Eds), Rewriting literacy: Culture and the discourse of the other (pp. 169-188). New York: Bergin & Garvey.
Nuttall, D. L. (1989). The validity of assessments. In P. Murphy & B. Moon (Ed.), Developments in learning and assessment (pp. 265-276). London: Hodder & Stoughton.
O'Connor, M. (1989). Aspects of differential performance by minorities on standardised tests: Linguistic and sociocultural factors. In B. R. Gifford (Ed.), Test policy and test performance: Education, language and culture (pp. 129-173). Boston: Kluwer Academic Publishers.
Preston, N. & Symes, C. (1992). Schools and classrooms: A cultural studies analysis of education. Melbourne: Longman-Cheshire.
Sakrzewski, M. (1997). Portfolios: An authentic system for assessment of literacy in primary schools. Unpublished manuscript.
Shepard, L. A. (1991). Negative policies for dealing with diversity: When does assessment and diagnosis turn into sorting and segregation? In E. H. Hiebert (Ed.), Literacy for a diverse society: Perspectives, practices and policies (pp. 279-298). New York & London: Teachers College Press.
Sykes, R. B. (1986). Incentive, achievement and community: An analysis of black viewpoints on issues relating to black Australian education. Sydney: Sydney University Press.
Walton, C. (1993). Aboriginal education in northern Australia: A case study of literacy, policies and practices. In P. Freebody and A. R. Welch (Eds), Knowledge, culture and power: International perspectives on literacy as policy and practice (pp. 55-81). London: Falmer Press.
West, L. (1994). Cultural behaviour, conflict and resolution. In S. Harris & M. Malin (Eds), Aboriginal kids in urban classrooms (pp. 7-19). NSW: Social Science Press.
Wexler, P. (1982). Structure, text and subject: A critical sociology of school knowledge. In M. W. Apple (Ed.), Cultural and economic reproduction in education: Essays on class, ideology and the state (pp. 275-303). London, Boston & Henley: Routledge & Kegan Paul.
Wiggins, G. (1989). A true test: Toward more authentic and equitable assessment. Phi Delta Kappan, May, 1989, 703-717.
Wiggins, G. (1993). Assessing student performance: Exploring the purpose and limits of testing. San Francisco: Jossey Bass.
Wolf, D., Bixby, J., Glenn, J. & Gardner, H. (1991). To use their minds well: Investigating new forms of student assessment. In G. Grant (Ed.), Review of Research in Education, 17, (pp. 31-125). Washington, DC: American Educational Research Association.
| Author details: Margaret Sakrzewski is a primary school teacher who is undertaking part time postgraduate studies in the Graduate School of Education at The University of Queensland. She has particular interests in the socio-cultural aspects of education.
Please cite as: Sakrzewski, M. (1997). Accounting for cultural diversity: Issues of equity and authenticity in assessment with particular application to Aboriginal and Torres Strait Islander students. Queensland Journal of Educational Research, 13(2), 59-76. http://education.curtin.edu.au/iier/qjer/qjer13/sakrzewski.html |