QJER logo 2
[ Contents Vol 10, 1994 ] [ QJER Home ]

Trial by Profile: Report on the trialling and validation of the Draft National Arts Profiles

An independent research study conducted in conjunction with Queensland schools


Janis Boyd and Susan McCadden


Background to the study

The Curriculum and Assessment Committee (CURASS) of the Australian Educational Council (AEC)has been working on reconciling several national initiatives - the key competencies of the Mayer (1992c) Committee, the Australian Vocational Certificate System proposals (Carmichael 1992), the Finn (1991) targets for participation in education and training, the Common and Agreed Goals for young persons' cognitive, affective and physical development as individuals, citizens and workers, accepted by the Australian Educational Council in the 1989 Hobart Declaration, and subsequent work on the AEC Profiles in each of the agreed areas for compulsory school years.

Common and agreed goals for schooling in Australia are currently being collaborated on by the various state governments. National statements in eight key learning areas of the primary and secondary school curriculum are being developed. They are Arts, English, Health, LOTE, Mathematics, Science, Studies of Society and the Environment, and Technology.

Statements and profiles are being developed within the context of the following national policies and projects:

CURASS believes that the profile outcomes will provide the underpinning of the Key Competencies. Each state and territory intends to implement the profiles, and the profiles therefore provide a basis for reporting student achievement. For example, the strand Visual Arts and Design would incorporate the process components 'transforming', 'presenting', 'arts criticism and aesthetics', and 'past and present contexts'. Within each strand, eight different achievement levels have been identified.

Figure 1: Visual Arts and Design Strand

Separate book or section for each of the arts strands should be formatted in the following way:

BANDSLEVEL
ONE
LEVEL
TWO
LEVEL
THREE
LEVEL
FOUR
LEVEL
FIVE
LEVEL
SIX
LEVEL
SEVEN
LEVEL
EIGHT

Level
Statement







COMPONENTOutcome
Statement







TransformingPointers
*
*
*








Outcome
Statement







PresentationPointers
*
*
*








Outcome
Statement







Arts Criticism
& Aesthetics
Pointers
*
*
*








Outcome
Statement







Past &
Present
Contexts
Pointers
*
*
*







Similarly, a skills progression overview chart could be developed. Also vocabulary and definition of terms.

The researchers believe that formatting would enhance teachers' knowledge and understanding of the scope and content of visual arts and design.

Presenting as a component needs to be more specific to visual arts and design, e.g. Design Presentation or Display.

Figure 1 represents the framework of the profile which will describe ]earning outcomes typical of student achievement at each level within the strand of Visual Arts and Design. It will also provide pointers as detailed assistance for teachers in recognising when students are achieving at each level. Profiles, however, will not define assessment procedure.

National Curriculum Statement in the Arts

The National Curriculum Statement in the Arts is intended as a guide for further curriculum development in the arts. It is a framework and not intended to be a syllabus or a course outline. While it will promote a more consistent approach to the development of arts curricula in Australian schools, it will also encourage diversity of practice at the school level. It will comprise the following strands: Visual Arts and Design; Media Studies; Dance; Music; and Drama.

In line with the common and agreed national goals for schooling in Australia, the Arts National Curriculum Statement will:

Learning Outcomes, Statements and Pointers

Outcome statements are intended as broad summative descriptions of achievement. For instance, 'Past and Present Contexts' (Visual Arts and Design) would take the following form:

Figure 2: Past and Present Contexts (Visual Arts Design)

VISUAL ARTS & DESIGN
PAST & PRESENT CONTEXTS - LEVEL 4

Knowledge of arts history, study of the arts from different cultural contexts and awareness of issues.

At Level 3, the student can:

Identify characteristics of some contemporary art works from their own and other cultures focusing on artistic traditions and comparing the way arts elements are used to express ideas and feelings

At Level 4, the student can:

Show an understanding of the arts and different social and cultural groups in Australia demonstrating a sense of history and tradition in relation to the arts.

At Level 5, the student can:

Write about the nature, purposes and presentation of arts works of past and present societies and explore typical themes, forms, styles and genres of specific periods.

Previous LevelOutcome StatementNext Level

Pointers are more specific and are used to illustrate how achievement may be recognised in each of the Visual Arts and Design processes.

Suggested pointers for Level 4

This will be evident when, for example, students in Art and Design: In each profile strand there is one statement for each of the eight levels. Because of their generic nature, the outcomes are necessarily broad and for this reason the more specific points for Visual Arts and Design are very important as they give meaning to the outcome statement. The pointers are intended as starting points for each teacher to use as a draft. They are only typical sample statements of student achievement and it is expected that teachers will write additional or alternative pointers relevant to particular contexts.

Briefly, the profiles comprise:

The level statements describe student performance outcomes across the strands at particular levels and assist teachers to make a holistic judgment about individual student achievement in the area of learning. The level statement is a valuable aid to teachers in setting targets and goals.

The outcome statements describe in progressive order the skills and knowledge that students typically acquire as they become more proficient in that area. Outcome statements should be expressed in terms of what the student is able to do, and should be amenable to a variety of assessment approaches to the curriculum area. They should be sequenced where possible so that teachers can use a specific outcome above and below to assist in identifying the level of student achievement.

The pointers support the development of a rich and detailed descriptive account of an individual student's achievement. They assist teachers in understanding the meaning of the outcomes. Pointers are neither comprehensive nor required and teachers might develop additional pointers. They are not intended to be the basis for judgment of achievement although they could be used to provide evidence supporting judgment of achievement and outcomes.

Annotated student work samples are examples of student work which is achieving one or more outcomes and they help teachers to develop a shared understanding of the standard implied by a level in the profile.

Focus of the Research Project

This research project focused upon the implementation of the draft profiles by teachers in eleven schools. The two major purposes of the study were to: In this specific study, the strand investigated by this independent research team was Visual Arts and Design inclusive of its four components.

Sample

The study sample comprised eleven volunteer teachers from eleven different schools. Teachers' experience ranged from preschool/primary through to secondary school. Average age of teachers was approximately twenty-seven years; most teachers had completed four to ten years of service. Preschool teachers (T3 and T4) were specialist teachers; T1, T2, T5, T7, T8 T9, T10, T11 were generalist teachers in primary classrooms and T6 was a trained secondary visual arts specialist who also taught mathematics, science and physical education as part of her beginning teacher repertoire. All teachers were three-year trained and enrolled in advanced professional development courses for the completion of a Bachelor of Education. All but one teacher was in full-time employment. All teachers were enrolled in TT44046 P-12 Visual Arts Curriculum.

Methodology and Research Techniques

The research study was qualitative and focused upon a descriptive study of teachers' implementation procedures of the profiles during a two-week trialling period. Results pertaining to the teachers' understanding of the profiles were derived from many interpretative sources of data: a reflection journal; two questionnaires; an interview; a visual art and design test; and samples of student work (photographic data).

Results were interpreted through successive approximation using open, axial and selective coding methods.

The study has internal reliability because of its short timeframe which ensures that information is fresh, relevant and not affected by time. The sample teachers were interested in visual art, therefore less inclined to give misleading information and more likely to give ethically reliable results. There was also consistency in research strategies because each facet in the data gathering process occurred at approximately the same time.

External reliability is apparent through the multiple forms of data collection.

There was no classroom intervention and therefore it has 'ecological validity'. The study was largely determined by the National Arts Curriculum Guidelines, thus it has 'natural history' credibility. Results were member validated by the teachers in the study and the researchers, both teachers by qualification, were 'competent inside performers'.

Results from the Focus Questions

1. The Strands

Do you find the four component / strands a useful way of organising the profiles? i.e. transforming, presenting, arts criticism and aesthetics, and past and present contexts.
AnswerMost teachers agreed with the need for an organised framework. There was confusion over the use of terms - the components are transforming, presenting, arts criticisms and aesthetics and past and present contexts; whereas the documents clearly outline the strands as being the discipline areas of visual arts and design, media studies, drama, music and dance.

What would be your view about using the arts forms as strands in the profile? i.e. art and design, dance, drama, media and music.
AnswerQuestion unclear. Answers reflect confusion.

2. Level Statements

Do the level statements broadly describe student achievement at each level?
AnswerTeachers believed that the outcome statements for the levels were too broad.

Have you any suggestions concerning the content of the level statements?
AnswerSome teachers believed that the statements should be placed in sequential order, together so that an overview could be formed to show progression from one level to another. There were problems with language used in the statements, for example Level 8 states '. . . students should have a sound knowledge of 'convention of presentation'.' Conventions of presentation are different for the arts strands and meaning is contextual. This is another instance of trying to place discrete learning under generalised headings.

3. Outcome Statements

Do the learning outcome statements describe progression of learning in the arts?
AnswerThe levels and their learning outcome precludes preschool teachers from being part of this study. Nowhere is there a description of what precedes Level 1 in terms of expected attainment levels, yet preschool teachers have extensive visual arts programs which allow for students' individual development. This vital foundation has been missed by the designers and writers of the program as a correct starting point. The researchers believe that the Level 1 statement incorporates preschool but perhaps a specific statement should be made effectively to give credit to the work that preschool teachers do in stimulating interest in the visual arts. The majority of teachers expected there to be progression of learning in this document. It was suggested that a glossary of terms be included as well as elements and principles of design.

Are there any particular outcome statements which are unclear or unambiguous?
AnswerTeachers found many ambiguities in language and visual arts terms.

It is the belief of the researchers that the language used in level outcomes reflect that specialist teachers would be teaching Levels 5 to 8 whereas Levels 1 to 4 would be taught by generalist teachers. There is a juxtaposition of knowledge whereby the specialist educator would have no problems because of the breadth of background knowledge, whereas the primary generalist may have problems with clarity and meaning of some statements.

Do the outcome statements adequately describe achievement in the five art forms?
AnswerIt is the view of the researchers that the writers are missing the opportunity to encourage problem solving by including open-ended questioning techniques which are less prescriptive and give teachers more scope. Most teachers agreed that the visual arts and design profiles had adequately described achievement levels.

What modifications do you think should be made to the learning outcome statements? Please give suggestions for modifications.
AnswerAnswer: Teacher 4 believes that ta one-page sequence chart of outcome statements of all levels needs to be included for each component area', (i.e. transforming, presenting, etc.). Specific statements for each of the arts strands is vital. General terminology cannot encompass contextual meanings and the non-propositional learnings that occur within each explicit and discrete discipline. Each art form/strand contributes to a person's development in an essentially different dimension.

What an individual comes to know through drama is very different from the 'knowing' which occurs in visual arts and design. Teacher 6 supports this belief by saying 'the outcome statements need to be more obviously directed to each of the arts forms.' Teacher 10 believes that statements must be clearer for 'teachers who have had no training in arts to understand.'

4. Pointers

Do the pointers assist you in interpreting the outcome statements?
AnswerAnswer: Most teachers agreed that the pointers assisted them in making judgments for reporting upon student levels of achievement. Many teachers surveyed had reservations about the 'narrowness' of examples. Teachers support the view that if 'a teacher's willing to use that profile and then expand on it, then it will broaden' their ideas and visual arts programs.

Concerns were felt by the researchers because they could see the pointers as becoming 'exclusive' to their ideas which would assist in interpreting outcome statements. The researchers believe that the pointers are only one of many reporting instruments for assessing students' levels of attainment. For example, participation and enjoyment in visual arts and design activities would need to be considered in the evaluation procedure. Problem solving, critical thinking, visual awareness and metacognition are involved in the strand components and should contribute towards the holistic reporting upon student attainment levels. The profiles report on the tangibles.

Are there any pointers which do not illustrate the outcome statements?
AnswerMost teachers conceded that they had deficits of knowledge in the components of 'Art Criticism and Aesthetics' and 'Past and Present Contexts'. This made the task of answering the question problematic because deficits in knowledge, terminology and skills 'undermined' the teachers' ability to see conceptual connections.

Are any of the pointers placed in the wrong level?
AnswerThe researchers believe that the focus questions about the use of pointers is to be answered with the commensurate knowledge of the teachers - some with visual arts training would have little difficulty, whilst others with little or no training in visual arts and design would have marginal success with their application and use of pointers as reporting instruments.

Are there any other key pointers that could be added to give a clearer idea of the meaning of the learning outcome statement? Please write these out in the format of the profile dot points.
AnswerTeachers added the following pointers:
  • specific vocabulary and terminology should be set at each level . . . Glossary of terms;
  • specific set of sequential learnings at each level;
  • specific pointers to be added as the teaching year progresses.
The major limitation in answering this question was the short trialling period (two weeks). Teachers were unable to build up a repertoire of activities which would broaden the scope of the pointers. The researchers believe that 'pointers' will be broadened considerably when teaching groups, at the school level, are able to collaborate to affect changed to the profile documents.

5. Work samples

Given the restrictions of reproducing work samples in the photocopied format of the profile format, would additional work samples be useful?
AnswerAll teachers believed that additional work samples were needed. A range of work samples, which indicate student attainment levels and ages were seen as additional criteria so that teachers could make informed reporting judgments. The researchers believe that because of teacher deficits in knowledge and skills in visual arts and design, quality 'exemplars' or work samples are needed in order to gauge 'what is expected' as a reasonable sample of art work is important. However, caution must be exercised so that teachers do not focus solely upon art production at the expense of the thoughtful insights which occur during the artistic process.

Do you have any suggestions to make about the presentation of work samples?
AnswerTeachers agreed that the presentation of work samples should take the form of slide kits, photographic material and videotapes, which would complement the documents. The researchers agree, but believe that this aspect would probably be the responsibility of each state or territory because each would make unique contributions in the form of visual materials, for example, Queensland could lead the way in the production of authentic Aboriginal artworks.

6. Inclusiveness

Is the profile gender inclusive?
AnswerAll teachers agreed, but conceded that this would become more apparent when working with the documents over a period of time.

Does the profile cater for achievement by students from all cultural groupings, in particular Aboriginal and Torres Strait Islander students?
AnswerMost teachers agreed. One exception was Teacher 5 who believed that the profile should be strengthened in this area, 'especially in the lower levels'.

7. Language

Do you have any concerns with the style of language used? Please give specific examples and alternatives.
AnswerMost teachers had no problems with the language. However, Teacher 10 suggests that 'maybe a page of definitions or a brief summary to make things clearer' could be introduced to the existing format.

8. Other comments

Please make specific comments which may help in the preparation of the final profile.
Answer
  • Teachers expressed concern that they had taught visual arts in the past and, upon reflection, could not find appropriate 'pointers' to enable them to allocate the activities to an appropriate level.
  • Frequency of the use of profiles for reporting upon student attainment levels needs to be specified. Many teachers believed that these had to be used after each session.
  • Outcome proforma was confusing - headings needed.
  • Need to stress that profile is only one device for reporting and should not be used in conjunction with behavioural and cognitive checklists.
  • Should all areas, e.g. transforming, presenting, arts criticism and aesthetics, past and present contexts, be given equal time?
  • A list of expected sequential skills be presented as an overview.
  • A background knowledge and skills book for each level is necessary for generalist teachers.

Results of questionnaire 2 and interview schedule

Emerging themes from this questionnaire and interview:

General conclusions from questionnaire 2: Interview and reflection journals

The researchers recognised the need to examine the teachers' background and experience in visual arts and design in conjunction with the focus questions. This strategy added a further dimension to the study because it revealed teacher attitudes to change, confidence in teaching, competence to teach and knowledge of reporting and assessment. Background and experience would 'colour' the way teachers perceived the focus question.

Conclusions reiterated by teachers were:

Content and scope of the Queensland curriculum for preschool to Year 12

A short answer test based upon scope and knowledge of visual arts and design was implemented to the teachers in the research sample. The aim of this test was to determine teachers' knowledge and understanding of the content of visual arts. With the new strand of design being added to the discipline of visual arts, it was important to ascertain whether teachers had the depth of knowledge and understanding to teach it, remembering that this group comprised mainly generalist teachers.

The visual arts and design, as defined by the Queensland Arts Teachers' Association, consist of a diverse range of processes integral to important life opportunities - training, employment, family, community, leisure and recreation. They are inclusive of:

Design ArtsProduct, interior, industrial landscape and graph design (illustration, cartooning, animation, advertising, posters, magazines, computer imaging), urban planning and architecture.
Fine ArtsDrawing, painting, print making, sculpture, photography, photocopy art, computer imaging, holography, virtual reality/interactive TV, ceramics, glass, fibre, installation, performance art and crafts, assemblage.
Folk ArtsTraditional handcrafts.
Popular ArtsFilm, television, magazines, posters, computer imaging, virtual reality/interactive TV, telecommunications.
Community ArtsFestivals, events and a wide range of community arts projects.
Information ArtsTV, telecommunications and computing
Arts Industry StudiesAdministration, marketing, facilitation, curation, journalism and vocational studies.

Results

Interpretation of the forms of art and design was very diverse. Overall, just over one-third of the answers given revealed any conceptual understanding of these art forms. Most often, questions were not attempted, answered very vaguely, or incorrectly guessed and thus, generally, teachers fared poorly in this area.

The definitions and examples given by teachers for design arts is a pertinent example because it is an entire component of the subject title. Some teachers broadly related design arts to industry. Others believed it to be a creative extension of previous works of art. Examples given aptly included 'graphic design', 'fashion design' and 'interior design' and yet these were side by side with others such as 'painting', 'sculpture' and '3-D'. This indicated that there is only a vague conceptual understanding regarding design.

Due to the diverse nature of the answers given in this text, particular generalisations pertaining to the teachers' understanding could not be made.

Conclusion

The researchers are concerned with the above findings, because this group of generalist teachers is a representative sample of many teachers throughout Queensland. It indicates the current level of understanding and knowledge of visual arts and design is deficient, so much so that teachers would have difficulty in implementing a program, effectively, in educational milieux.

Validation process (Photographic data)

Because the researchers did not have access to CD Rom information for the validation exercise, teachers photographed student works and then made judgments about students' achievement when referring to both the outcome statement and pointers. Teachers used the methodology outlined (first five girls and first five boys on their class lists), listed in the instructions for the validation exercise.

(ACER Validation of the National Arts Profile - March 1993;
Notes from Ann Carroll. Queensland contact for Validation and Trialling. )

The researchers for this study believe that this methodology is flawed in the fact that many children who do not possess Anglicised names would not be included within this study. Therefore, the process is discriminatory and excludes input from children from multicultural background whose visual arts images may add a further dimension to the pointers, outcome statements and validation process.

One of the tasks in the reflection journals was for the teachers to report on their students' level of learning outcomes using the profile recording sheet supplied by the curriculum developers. Reporting was compared with the photographic data of the sample students' work.

Results

This task was diversely interpreted by the teachers and therefore there was little consistency among reporting techniques. This suggests that the directions for reporting on student learning outcomes were unclear or open-ended enough to be interpreted individually. This factor further reflects the differences in teachers' perceptions and background knowledge. Some teachers often stated a general level of outcome but failed to report on the level outcomes of the individual process components. Many only reported on 'transforming' and then generalised this single report to an overall level outcome. Teachers really only taught one component yet made the reporting process fit all components.

Most teachers included 'presenting' in their reports. Some incorporated 'arts criticism and aesthetics 'into their lesson and only one teacher used 'past and present contexts'. Arts criticism and aesthetics was often misinterpreted as being simply an extension of 'presenting', for example in Teacher 3's class, students were encouraged to 'tell the story about their own art work'. This was the objective for 'arts criticism and aesthetics'. In fact, this exercise or process should have been under the 'presenting' component.

'Process' is just as important as 'product' when reporting on a child's progress in art and design (Eisner 1972). The profiles fail to identify that skill acquisition achieved through the learning outcome statements is only one way of determining their students' progress. Many teachers did report on the process involved during the development of the product despite this, but some did not.

Those who strictly reported the measurable learning outcomes achieved by students suggest to the researchers that art and design could develop into a skill acquisition subject assessed for competencies rather than knowledge, understanding, aesthetics and expression.

For further details of teacher responses and appendices see (Boyd & McCadden) (1993) Trial by Profile: Report on the Trialling and Validation of the National Arts Profiles (Draft), an independent research study conducted in conjunction with Queensland Schools; Focus Strand: Visual Arts and Design.

Recommendations

This research study has revealed that this is a need for substantial introspection and possible change to the national arts profiles if they are to be implemented successfully in the future.

Recommendations for the writers and designers of the national arts profiles are defined below around the following concepts:

Recommendations for systems and institutions

State governments and non-systematic organisations will need to support curriculum innovations in the following ways:

The need for further study

This research highlights the need to study the impact of the national arts profiles as an aspect of change within Queensland. Many teachers within Queensland have been constantly exposed to curriculum innovation over the past years with little or no support in visual arts and design. 'Living by Design' (resource booklets) were issued to schools without consultancy support. P-10 Arts Framework was printed but released only upon demand. The school level respondents in this project repeatedly referred to the importance of curriculum and instructional leadership at all school levels, particularly in the primary schools where generalist teachers are expected to implement innovations.

Further research should analyse the roles of specialist personnel in assisting implementation at the school level. Additional research focusing upon the provision of teaching resources is vital for the validation process. Perhaps better use of expertise which already exists in schools (Boyd 1993) and the community could be a supportive step in the direction of curriculum innovation.

References

Boyd, J. (1993) 'Review of Queensland Schools Curriculum, Visual Arts, Submission from School of Communication, Language and the Arts, Faculty of Education, Griffith University.

Carmichael, L. (Chair) ( 1992) The Australian Vocational Certificate Training System, Employment and Skills Formation Council, Canberra: National Board of Employment, Education and Training.

CURASS (1992) Towards a National Standards Framework for the Schooling Sector, AEC Curriculum and Assessment Committee, Report 7/92/6.1. (Available from Commonwealth and State Departments of Education.)

Duck, G. (1987) Teacher Education and the Arts in Queensland Primary Schools, prepared for the Board of Teacher Education, Queensland.

Eisner, E. (1972) Educating Artistic Vision, New York: Macmillan.

Finn, B. ( 1991) Young People's Participation in Post-Compulsory Education and Training, report of the AEC Review Committee, Canberra: AGPS.

Gaitskell, C., Hurwitz, A. & Day, M. (1982) Children and Their Art: Methods for the Elementary School, fourth edition, New York: Harcourt Brace & Jovanovich.

Mayer, E. (Chair) (1992c) Putting General Education to Work, the key competencies report, the Australian Education Council and Ministers for Vocational Education, Employment and Training, Canberra.

Boyd, J. & McCadden, S. (1993) Trial by Profile: Report on the Trialling and Validation of the National Arts Profiles. (Appendices for the report include: (1) Focus Questions provided by Curriculum Association (CURASS); (2) Questionnaire 2; (3) Interview Schedule; (4) Short Answer Test.)

Please cite as: Boyd, J. and McCadden, S. (1994). Trial by Profile: Report on the Trialling and Validation of the Draft National Arts Profiles. Queensland Researcher, 10(2), 26-45. http://education.curtin.edu.au/iier/qjer/qr10/boyd.html


[ Contents Vol 10, 1994 ] [ QJER Home ]
Created 3 Sep 2005. Last revision: 3 Sep 2005.
URL: http://education.curtin.edu.au/iier/qjer/qr10/boyd.html