Engaging students in peer review: Feedback as learning
Catherine Moore and Susan Teather
Edith Cowan University
There is a growing interest in tertiary education in Australia about developing the capacity of learners to evaluate and improve both their own work and that of others (Boud & Falchikov, 2006; Oliver, 2011). In order to successfully direct their own learning beyond university (and engage in lifelong learning), students need to be able to evaluate their performance in relation to a standard, identify gaps, and determine how to bridge them in order to achieve the desired standard if required. One strategy that can be employed to help students achieve this is engaging them actively in a feedback process, so that feedback becomes an integral part of learning. In this paper the authors define feedback as a loop, meaning that the feedback process is only complete when there is not only some identifiable influence on the recipient of the feedback (Boud & Molloy, 2012), but feedback results in improved performance - a sign that learning has occurred. This paper describes how peer feedback was embedded in a third year social studies unit. Pre and post surveys of students reveal their responses to the requirement that they actively engage in evaluation and feedback. This paper explores how engaging students in peer review as part of a major assessment task affected their learning, their capacity to direct their learning, and their self-efficacy in relation to academic and real-world tasks.
Feedback can motivate and improve learning, so it is essential for students to be provided with effective, timely and appropriate feedback (Pearce, Mulder & Baik, 2009). Feedback that focuses on "growth rather than grading" (Sadler, 1983, p. 60) tends to make sense to students and is far more likely to advance student learning than feedback that does not make sense to students (Gibbs & Simpson, 2004).
Students are more motivated to engage with and use feedback when the immediate utility of that feedback is clear. They want feedback to be something that helps them do better in the next task or, when feedback is given on drafts, something that can immediately be used to improve the final product (Price, Handley, Millar & O'Donovan, 2010). The difficulty students have in interpreting feedback was also highlighted by Price et al. (2010) who reported that written feedback without dialogue is often viewed by students as frustrating and can lead to disengagement. They further noted student perceptions that dialogue and discussing examples of work assists interpretation of feedback and increases the utility of feedback.
Black and Wiliam (1998) referred to the substantial evidence from research that demonstrates classroom assessment will most effectively enhance learning when learners:
This paper outlines a feedback process involving several related activities which facilitated student understanding of the criteria by which their work would be judged, developed their capacity to identify both current and desired levels of achievement, generated information about how to close any gap, and gave them the opportunity to use that information to actually close the gap. We believe this to be a sustainable feedback process (Hounsell, 2007) as it contains the four characteristics of sustainable feedback as identified by Carless, Salter, Yang and Lam (2011):
Students were surveyed to gauge their response to the peer review process in which they had engaged.
The purpose of implementing this process was to facilitate a paradigm shift of role, from student to professional social worker (which involved students becoming empowered rather than merely engaged), by developing their ability to monitor, manage and evaluate their own learning, and to gain experience in the collaborative ways of working they would encounter after graduation.
The authentic assessment adopted in the unit assists students to develop a number of graduate attributes and employability skills. The development of graduate attributes such as effectively communicating with others, working in teams, generating ideas, considering cross-cultural and international perspectives, and critical reflection and appraisal skills are particulary focused upon in this unit. The development of these attributes is closely linked to what employers identified as essential skills that these final year students will require when they embark upon their new careers.
The planned learning activities in this unit were based around authentic assessment in a situated learning context, designed to encourage active and deep learning (Bonwell & Eison, 1991; Lave & Wenger, 1991; Wiggins, 1990). The assessment consisted of two linked assignments which asked each individual student to construct a full family support program for a real population. Students were allowed to select the population for which they designed the program. By offering learning opportunities relevant to the students and in their area of interest, independent learning was designed to be enhanced (Candy, 1991).
In order to ensure the learning in the unit was higher order, teaching strategies that promote active learning were embedded extensively throughout the unit. These included essential common elements cited by Bonwell and Eison (1991), such as student involvement in discussing and writing rather than just listening, emphasis on skill development rather than content transmission, and frequent use of higher order thinking skills such as synthesis and evaluation, rather than those of explanation and application.
Preparation of students for this way of learning began at the commencement of the semester when students were informed that they would be required to be actively involved in conversations involving critical thinking during the semester. This was designed into assignments and on campus students spent class time discussing their assignment with their peers, while off campus students were encouraged to actively engage in this unit's Discussion Board so that they also had the opportunity to share with, and learn from, their peers. This activity provided the opportunity for the students to critically consider and appraise their work, which created deeper understanding of the task requirements. Students demonstrated active and engaged learning in dialogue with their peers. We went further with this process as we were conscious of the critical importance of effective formative feedback and the value of engaging students in that process, so a peer reviewing process was introduced. This was designed to offer students further opportunities to enhance the development of graduate attributes and their employability skills whilst they were engaging in meaningful, relevant and deep learning.
A contributing factor in the decision to use a peer review process was the unsustainable lecturer workload that occured in previous years. This method of providing detailed, constructive and individual feedback to students on their first assignment, so that they could revise and improve their work for the second assignment, created a very large workload for the lecturer. Whilst the provision of extensive feedback was considered pivotal to the students' learning and improvement in skills, and the importance of the feedback was acknowledged by the students, it became increasingly challenging for the lecturer to sustain this workload, particularly in the face of growing student numbers.
Furthermore, as responsibility for directing the student learning required to close the gap between current and desired performance fell to the lecturer rather than the students, the lecturer noted that students tended to adopt a very passive approach to the revision of their assignments. Although marking guides were supplied with tasks, the interpretation of these guides fell to the lecturer to explain while students passively awaited explanation. Being passive, students never fully developed an understanding of what the various levels of performance actually looked like. This realisation by the lecturer contributed to her exploring alternative ways of providing quality feedback, whilst enhancing student learning, involving them more as independent learners, and developing employability skills and graduate attributes.
A beginning point of exploration was the realisation that, while the marking guides for the assessment were adequate for the lecturer's marking, they were not as effective a learning tool for students as they could be. In order for students to be more independent in their learning and to be more in control of their learning, the marking guides needed to be improved so that students could accurately interpret them and develop a clear understanding of the learning goals and standards of performance required for each task. This was particularly necessary for the peer reviewing process to be implemented. For a student to validly review a piece of work they had to have a clear understanding of what was required. Whereas in the past the lecturer had interpreted the criteria for the students in the process of providing written feedback, this was not appropriate for peer reviewing, where student would need to provide the feedback. Consequently, a more comprehensive marking guide was developed that was easier for students to interpret.
With the strong motivation to implement a change and the development of a useful marking guide, the path was set for the introduction of peer reviewing. Prior to introduction, planning and contemplation of potential challenges was conducted.
Implementing changes produced a number of challenges for the lecturer, such as how to overcome student resistance to openly sharing their work with their peers. This resistance was not surprising, as Drury, Kay and Losberg (2003) found that, even after explicit teaching of group work skills and structured discussion and reflection on group processes and dynamics, a significant minority of first year students reported negative experiences of group work. Perceived demands that students compete with each other for marks can also create resistance to sharing their work with their peers. Discussions with students revealed that they feared that their work would not be good enough, and that they would be criticised. Yet for graduates of this course, working collaboratively with peers in the workplace is essential. Therefore, the lecturer needed to be aware of the challenges that students may encounter, and their perceptions, and she then had to ensure that she introduced strategies to address the hesitation and resistance of the students. Clear and open communication of the process and the benefits for the students, as well as ongoing support and encouragement by the lecturer, were utilised to ensure students understood the value of being involved in the peer reviewing process.
Ways of engaging students in reviewing their peers' work required considerable planning and preparation, particularly around the actual feedback required and the form it should take. The development of support materials started several months before the semester began. The lecturer developed instructions detailing the process and how to provide the feedback, so that all students had written guidelines of what was expected of them in the peer review process. On campus students were given both verbal and written instructions of the process and time was spent, from the beginning of semester, explaining the process and the expectations the lecturer had of the students. Off campus students were encouraged through Discussion Board and email contact to discuss their concerns and to ask questions in order to gain a clear understanding of the processes.
Time and timing were critical to success. A timeline (see Appendix 1) was set for the semester to ensure that students had sufficient time to learn about, and to reflect on, the peer reviewing process. The process was not rushed as it was planned prior to the commencement of the semester. This long preparation period allowed the lecturer time to consider adjustment and refinements to the process and the supporting documents, such as the guidelines and the marking guides.
All aspects of this unit relating to peer reviewing were planned for prior to the commencement of the semester. This included preparing students for their involvement in the surveys. On campus students were informed both verbally and in written format that they would be asked to be involved in completing two surveys. Similarly, off campus students were also informed via Discussion Board and emails. At the point of administering both the pre and post surveys, on campus students were provided with paper versions and off campus students were emailed the survey.
The pre-survey elicited responses in relation to students' prior experiences of peer review, their response to the preparatory peer review activity conducted using work samples, and their feelings about the review of their peers' work that they would be undertaking a few weeks later. In order to gather this information prior to peer reviewing, this survey was distributed to the students in week four. To cater to the different modes of study, on campus students completed the survey in class whereas off campus students were provided with the survey form to complete electronically. The pre-survey gathered both qualitative and quantitative data and included six questions, with four containing a rating scale as well as the opportunity to provide comments and the remaining two questions requiring comments only.
Immediately prior to the pre-survey being distributed, students were involved in reviewing a sample assignment. The sample assignment was provided to on campus students in class, whereas off campus students were able to access the assignment via Blackboard. The on campus students were given the peer reviewing key, based upon the marking key, and guidelines providing detailed instructions on the peer reviewing process. Students were allocated time to read the guidelines and ask any questions in order to clarify what was required and address any concerns they might have. After they completed this, they started to review the sample assignment. This process took 30-40 minutes and after all students had completed this task individually, students were gathered into groups of three to four and began discussing the sample assignment. In these discussions students were required to justify their comments on the peer reviewing key. They were also encouraged to question each other and to seek clarification from their peer as to their comments.
Similarly, off campus students engaged in the same process, but due to their mode of study students engaged in discussions via Discussion Board. The lecturer encouraged involvement and discussion by monitoring Discussion Board and providing comments where necessary. Off campus students were not limited in time to complete this learning task.
The post-survey canvassed students' views on the usefulness of the peer review exercise and their feelings about engaging in such activities in the future. This survey was administered at the end of the semester in week 12, after students had provided feedback to a peer and had responded to that feedback. Similar to the pre-survey, students undertaking their studies on campus completed the post-survey in class, and off campus students were contacted electronically. The post-survey also utilised the collection of qualitative and quantitative data. This survey contained 14 questions, some of which had rating scales and opportunities for comments, while others required only comments.
Both on and off campus students completed the first survey. There were 15 on campus and three off campus surveys completed. The second survey was completed by 12 on campus students. Unfortunately, no off campus students responded to the second survey. The data gathered from these pre and post surveys were collected, collated and analysed.
Students liked the opportunity to
Evaluating work samples
Question 2 asked students how useful they found the experience of evaluating the work samples. All students appeared to find the experience useful, with five rating it as fairly useful, seven as very useful and three as incredibly useful.
Students who found it incredibly useful liked
Students also found that evaluating work samples
Students commented that the experience:
Students' lack of confidence in their ability to identify areas for improvement was accompanied by comments that
Feelings about the forthcoming review of their own work
Questions 5 and 6 related to the peer review activity which was to be undertaken in week 6, where students would review the work of a peer from their class rather than a work sample.
Question 5 asked: How do you feel about reviewing another student's work? Most students expressed some apprehension, with responses like:
Learning from the experience
Determining the usefulness and the degree of learning students gained from the process was explored in Questions 2-7. Students were asked what aspect of the peer reviewing contributed most to their learning. A total of 83.3% of students indicated that the process of both giving and receiving feedback provided the most learning. Comments from students were insightful and positive. For example one student commented that, "My way of giving feedback was a learning experience as I hadn't formally done anything before. I also learnt how to receive feedback without being closed minded."
Further comments from students were that from the process they identified areas they needed to focus upon to improve their assignments. "I learnt from the strengths in my peers' assignment, and realised how important it was that all parts of the program were linked". Also one student noted that peer reviewing, "Was useful to see where improvements can be made on my own assignment and was good to learn about another issue/program."
A number of students (75%) commented that they appreciated the suggestions they received from their peers. Students were then asked if they would modify their assignment as a result of the peer review activity and there was a response of 91.6% saying that they would do this.
Giving and receiving feedback
A range of questions (Questions 8-12) were presented to the students about giving and receiving feedback. Students were asked about the quality of the feedback that they received from their peer. A total of 83.2% of the students indicated that the quality was excellent (41.6%) or good (41.6%). The remaining 16.8% indicated that the feedback was fair. Feedback provided by the students was also surveyed and 91.6% indicated that they thought the feedback they had provided was excellent (25%) and good (66.6%). One student (8.4%) indicated that she thought she had provided poor feedback.
Students were also asked to indicate if they thought they were prepared for the task. Almost all students (91.6%) indicated that they considered that they were. Only one student said that she wasn't prepared for the task. It is worth noting that this student was the same student who indicated that her feedback to her peer was poor, but also who had a poor attendance record.
How students felt about reviewing and about having another student review their work was explored. Some comments from students indicated that they were apprehensive about the task prior to engaging in the process, but they did realise that the task was useful:
Peer reviewing provides students with the opportunity to encounter greater diversity of perspectives. As indicated by Pearce, Mulder and Baik (2009, p. 3) this is an important benefit of peer reviewing. Through engaging in this experience students received feedback from two people instead of the just the lecturer. Whilst the task of marking and assigning a grade to the assignment remained with the lecturer, the two feedback perspectives offered diversity which is very useful for the students, and reflected the way their work would most likely be reviewed as future professional social workers.
An additional benefit relates to this way of providing feedback. As students engaged in the process they were required to analyse, review, clarify and then provide verbally respectful and meaningful feedback to their peer. The benefits of involvement in the process of reviewing and providing feedback have been acknowledged by many (see, for example, Lundstrom & Baker, 2009; van den Berg, Admiraal & Pilot, 2006). These authors claim that it is not just receiving feedback that is beneficial; it is the giving of feedback that provided clarity for many students. Students in this unit reported that they benefitted from engaging in this process as it enabled them to interact with their peers, to discuss their reviewing, and to provide a different perspective. This perspective assisted some students to view their assignment differently. Students in general appreciated the feedback they received, and stated that they were going to utilise this information to adapt and improve their second assignment. This demonstrates that providing and receiving feedback had been repositioned as an effective learning tool.
Additional benefits to students from engaging in peer reviewing in this unit were developments or expansions of a range of generic skills. Pearce, Mulder and Baik (2009) suggest that engaging in peer reviewing provides opportunities to develop diverse skills, such as problem solving and reflection. It is also suggested that peer reviewing increases a sense of responsibility, promotes independent learning, reduces dependence on the lecture and, very importantly, prepares students for the professional workplace. Students' engagement in this process provided them with opportunities to develop the diverse skills that Pearce, Mulder and Baik (2009) identified.
This experience provided an opportunity for students to enhance their employability skills, while at the same time attending to the development of ECU graduate attributes such as ability to communicate and critical appraisal skills. Engagement in a collaborative learning environment that provided authenticity enabled students to learn skills that would assist them when they entered their profession. Reviewing of peers' work within the workplace is common within the community work sector. Having the opportunity to encounter this, and to learn about the process of reviewing, is consequently beneficial to graduating students. In this unit students worked within groups on a range of activities where it was necessary to communicate respectfully and clearly, both verbally and in written format. They were required to consider ideas and concepts critically, to question and to generate ideas. As students within this unit focused upon a population of their choice, a number of students chose populations from international and cultural perspectives different from their own cultural background. Again, exposure to different cultural perspectives is important for graduates who potentially will be working with diverse communities. Peer reviewing provided the ideal opportunity to develop and/or expand a diversity of skills and enhance a range of attributes essential for graduates.
The benefit of engaging students in peer reviewing extends beyond the students. Lecturers require considerable time to provide appropriate and meaningful feedback that is essential for students' learning (Boud & Molloy, 2012). Through the use of peer reviewing and the comprehensive marking guide created for this task, the allocation of marking time required by the lecturer was reduced. This was a welcome positive consequence.
Some limitations of this research relate to the gathering of the data. Whilst the survey was distributed to both on and off campus students, the return of the survey from off campus students was far less than from campus students. Consequently there is an impact on the data collected. In the future, further consideration will be given to better engaging off campus students in the data collection process. It will be beneficial to gain more information from the students undertaking this mode of study, as the experiences of these students may be different from on campus students.
The survey data collected to date draws attention to the need for careful preparation of students, for both giving and receiving feedback. While the importance of respectful and constructive feedback was emphasised before and during the work sample activity, there was no real focus on how students should receive feedback. This is an area which the researchers would like to refine for future activities.
This process of peer reviewing gave students the opportunity to engage in deep learning within an environment that simulated the workplace. They have, within this unit, had an opportunity to develop a range of attributes and skills that are essential for this next step in their learning journey.
Our experience was that student time spent in guided discussion and reflection about all aspects of the process was extremely valuable. Students' perceptions of the value of the peer review exercise were shaped during these discussions and their view of themselves as professional social workers was enhanced. Students were able to see clear connections between their ways of working and ways of being in the classroom, and their future professional social work roles.
Engaging students in further opportunities to peer review would expand the benefits to students' learning and the development of a range of skills important for these potential graduates. By expanding the use of peer reviewing to more than one isolated unit within a degree, learning and skills development would be expanded and reinforced. As student became more familiar with peer reviewing the issues initially confronted would be reduced as students would be experienced with the process and therefore less likely to be initially intimidated.
Bonwell, C. C. & Eison, J. A. (1991). Active learning: Creating excitement in the classroom. ERIC Digest. http://www.eric.ed.gov/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=ED340272
Boud, D. J. (2012). New conceptions of feedback and how they might be put into practice. Keynote address presented at eAssessment Scotland 2012 Conference, University of Dundee. [verified 23 Jan 2013; PowerPoint slides] http://www.srhe.ac.uk/downloads/events/69_Boud.pdf
Boud, D. J. & Molloy, E. (2012). Rethinking models of feedback for learning: The challenge of design. Assessment and Evaluation in Higher Education. http://dx.doi.org/10.1080/02602938.2012.691462
Boud, D. J. & Falchikov, N. (2006). Aligning assessment with long-term learning. Assessment and Evaluation in Higher Education, 31(4), 399-413. http://dx.doi.org/10.1080/02602930600679050
Candy, P. C. (1991). Self-direction for lifelong learning. San Francisco: Jossey-Bass Higher and Adult Education Series. Carless, D., Salter, D., Yang, M. & Lam, J. (2011). Developing sustainable feedback practices. Studies in Higher Education, 36(4), 395-407. http://dx.doi.org/10.1080/03075071003642449
Drury, H., Kay, J. & Losberg, W. (2003). Student satisfaction with groupwork in undergraduate computer science. Australasian Computing Education Conference. http://crpit.com/confpapers/CRPITV20Drury.pdf
James, R., Krause, K.-L. & Jennings, C. (2010). The first year experience in Australian universities: Findings from 1994 to 2009. http://www.cshe.unimelb.edu.au/research/experience/docs/FYE_Report_1994_to_2009.pdf
Lave, J. & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge, UK: Cambridge University Press.
Li, L., Liu, X. & Steckelberg, A. L. (2009). Assessor or assessee: How student learning improves by giving and receiving peer feedback. British Journal of Educational Technology, 41(3), 525-536. http://dx.doi.org/10.1111/j.1467-8535.2009.00968.x
Lundstrom, K. & Baker, W. (2009). To give is better than to receive: The benefits of peer review to the reviewer's own writing. Journal of Second Language Writing, 18(1), 1-43. http://dx.doi.org/10.1016/j.jslw.2008.06.002
Nicol, D. (2009). Transforming assessment and feedback: Enhancing integration and empowerment in the first year. In Quality enhancement themes: The first year experience. The Quality Assurance Agency for Higher Education, Mansfield, UK. http://www.reap.ac.uk/reap/public/papers//QAA_DN_Paper_final_08.pdf
Nicol, D. (2010). From monologue to dialogue: Improving written feedback in mass higher education. Assessment and Evaluation in Higher Education, 35(5), 501-517. http://dx.doi.org/10.1080/02602931003786559
Nicol, D. & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218. http://dx.doi.org/10.1080/03075070600572090
Oliver, B. (2011). Assuring graduate outcomes. Australian Learning and Teaching Council. http://www.olt.gov.au/system/files/resources/Assuring%20graduate%20outcomes.%20
Pearce, J., Mulder, R. & Baik, C. (2009). Involving students in peer review: Case studies and practical strategies for university teaching. University of Melbourne: Victoria. http://www.cshe.unimelb.edu.au/resources_teach/teaching_in_practice/docs/Student_Peer_Review.pdf
Price, M., Handley, K., Millar, J. & O'Donovan, B. (2010). Feedback: All that effort, but what is the effect? Assessment & Evaluation in Higher Education, 35(3), 277-289. http://dx.doi.org/10.1080/02602930903541007
Rust, C., O'Donovan, B. & Price, M. (2006). A social constructivist assessment process model: How the research literature shows us this could be best practice. Assessment and Evaluation in Higher Education, 30(3), 231-240. http://dx.doi.org/10.1080/02602930500063819
Van den Berg, B. A. M., Admiraal, W. F. & Pilot, A. (2006). Design principles and outcomes of peer assessment in higher education. Studies in Higher Education, 31(3), 341-356. http://dx.doi.org/10.1080/03075070600680836
Wiggins, G. (1990). The case for authentic assessment. Practical Assessment, Research & Evaluation, 2(2). http://pareonline.net/getvn.asp?v=2&n=2
|Week 1||Details of peer reviewing provided to students.|
|Week 2||Students engaged to look at the guidelines to be used in the peer reviewing process.|
|Week 3||Lecturer considers feedback from previous week and if there is any, adjustments are made to peer reviewing document.|
|Week 4||Students provided with sample assignments and Peer Reviewing Key. On campus students undertake a review of these assignments in class. Off campus students engage in the same process through using Blackboard.|
Pre-survey distributed to students.
|Week 6||Students engage in the peer reviewing process in class and online.|
|Week 8||Students are provided with a copy of the review of their work when marked assignments returned.|
|Week 12||Students submit their final assessment in this week. Students are required to consider the feedback provided by their peer. Whilst they do not have to use this feedback the student must consider it and reflect on the feedback, explaining why used or why they didn't use the feedback.|
Post-survey distributed to students.
|The articles in this Special issue, Teaching and learning in higher education: Western Australia's TL Forum, were invited from the peer-reviewed full papers accepted for the Forum, and were subjected to a further peer review process conducted by the Editorial Subcommittee for the Special issue. Authors accepted for the Special issue were given options to make minor or major revisions (minor changes in the case of Moore & Teather). The reference for the Forum version of their article is:
Moore, C. & Teather, S. (2013). Engaging students in peer review: Feedback as learning. In Design, develop, evaluate: The core of the learning environment. Proceedings of the 22nd Annual Teaching Learning Forum, 7-8 February 2013. Perth: Murdoch University. http://ctl.curtin.edu.au/professional_development/conferences/tlf/tlf2013/refereed/moore.htmlAuthors: Catherine Moore is an academic developer in the Centre for Learning and Development at Edith Cowan University. Her work includes supporting staff with the implementation of a University wide curriculum change, with a particular focus on assessment and the development of graduate attributes and employability skills. Her research interests include the use of video in teacher reflection.
Susan Teather is a lecturer in School of Psychology and Social Science at Edith Cowan University. She has over a decade of experience in tertiary teaching in addition to extensive active practice as a social worker. Her passion for her discipline area is equalled by the enthusiasm with which she creates stimulating and authentic learning communities where students are challenged to address real situations by applying knowledge in a personally meaningful context.
Please cite as: Moore, C. & Teather, S. (2013). Engaging students in peer review: Feedback as learning. In Special issue: Teaching and learning in higher education: Western Australia's TL Forum. Issues in Educational Research, 23(2), 196-211. http://www.iier.org.au/iier23/moore.html