Cognitive interviews for reviewing multiple-choice items in mathematics

Joan Burfitt
The University of Western Australia

During the development of a test of multiple-choice items, cognitive interviews were conducted with students from lower secondary school. The purpose of these interviews was to confirm that the prospective respondents' interpretation of mathematics test items was consistent with the interpretation intended by the item writer. The conversations with the students provided unexpected and intriguing insights into the way students think and speak about mathematical concepts: ideas worthy of comment and further research. The benefits and limitations of using cognitive interviews to improve assessment items, and the recommended procedures for conducting such interviews, had been identified from earlier research studies. With these findings, a framework was developed for designing and implementing a series of interviews with ten Year 9 students in Western Australia. The items reviewed by these students were intended for an online test of the skills and understandings essential for the development of sound proportional reasoning for Year 8 students. The responses to the interview questions not only provided useful suggestions on how to improve the test items and on the suitability of the test instrument, but they also provided valuable information about the students' understanding and communication of what they had learned.
[ PDF full text for this article ]

Author: Joan Burfitt is enrolled as a PhD candidate at The University of Western Australia. Her interests include the teaching, learning and assessment of mathematics for students in the middle years. Joan's current research focuses on improving the function of multiple choice items in assessing student achievement. Email: joan.burfitt@research.uwa.edu.au Please cite as: Burfitt, J. (2019). Cognitive interviews for reviewing multiple-choice items in mathematics. Issues in Educational Research, 29(2), 346-362. http://www.iier.org.au/iier29/burfitt.pdf