Using ChatGPT as an assessment tool in education: A systematic literature review of practices and limitations

Thinley Wangdi
Walailak University, Thailand
Karma Sonam Rigdel, Tashi Dawa and Kinga Tshering
Ministry of Education and Skills Development, Bhutan

In the last two years, there has been a significant increase in research studies on ChatGPT and its role in educational assessment. However, there is no comprehensive systematic literature review (SLR) on the potential use of ChatGPT for educational assessment, particularly with a focus on its practices and limitations. To address this gap, our SLR examined 17 research articles published between November 2022 and February 2024 that addressed assessment practices using ChatGPT and limitations in educational fields. These articles were selected using the PRISMA protocol from six research databases. The existing research highlights ChatGPT's potential role in educational assessment, particularly in generating assessment content and questions, enhancing the readability of assessment questions/items, grading student work, and providing adaptive feedback, among other assessment-related tasks. However, it has also been reported that ChatGPT is an unreliable assessor/grader and produces inconsistent, lengthy, and semantically overlapping/incorrect texts when developing assessment content and questions, often requiring human intervention to fine-tune them. Our review concludes with a suggestion for further research on assessment practices and the limitations of using ChatGPT as an educational assessment tool, employing various research methodologies across different educational levels, disciplines, and contexts to fully understand its potential. The implications for practice and directions for future research are also discussed.
[ PDF full text for this article ]

Authors: Thinley Wangdi (corresponding author) is an Assistant Professor at the School of Languages and General Education, Walailak University, Thailand. He is recognised as a Fellow by the UK's Higher Education Academy (FHEA). His research interests include educational psychology, teacher education, and English language teaching. His recent publications have appeared in journals such as South Asia Research, TESL-EJ, and the Journal of Multilingual and Multicultural Development. ORCID: http://orcid.org/0000-0002-0811-0525 Email: thinley11@gmail.com Karma Sonam Rigdel teaches at Wangbama Central School, Thimphu, Bhutan. He holds an MEd from Samtse College of Education, Royal University of Bhutan. His research interests include areas related to language and geography teaching, educational psychology, creative education, educational technology, and education. ORCID: https://orcid.org/0009-0003-5540-3990 Email: ksrigdel2018@education.gov.bt Tashi Dawa currently serves as a teacher in the Department of Inclusive Education at Yangchen Gatshel Higher Secondary School, Thimphu, Bhutan. He holds a Bachelor of Education in Primary Education from Samtse College of Education, Royal University of Bhutan. His research interests include inclusive education, English language writing, and educational technology. ORCID: https://orcid.org/0000-0001-7980-9088 Email: dawatashy@gmail.com Kinga Tshering is an Education Monitoring Officer under the Ministry of Education and Skills Development of Bhutan. He possesses a Bachelor degree in Education with a specialisation in Physics and Mathematics, as well as a Masters degree in Education in Physics. His research interests focus on STEM pedagogy and educational methodologies. ORCID: https://orcid.org/0000-0001-7907-9767 Email: kingat@moesd.gov.bt Please cite as: Wangdi, T., Rigdel, K. S., Dawa, T. & Tshering, K. (2025). Using ChatGPT as an assessment tool in education: A systematic literature review of practices and limitations. Issues in Educational Research, 35(2), 818-837. http://www.iier.org.au/iier35/wangdi.pdf