QJER logo 2
[ Contents Vol 6, 1990 ] [ QJER Home ]

A summary of a small survey on subjective assessment of student ability

Peter Hallinan
(Peter Hallinan is a lecturer in educational psychology at the University College of Central Queensland.)

This paper concerns itself with a relatively informal and unstructured approach to the assessment of student ability,, through the subjective perception of the teacher. Such assessments are frequently made in the formative stages of learning, and the writer was interested to see what degree of agreement existed among the teachers concerned.


Ten students enrolled in a Graduate Diploma of Teaching (Secondary Mathematics and Science) in a non-metropolitan college of advanced education in 1989 constituted the entire student body under discussion. Seven lecturers, all of whom taught the entire group in different subjects, were provided with the following written instructions in October 1989:
"Please rank these students from 1-10, on the basis of your perception of their academic progress, in the subject(s) you teach.
  1. Avoid tied ranks.
  2. Do not discuss this exercise with any other lecturer before the examiner's meeting.
  3. Please return this sheet to the researcher."
The ten students' first names were placed in random order over the page above these instructions.


Table 1: Staff Judgments of Student Rank Order

(rank order)
KeithBrendaMary SamBillQuentinSarah
Kerry2.2922 31135
Don 2.4311 43341
Marian 2.7155 22212
Isabel 4.2944 17=833
May5.713 3710467
Hilda 5.7966 57=57.5=4
Bobby 6.6477 67=6.5=58
Neville 7.5788 107=6.5=7 5=6
Raymond 8.7199 87=1099
Edith 9.231010 97=91010
* Names have been altered to preserve anonymity.

Table 2: Spearman Rank Difference Correlation Coefficients of Lecturer Perceptions

KeithBrenda MarySamBillQuentinSarah
Agreement with Mean .94* .94*.82*.64.85*.89*.81*
1.00*+ .58.39.78*.75*.73*






* p<.01
+ spuriously high correlation

Table 3: Correlation Between Mean Subjective Perception and Mean Final Grade

Kerry2.29 15.172
Don2.43 25.351
Marian2.71 34.884.5
Isabel4.29 44.826
Bobby 6.64 74.884.5
Neville7.57 84.358
Edith 9.23103.6810

Discussion of Results

This was a deceptively simple exercise and the data should be treated with caution. Despite the admonition to avoid tied ranks, three of the seven staff found these impossible to avoid. Bill wanted to use letter grades rather than ranks. Sarah found it difficult to recall the students in detail as she only taught them in the autumn semester, and this exercise was carried out towards the end of the following spring semester. Sam insisted on a large number of tied ranks, as he found it impossible to discriminate the group by ability, except for the highest and lowest individuals.

In considering Table 1, Bill may have a point, as the staff generally seemed more in agreement about the top students (Kerry and Don) and the bottom students (Raymond and Edith) than about the bulk of the students in the middle ability range. Despite the exhortation to avoid discussion of results, there are grounds for skepticism in Keith and Brenda's case.

In considering Table 2, Spearman's rho was used due to the nature of the data and the small size of the sample (Van Dalen, 1979:491). Even discounting Keith and Brenda, the large number of significant correlations testifies to a fair degree of consensus among the staff members concerned. In Sam's case, his idiosyncratic approach, possibly linked to his particular academic discipline, led to his being the 'odd man out'. By contrast, despite her concern over elapsed time, Sarah reached a high degree of consensus with other staff. In terms of shared discipline areas, Keith with Brenda, Mary with Quentin, and Sam with Bill, could be said to share some common professional linkages.

Turning to Table 3, a grade point average over both semesters was also computed, using a five-point range from three points for a failing grade through to seven points for a grade of high distinction. With regard to Table 3 above, since the data were of rank-order nature, the SPSS NONPAR CORR procedure was used to compute the correlation between SS Mean and Mean GPA (SPSS, User's Guide, 1986, p.801). Since the direction of the relationship between variables was apparent in advance of the analysis, the significance levels of the coefficient was based on a one-tailed test (SPSSX User's Guide, p.803). Spearman and Kendall coefficients of correlation were computed. Both showed a high correlation between mean subjective assessment of rank order and mean GPA (p <.000 and p <.001 respectively).

In summary, while teacher-initiated chance meetings and conversations over the teaching year are clearly at the less formal and less structured end of the assessment continuum (Grace and Cassidy, 1988:78), a high degree of consensus among staff does exist, particularly for those students at the ends of the ability ranges, both high and low. This consensus is all the more noteworthy given the range of variability in attitudes and practices in assessment across the tertiary disciplines represented: language arts, mathematics, philosophy, psychology and sociology. Future studies might focus on the degree of consensus within and among specific disciplines, and the degree of consensus between judgments of tertiary staff at the preservice phase of the students' career, and those of educational administrators such as principals and inspectors, in subsequent years.


Thanks are due to Maj Reugebrink and Blair Smith for assistance with the statistical aspects of this paper.


Grace, N. & Cassidy, J. (1988) Interim Report of the Task-Force on Assessment Alternatives in Mathematics, Years 1 to 10 Mathematics, Brisbane: Curriculum Development Services, Department of Education, Queensland.

SPSSX Users Guide: Edition 2 (1986) Chicago: SPSS.

Van Dalen, D.B. (1979) Understanding Educational Research: An Introduction, New York: McGraw Hill.

Please cite as: Hallinan, P. (1990). A summary of a small survey on subjective assessment of student ability. Queensland Researcher, 6(2), 16-20. http://www.iier.org.au/qjer/qr6/hallinan.html

[ Contents Vol 6, 1990 ] [ QJER Home ]
Created 30 Sep 2006. Last revision: 30 Sep 2006.
URL: http://www.iier.org.au/qjer/qr6/hallinan.html