Maximising return rates in school based research
Hale School, Perth
Edith Cowan University
Maximising the return rate of questionnaires is a potential problem facing researchers investigating school-based issues. In Western Australia, this potential problem has become reality. The problem is particularly acute for part-time researchers engaged in postgraduate studies.
This article outlines the data collection procedures implemented by a part-time researcher in an attempt to maximise the return rate. The discussion then examines strategies adopted by researchers who have achieved a return rate in excess of seventy per cent. The article concludes by suggesting that potential researchers consider carefully the strategies discussed before embarking on their own investigations.
Maximising the return rate of questionnaires is one of the issues facing school-based researchers. Such an issue is of particular importance to part-time researchers, many of whom have limited time and financial resources. In the context of this discussion a part time researcher is defined as an individual who has a full-time occupation outside the field of educational research. By comparison, a full-time researcher is an individual whose occupation is conducting research in education.
A recent inquiry into occupational stress among primary school teachers in Western Australia (Lock, 1994) achieved a return rate of only thirty-five per cent. This return rate is low by comparison with other school-based research conducted in Australia which also investigated teacher stress (for example, Fordham and Hunt, 1984 (76%); Laughlin, 1984 (70.3%); Van Schoubroeck and Tuetteman, 1986 (77%) ). Anecdotal evidence collected from other researchers (at seminars and conferences) during the period of Lock's (1994) study suggested that the low return rate which he experienced is not an atypical phenomenon being encountered by researchers conducting inquiries on a wide range of school-based topics in Western Australia.
The following discussion will describe the procedures used by Lock (1994) (a part-time researcher) to collect data for his inquiry, outline the reasons given to him concerning teacher non-participation in his investigation and discuss alternative approaches to conducting school-based research with the aim of maximising the return rate of questionnaires. While most of the information for the latter section of the discussion will be derived from studies which investigated teacher stress, the strategies outlined should be of use to any researcher who relies on questionnaires to collect data.
Collecting data for an investigation into teacher stress
Before commencing his study Lock (1994) sought permission from the Ministry of Education (now the Education Department) of Western Australia to conduct research in government schools. In addition to the letter written to the Director of the Policy and Resources Division of the Ministry of Education in which the nature of the study was outlined, letters were also written to the State School Teachers' Union of Western Australia and the Western Australian Primary Principals' Association seeking support for the project. Positive responses were received from the Ministry of Education and the State School Teachers' Union, but no reply was forthcoming from the Primary Principals' Association.
Ironically, the support received from the Ministry of Education and the State School Teachers' Union sometimes proved to be a barrier to teacher participation in the study. Some teachers refused to complete the questionnaire when they were informed of Ministry and/or Union support for the research. Following acknowledgment of approval from the Ministry of Education, letters were mailed to principals of schools selected to participate in the research. These letters requested principals to approach their staff to seek willingness to become involved in the research and provided a sample of the questionnaire (which contained eight sections with a total of one hundred individual items) to ensure that potential participants knew the requirements of such involvement.
Between one and two weeks after the letters had been mailed, telephone calls were made to the principals of the schools to ascertain the response of their members of staff. In the case of positive responses, the number of questionnaires required was noted and distribution dates were arranged. By ascertaining, relatively accurately, the number of questionnaires required, cost savings occurred by the avoidance of unnecessary duplication. Overall, seven hundred and fifty four teachers were approached in this manner. From this number, two hundred and sixty-four usable questionnaires were collected. In effect, this represented a return rate of thirty-five per cent. Reasons for teachers' unwillingness to participate in the study, as communicated to Lock (1994) by letters and conversations on the telephone, included
Distribution and collection of the questionnaires were undertaken personally. While this proved to be a time-consuming exercise, a number of advantages resulted from making such an effort. First, personal contact was made with the principals of the various schools. This helped to establish the credibility of the researcher, and, in some cases, resulted in discussions from which valuable qualitative information was obtained. Second, clarification of the instructions which accompanied the questionnaire could be accomplished. Third, the principals were able to be assured personally that they would receive results of the survey - both for their own school and the total sample. Some principals indicated that the results of the study would be used as a basis for future policy decisions within their school. Fourth, the fact that the researcher was willing to undertake personally the distribution and collection of the questionnaires demonstrated a desire to cause minimum inconvenience to those who participated in the research. Most of the principals commented favourably on this aspect of the research process, particularly in comparison to other surveys in which their school had participated.
Overall, the data collection procedures adopted by Lock (1994) were designed to make effective use of his limited available time. While these procedures were appropriate in regard to this criterion, they were relatively ineffective in maximising the participation rate among the sample population. Although savings occurred by avoiding the unnecessary duplication of questionnaires, these were somewhat negated by the transportation costs due to having to visit a larger number of schools. The ideal situation would have been the achievement of a higher return rate from a smaller number of schools. The next section will examine procedures by which this might be achieved.
Other data collection procedures
To ensure a higher return rate than that obtained by Lock (1994), future researchers investigating school-based issues in Western Australia might consider alternative approaches to those described previously. Reference to literature on undertaking survey research should prove to be a useful starting point. Cohen and Manion (1985), for example, provide ideas in regard to planning and implementing a survey. They discuss general aspects including the purpose of a survey, the survey population and available resources. Specific issues which these authors examine include sampling, sample size, sample error, conducting postal surveys - including maximisation of the return rate, designing a self-completion questionnaire, and processing survey data.
While this type of literature may prove to yield some useful information, reference to studies conducted in schools which have enjoyed high return rates should also be undertaken. In regard to studies which have achieved return rates in excess of seventy per cent, four characteristics of the data collection procedures emerge. First among these characteristics is the fact that some researchers (Galloway, Panckhurst, Boswell, Boswell and Green 1984; Pettegrew and Wolf, 1982; Whiteman, Young and Fisher, 1984) visited the schools in which the inquiries were to be conducted to discuss the studies with the teacher concerned. In the study conducted by Galloway et al. (1984) a research officer visited the schools prior to the questionnaires being sent to the teachers. This officer explained the nature of the study and reassured teachers about the confidentiality of their responses. Pettegrew and Wolf (1982) gained entry into their target schools via introductions from the district superintendents. They advised principals about the nature of their study and that their participation, and that of their staff, would be voluntary. The researchers were then invited to a staff meeting at which the intentions of their inquiry were outlined and the cooperation of the teachers sought. Whiteman et al. (1984) communicated directly with the principals of each school to establish the best time in which their instrument could be administered.
The second characteristic was the successful use of postal surveys which incorporated follow-up reminders. Researchers who adopted this approach included Dewe (1986) and Foxworth and Karnes (1983). Both inquiries followed similar procedures. Dewe (1986) mailed questionnaires to the teachers at their school addresses. Three weeks later all teachers were sent a follow-up questionnaire. Foxworth and Karnes (1983) also mailed questionnaires to teachers. Each questionnaire was accompanied by a letter which sought cooperation, explained the purpose of the study and assured the recipient of confidentiality. Two weeks later a follow-up letter was sent which expressed appreciation for early response and a reminder to late respondents.
The use of a questionnaire containing less items than in the present study represented the third characteristic. Examples of investigations in this respect were conducted by Abbey and Esposito (1985), Laughlin (1984), Manthei and Solman (1988) and Whiteman et al. (1984). Laughlin (1984) and Manthei and Solman (1988) requested information about seven socio-biographical characteristics. The former's survey also included twenty seven fixed response items and one open-response question. The latter's questionnaire comprised, in addition to the socio-biographical section, thirty-eight fixed response items and one question which required the respondents to write the number of days lost through sickness during a specified time period. Also of interest is the fact that Laughlin (1984) and Manthei and Solman (1988) used essentially the same procedures to those used in the present study when making contact with the schools selected to participate in their investigations. The questionnaire developed by Abbey and Esposito (1985) contained thirteen fixed-response items, while that used by Whiteman et al. (1984) comprised five questions in relation to socio-biographical characteristics and eight fixed-response items.
The size of a questionnaire to be used in an inquiry is generally related to the aims of the study. In the case of Laughlin (1984) information was required on a variety of issues including sources of stress, rating of occupation in relation to overall stress and satisfaction and sources of job satisfaction. Manthei and Solman (1988) sought data on sources of stress, rating of teaching as an occupation in relation to overall stress and satisfaction and responses to the twelve-item version of the General Health Questionnaire. Abbey and Esposito (1985) based their questionnaire on previously developed instruments which were used to collect information regarding a principal's compliance system and perceptions of social support. Whiteman et al. (1984) also used previously developed instruments to examine teacher burnout and perception of student behaviour.
The issue of questionnaire size might not be easy to resolve. If previously developed instruments can be utilised this may prove to be of benefit in keeping a questionnaire relatively brief. However, if the researcher is exploring an area in which previous research has not occurred, or the aims of the inquiry require the collection of a large quantity of data, then the size of the questionnaire, might, of necessity, be quite considerable.
The fourth characteristic was evident in research conducted by Pettegrew and Wolf (1982) and Whiteman et al. (1984). During these investigations teachers completed the questionnaires during staff meetings. The questionnaire used by Pettegrew and Wolf (1982) was completed during a regular staff meeting, while for Whiteman et al. (1984) time was granted, after negotiation with principals, for staff to complete the instrument either during afternoon release/inservice time or staff meeting time immediately following the close of the school day.
Future researchers who intend to investigate school-based issues in Western Australia may wish to heed the information in the previous discussion. Their ability to utilise any of the procedures discussed may be influenced according to whether they are part-time or full-time researchers. No information on this issue was discernible from the references on which the previous discussion was based. Nonetheless, adherence to all, or some, of the above-mentioned methodological aspects may result in a return rate in excess of that obtained by Lock (1994).
Abbey, D.E. & Esposito, J.P. Social support and principal leadership style. Education, 105, 1985, (3), 327-332.
Bacharach, S.B., Bauer, S.C. & Conley, S. Organizational analysis of stress: the case of elementary and secondary teachers. Work and Occupations: An International Sociological Journal, 13,1986, (1), 7-32.
Cohen, L. & Manion, L. Research methods in education (Second Edition). London: Croom Helm, 1985.
Dewe, P.J. An investigation into the causes and consequences of teacher stress. New Zealand Journal of Educational Studies, 21, 1986, (2), 145-157.
Fordham, A. & Hunt, J. Teachers' views of ACT high school education: Research report. A study undertaken by the Evaluation and Research Section of the ACT Schools Authority for the Committee to Review Education in ACT Government High Schools. Canberra: ACT Schools Authority, 1984.
Foxworth, M.D. & Karnes, F.A. Occupational stress and the teacher of the gifted elementary students: a preliminary investigation. Roeper Review, 6, 1983, (2), 105-107.
Galloway, D., Panckhurst, F., Boswell, K., Boswell, C. & Green, K. Mental health, absences from work, stress and satisfaction in a sample of New Zealand primary school teachers. Australian and New Zealand Journal of Psychiatry, 18, 1984, (4), 359-363.
Laughlin, A. Teacher stress in an Australian setting: the role of biographical mediators. Educational Studies, 10, 1984, (1), 7-22.
Lock, G.J. (1994). The prevalence and sources of perceived occupational stress among teachers in Western Australian Government metropolitan primary schools. Unpublished doctoral dissertation, Edith Cowan University, Perth.
Manthei, R. & Solman, R. Teacher stress and negative outcomes in Canterbury state schools. New Zealand Journal of Educational Studies, 23, 1988, (2), 145-163.
Pettegrew, L.S. & Wolf, G.E. Validating measures of teacher stress. American Educational Research Journal, 19,1982, (3), 373-396.
Van Schoubroeck, L. & Tuetteman, E. Teacher stress. report to the joint committee of inquiry into teacher stress. Perth: Ministry of Education, 1986.
Whiteman, J.L., Young, J.C. & Fisher, M.L. Teacher burnout and the perception of student behaviour. Education, 105,1985, (3), 299-305.
|Authors: Dr Graeme Lock is Head of the Department of History
at Hale School in Perth, Western Australia. Recently he completed his PhD
at Edith Cowan University, where he investiagated teacher stress among
teachers at primary schools. Associate Professor Dr Sybe Jongeling is the Director of Research and
Development Services at Edith Cowan University in Western Australia.
Please cite as: Lock, G. & Jongeling, S. (1994). The mechanics of qualitative analysis. Issues In Educational Research, 4(2), 109-105. http://www.iier.org.au/iier4/lock.html
© 1994 Issues in Educational Research
Last revision: 6 Nov 2013. This URL: http://www.iier.org.au/iier4/lock.html
Previous URL: http://education.curtin.edu.au/iier/iier4/lock.html
Previous URL from 24 July 1997 to 7 Aug 2001: http://cleo.murdoch.edu.edu.au/gen/iier/iier4/942p109.htm
HTML: Clare McBeath [email@example.com] and Roger Atkinson [firstname.lastname@example.org]