Supplement to Queensland Researcher, Journal of the Queensland Institute for Educational Research, ISSN 0818-545X Vol.7, No.2, 1991
It is an honour to be invited by QIER to open this Educational Research Forum. Over a number of years I have witnessed various innovations in the area of reporting research. As research meetings in the USA became horrendously large and programs became overcrowded, the round table was introduced. In this format, several quiet discussions were conducted simultaneously in the same room, with delegates able to spend short or long periods at one round table before moving on to the next. This approach was efficient for both presenter and audience, and often less daunting because of the more informal setting. Another approach which has become popular is the poster display. By combining features of both approaches, QIER have created a forum for researchers to disseminate their recent and continuing work, and to foster discussion and interaction among the research community. Over the past several years, several staff from the Schonell Centre have used poster displays as an end-of-course assessment and this has been well received by students.
Some of you will know that I have had quite a bit of involvement in the Australian Association for Educational Research. At one of its Annual Conferences held at Griffith University some years ago, we sought to bring research and practice closer together by instituting a teachers' day and by holding a symposium on the relevance of educational research to the daily activities of teachers. As I recall on that occasion, we 'professional researchers' found the task of convincing teachers of the value of educational research to be no 'pushover'.
One of the most frequent comments is that educational research only produces obvious answers to questions. The eminent researcher, Nate Gage of Stanford University, recently examined this issue. He noted that exemplary research has often produced findings which a keen observer of human behaviour would have noted. Indeed, a partial reflection of the truth of this statement can be found in the newer, qualitative research methods which have become more popular. The techniques of the investigative journalist, the photo-essayist and the novelist or playwright can now be subsumed within the framework of educational research.
Some of the critics of educational research (other than teachers whose views we ought to treat seriously) have argued that some educational research findings are merely truisms. Let us take one such example - the research on instructional time. It would indeed be strange if children learned more if they spent less time on some topic. But what if what were 'obvious' were to prove false? We would then need to test the truth of all patently true or common sen se notions in case they were false. For example, there may be limits to the duration over which there is a simple relationship between amount learned and time expended. Another well known example is that relationships can be non-linear, which may not be at all obvious. Take class size and achievement. If we are contrasting a reduction of five students between 35 and 30, the impact is clearly different from between 12 and seven, or six and one!
A popular research paradigm is referred to as naturalistic/ethnographic and we might ask how this differs from the data gathering and conclusion drawing of the person in the street. One obvious factor is that there is so much more richness, so many caveats, qualifications and cautions, so much acknowledgment of the complexity of phenomenon such research work which are usually missing in the conclusions drawn by the casual observer, who is likely to make a bald assertion.
Take a phenomenon like literacy. In everyday use, we might think this to be a simple concept. But qualitative researchers have gone beyond simple definitions. Scribner and Cole (1981) described the three literacies of the Vai people - Arabic oral memorisation in the mosque, English for schools and Vai script for trade and community activities. Fishman's (1988) ethnography of Amish literacy noted the constraints on what, where, how and why writing and reading occurred in Amish homes and schools. Such insights give the lie to statements that everybody knows what reading is.
Gage (1991) goes on to report a study by Baratz who found that when presented with a statement like 'single women express more distress over their unmarried state than single men do', most people agree. Yet when presented with the opposite statement, the majority also agree. Thus to paraphrase Shakespeare 'there's nothing true or false but making an authoritative statement makes it so' or following Lewis Carroll 'I have said it three times ...'. Might I commend Gage's article to you.
What of the political level - one which has been in the news this past week! We have been exhorted to be a clever country. But true intellect takes time and planning to develop. A better educational experience for students at all levels demands a research base. I am sure all the researchers represented here tonight wish that funding was more easily available. I believe that we need to publicise our work better.
Of course, what we should study is a question likely to produce different answers among politicians, parents, principals and teachers. For example, the Australian College of Education Newsletter recently reported two priorities - educational management and the teaching profession. I expect that others might think that students are worth studying also!
Finally, I want to comment on how we make progress in educational research. First, as my colleague Royce Sadler (1990) has explained, researchers must travel Up the Publication Road. Second, the heart of publication is peer review. By way of personal confession, I want to state that I have learned more about research through reviewing articles for scholarly journals than any other activity. It is salutary to receive a manuscript, make one's best academic analysis of it, then receive the views of two other reviewers and the journal editors (and occasionally a response from the authors). That this is a process open to human error is plain - yet it is the best system possible. Let me finish in a light hearted fashion by sharing a few gems from the retiring editors of the Reading Research Quarterly for which I have served as a reviewer over the past eight years.
The It Ain't Over Till It's Over Award for resubmitting his paper (at the Editor's request) after it had been rejected by the same editors one month earlier to A (it may have had its flaws, but we kept thinking back to it because it was so interesting - published in Vol. 23, No. 3).
The Most Surprising Research Finding Award to B and C, who wrote that 'girls at these age levels [10 to 13] tend to devote significantly more time to reading than boys ...'
The "Midnight Oil" Marathon Award for completing the most reviews during our tenure as editors to D and E - 40 each.
Hare Awards for speediest turn-around on reviews (under three weeks on average!) go to F and G.
Tortoise Awards for completing reviews in their own time go to H and I. The Friends and Colleagues Don't Always See Eye-to-Eye (OR: Reviewing Is An Inexact Science) Award to J and K - see the following reviews of the same paper
Recommendation: RejectThe What's My Line? Award for suggesting the perfect person to review the piece when the reviewer didn't have time to review it himself - the person suggested was, in fact, the author herself. This award goes to L (the article was by Jennifer Monaghan, and is due to come out in our last issue).
This is a paper that, by the title, seems to hit all of the current 'hot' topics. However, it fails to deliver any useful information, containing serious flaws at almost every level. First, ... there is no coherent theoretical reason for conducting the study ... More serious is the design of the study ... Despite any random assignment, the treatments are confounded with other units and consequently indeterminate with regard to effects ... Despite the significant differences among treatment, I cannot find any interpretable bit of evidence in this study.
Recommendation: Conditionally Accept
Frankly, I find little to quarrel with in this study. As an instructional study, the design is exemplary, including random assignment of subjects to treatment, ... multiple dependent measures, ... safeguards against threats to internal validity, and a rigorous statistical design. And, as if that wasn't enough, the question is theoretically interesting and educationally relevant.
The I'm No Fool Award for actually sending in a review of his own manuscript to Charles Perfetti.
Recommendation: Accept as is[With acknowledgments to the retiring editors of the Reading Research Quarterly - Phillip Gough, James Hoffman, Connie Juel and Dianne Schallert.]
This is a very interesting, well written paper ... My only reservation about the paper is the relatively small sample size. A large number of subjects would have been better. The authors, however, recognise the limits imposed by their sample size ... I do have one additional question to raise ... Probably one of the reasons I found this work so interesting is that I have done some work on the same topic, raising some of the same issues and using some of the same experimental tasks. I have even reached conclusions identical to those of the authors. While reading the paper, I kept thinking half out-loud, 'This sounds like something I would say.' Other times I found myself thinking, 'This is great! I wish I had said this.' As I mentioned, this is awkward for me to raise, because I'm afraid it will sound like sour grapes.
To be clear, I am not saying that the authors are trying to pass off my ideas as their own. They appear to acknowledge that they were influenced by some of my published research and have cited a number of my papers. Of course, it is possible that they cited me because they thought I might be asked to review the paper, in all-too-frequent form of scholarly aggrandisement.
In any case, my concern is not about citations, but about the implications of this kind of twin-earth-problem for the field as a whole. If other people think about things as I do, then, naturally, I should be flattered. But there is a value to diversity. It would be better for the field to have papers on the same topic that challenge one another, rather than ape one another. This is my real concern, not sour grapes.
I guess my bottom line here is that this is a fine paper. I wish I had written it.
Gage, N. (1991) The obviousness of social and educational research results. Educational Researcher, 20, 10-16.
Sadler, R. (1990) Up the publication road (2nd ed.). Campbelltown, NSW: HERDSA.
Scribner, S. & Cole, M. (1981) The psychology of literacy. Cambridge, MA: Harvard University Press.
|Please cite as: Elkins, J. (1991). Opening address, QIER Research Forum, June 1991. Queensland Researcher, 7(2), supplementary pages 1-4. http://www.iier.org.au/qjer/qr7/elkins.html|