IIER's Tier B ranking survived unchanged during the ARC's revisions from its draft list in 2008 to its final list in 2010, although a number of Australian educational research journals suffered demotions, for example Higher Education Research and Development, Australian Educational Researcher, Australian Journal of Education, Australasian Journal of Educational Technology and Australian Educational Computing (Atkinson, 2010a; Atkinson, 2010b).
However, the Tier ranking for IIER (and all 20,712 journals in the current list) is now under a threat created by the ARC's recent announcement (ARC, 2010b) of a "Review of the ERA 2010 Ranked Outlet Lists":
The ranked journal and ranked conference lists form an integral part of the ERA evaluation process.Our perception of a threat arises from two aspects. Firstly, the first sentence of the ARC's announcement, phrase "...lists form an integral part..." seems to reiterate its determination to continue the Tiers ranking system, notwithstanding the criticism it has received. Secondly, the ARC's schema is rigidly normative (5% A*, 15% A, etc) and thus in order to promote one journal to a higher Tier, another journal has to be demoted to a lower Tier. Whilst Tier A* and Tier A journals are likely to be under a greater threat of a competitive demotion compared with Tier B journals, it would be prudent for IIER to be represented vigorously in the forthcoming public consultation period promised for "early 2011".
In preparation for the ERA 2012 round, the ARC will revise the ERA 2010 ranked journal and conference lists. This process will involve a public consultation period, followed by a review and finalisation phase supported by peak bodies and other academic groups. (ARC, 2010b)
The public consultation period and its follow up in "a review and finalisation phase supported by peak bodies and other academic groups" is likely to become, as one well-known senior academic suggests, a stoush:
It's evident from recent coverage, that Australian academe is winding itself up for another stoush about journal rankings for the Excellence in Research for Australia exercise.The word stoush is no doubt well-chosen, though as the Macquarie Dictionary indicates, it is quite a forceful word:
... It doesn't make sense to stick with the existing rankings as we move towards a second ERA in 2012. But can we afford to go through all that aggravation again? (Goodyear, 2010)
stoush Colloquial - noun 1. a fight 2. physical aggression; violence: the group were dealing out stoush to the terrified customers. 3. an artillery bombardmentRecent criticism about journal rankings has appeared in The Australian and other publications. Numerous illustrative snippets may be found readily, for example 'clique bodies' (Atkinson, 2010c); 'Ditching journal rankings and using a direct comparison of citations...' (Smith, 2010); '...the mysterious last-minute fiddling with rankings' (Goodyear, 2010); '...ranking of some Australian journals was improved if the editors and interested parties lobbied...' (Levey, 2010); '...the draft ranking was overturned by a process shrouded in mystery' (Bourke, 2010); 'Transparency should be a top priority when the ARC reviews the controversial journal ranking component of its research assessment process' (Rowbotham, 2010); 'Tier Review Process' (Atkinson, 2010d). We hasten to add that the purpose for 'snippets' or brief phrases is to tempt you to read widely on this topic, rather than fully inform you in a direct way! Academic journals and conference proceedings are now beginning to publish research probes into the ARC's use of journal rankings and its impact. Some examples are listed by Atkinson (2010c, 2010d), and this issue of IIER contains a fine contribution by Brian Hemmings and Russell Kay (Hemmings & Kay, 2010).
Criticism of the ARC's journal rankings is linked in many instances to perceptions of deficiencies in the assignment of journals to Fields of Research ('FoR') (ARC, 2010c), for example as indicated by Gough (2008), Graham (2008) and Sid Bourke (a former editor of IIER) (Bourke, 2010). In IIER's case, the FoR assignment is '13. Education', an FoR containing 130 ranked journals comprising 9 A* (6.9%), 16 A (12.3%), 42 B (32.3%) and 63 C (48.5%) (plus 1 unranked). From a selfish perspective, a journal in '13. Education' (for example) may seek to help retain or advance its Tier ranking by lobbying for re-assignments of higher ranked journals into other FoRs (e.g. 1301 Education Systems; 1302 Curriculum and Pedagogy; 1303 Specialist Studies in Education), and for re-assignments of lower ranked journals from other FoRs into '13. Education'.
Another angle for lobbyists will be to look for alternative rankings done by researchers other than the ARC's unknown rankers who did the '...mysterious last-minute fiddling with rankings' (Goodyear, 2010) or were drawn from 'clique bodies' (Atkinson, 2010c). For example, Australasian Journal of Educational Technology has published an alternative ranking for educational technology journals, using the Thomson Reuter's Impact Factor as a basis, and (naturally enough) it is favourable to AJET's case for promotion (Atkinson, 2010e). Genoni, Haddow and Dumbell (2009) provide a more extensive research study of selected social sciences and humanities journals.
We express the hope that the forthcoming "stoush" will be scholarly and evidence based, and that it will not degenerate into "... a 'divide and conquer' regime, as societies and other interested parties may seek to both promote 'their' journal, and belittle some of the higher ranked journals that 'their' journal could displace" (Atkinson, 2010d). We also wish to record with gratitude the support that Tier B ranked IIER continues to receive from authors and reviewers. We dare to hope that mostly or even in sizable majority they are not allowing The tyranny of Tiers to influence them unduly in their support for this journal.
Roger Atkinson and Clare McBeath
IIER Website Editor and IIER Editor
ARC (2010b). Ranked Journal List. http://www.arc.gov.au/xls/ERA2010_journal_title_list.xls (Excel format, 5.27 MB, apparently dated 9 February 2010). For those who find the ARC's file too cumbersome and slow to handle on their personal computers, we recommend highly John Lamp's service (Lamp, 2010).
ARC (2009c). Australian and New Zealand Standard Research Classification (ANZSRC). (PDF, 4.5 MB). http://www.arc.gov.au/era/ANZSRC.htm (again, we recommend highly John Lamp's service).
Atkinson, R. (2008). AJET Editorial 24(2): IM 26: Obituaries for the RQF. http://www.ascilite.org.au/ajet/ajet24/editorial24-2.html
Atkinson, R. (2010a). AJET Editorial 26(1). Journal rankings: AJET demoted. http://www.ascilite.org.au/ajet/ajet26/editorial26-1.html
Atkinson, R. J. (2010b). Bibliometrics out, journalmetrics in! HERDSA News, 32(1). http://www.roger-atkinson.id.au/pubs/herdsa-news/32-1.html
Atkinson, R. (2010c). AJET Editorial 26(6). A Letter to the Editor: Defending the ERA initiative. http://www.ascilite.org.au/ajet/ajet26/editorial26-6.html
Atkinson, R. J. (2010d). ARC announces a Tier Review Process. HERDSA News, 32(3). http://www.roger-atkinson.id.au/pubs/herdsa-news/32-3.html
Atkinson, R. (2010e). AJET Editorial 26(5). Idle Moment 40: Impact Factor revisited. http://www.ascilite.org.au/ajet/ajet26/editorial26-5.html
Bourke, S. (2010). Rankings still rankle. In Letters to the Editor, Higher Education, The Australian, 17 November. http://www.theaustralian.com.au/higher-education/letters/rankings-still-rankle/story-e6frgcox-1225954652506
Genoni, P., Haddow, G. & Dumbell, P. (2009). Assessing the impact of Australian journals in the social sciences and humanities. In Proceedings ALIA Information Online 2009, Sydney, 20-22 January. http://conferences.alia.org.au/online2009/docs/PresentationC16.pdf
Goodyear, P. (2010). Rankings rancour. In Letters to the Editor, Higher Education, The Australian, 24 November. http://www.theaustralian.com.au/higher-education/letters/rankings-rancour/story-e6frgcox-1225959639713
Gough, N. (2008). Educational research and the ERA. AARE News, No 63, July. http://www.aare.edu.au/news/newsplus/news63.pdf
Graham, L. J. (2008). Rank and file: Assessing research quality in Australia. Educational Philosophy and Theory, 40(7). [verified 6 Dec 2010] http://bulletin.edfac.usyd.edu.au/wp-content/uploads/2008/11/rank-and-file_era-2008_graham-doc.pdf
Hemmings, B. & Kay, R. (2010). Journal ratings and the publications of Australian academics. Issues in Educational Research, 20(3), 234-243. http://www.iier.org.au/iier20/hemmings.html
Lamp, J. (2010). ERA Current Rankings Access. http://lamp.infosys.deakin.edu.au/era/
Levey, G. B. (2010). Rankings rancour. In Letters to the Editor, Higher Education, The Australian, 24 November. http://www.theaustralian.com.au/higher-education/letters/rankings-rancour/story-e6frgcox-1225959639713
The Macquarie Dictionary (1997). 3rd ed. Sydney: The Macquarie Library.
McBeath, C. & Atkinson, R. (2007). Editorial 17(2). Issues In Educational Research, 17(2), v-vii. http://www.iier.org.au/iier17/editorial17-2.html
Rowbotham, J. (2010). Journal rankings rankle academics. Higher Education Section, The Australian, 10 November. http://www.theaustralian.com.au/higher-education/journal-rankings-rankle-academics/story-e6frgcjx-1225950277254
Smith, A. (2010). Matching input and output assessments. In Opinion & Analysis, Higher Education, The Australian, 10 November. http://www.theaustralian.com.au/higher-education/opinion-analysis/matching-input-and-output-assessments/story-e6frgcko-1225950268027
|Please cite as: Atkinson, R. & McBeath, C. (2010). Editorial 20(3): ARC to revise the ranked journal lists. Issues In Educational Research, 20(3), ii-iv. http://www.iier.org.au/iier20/editorial20-3.html|