Scenarios of London local authorities' engagement with evidence bases for education policies
Mariam Al Hallami and Chris Brown
Institute of Education, University of London
This paper examines the use of research and evidence in the formation of education policy within London local authorities. In particular it explores the policy processes in three local authorities, and observes the role of research and the interplay between research and policy within each. We begin the paper with a general overview of policy process literature, before analysing the policy formation stage in detail. We then explore the findings of in-depth semi-structured interviews held with cabinet members who were responsible for children's and education services at three different local authorities. The paper concludes by presenting three scenarios of London local authority (LA) engagement with evidence and illustrates how such engagement varies, both according to the ideological leanings of authorities and the relationship such authorities have with central government.
Whilst micro and macro levels are important, an over-focus on these two areas tends to come at the expense of the meso level, namely those policy actors who reside at the level of local government authorities and who act as a policy gateway, or as an interpretative buffer, between schools and state. Local authorities are also important policy actors in their own right: for example, the exploration of their policy processes in a number of studies has revealed that certain amounts of power still reside within them (Al Hallami, 2013; Trowler, 2003; Dale, 1989; Simon, 1988). Local authorities continue to play a major role in how polices are formed and rolled out (Gilbert, Husbands, Wigdortz & Francis, 2013) in despite of the potential risk of being marginalised by the autonomy that has been afforded to schools via new forms of governance, the freedoms now provided to Academies and Free Schools, the drastic expansion of the academy program (from 203 schools in 2010 to 2456 in 2012), and the influx of organisations seeking to operate clusters of Academies. Further explanations on the roles and structures of local authorities will be discussed in the 'Policy formation in local authorities' section.
This paper, which stems from a recent study (Al Hallami, 2013), attempts to shed light on the policy process in a number of local government authorities in England - specifically, how such entities develop policy and the role of evidence within this process.
We can see policies as representations, which are encoded in complex ways (via struggles, compromises, authoritative public interpretations and reinterpretation) and decoded in complex ways (via actor's interpretation and meanings in relation to their history, experiences, skills, resources and context) (Ball, 1994: 16).Trowler (2003) clarifies that policy as a process is dynamic, and that this dynamism is a result of conflict between those involved in the process. These conflicts arise in specific ways when deciding which issues or problems the policy addresses, and what its desired goals are. Much of the conflict is apparent at the national level between policy-makers and those who actually implement these policies. For example, Lipsky's (1980) "street-level bureaucracy" model argues that policy outcomes will always, in the end, depend on who actually implements them. According to Lipsky (1980), implementers tend to shape policy according to their understanding and socio-historical backgrounds, adding personal aspects to the delivery of policy on the ground. As such, it is clear that education policy is very much a complex multi-dimensional notion, and interpreting its implementation requires an understanding of the diverse influences it faces.
Figure 1: The public policy process (Jones, 1970)
At the same time it is also acknowledged that the rhetoric of evidence-based policy often differs from the reality of how policy is developed, and how research findings are utilised as part of the policy making process. For instance, Campbell et al. (2007) noted from their study of 42 policy-makers within government that the majority felt, as a process, policy making was more 'messy' than 'linear' and that evidence was just one factor to be taken into consideration. Thus, Campbell et al. surmised that, amongst the policy-makers they interviewed, few would be likely to propose that a literal approach to evidence-based policy making should actually be undertaken. This conclusion is compounded by the suggestion that, in the short term, policy-makers' 'use' of evidence is more likely to be 'conceptual' rather than 'instrumental' in nature: that is, evidence is more likely to lead to changes in individual policy-makers' overall levels of knowledge or understanding ('conceptual' use) than to changes in their actual behaviour or practice ('instrumental' use), unless a significant and overwhelmingly accepted weight of evidence, has built up over time (Weiss, 1982; Huberman, 1993; Gladwell, 2000; Landry, Lamari & Amara, 2003; Levin, 2008). As a consequence, the development of policy is unlikely to be either immediately related to the findings of a study that has been relayed to policy-makers, or based solely on the findings of just one study. As a result, Duncan (2005) suggested that the idea of 'evidence-inspired' policy might be seen as more appropriate than a strict interpretation of the term 'evidence-based'. A similar view was held also by Davies, Nutley and Smith (2000) and Sebba (2007) who, in the main, refer to 'evidence-informed' policy. For the purposes of this paper, whilst the terms 'evidence-based' and 'evidence-informed' may be used interchangeably, the underlying meaning behind them will be that proposed by Davies (2004):
An approach that helps people make well informed decisions about policies, programmes and projects by putting the best available evidence from research at the heart of policy development and implementation. (p.5)Given that evidence does not simply exist, but must also be created, we also, however, consider that the act of collecting and disseminating data for research purposes should be considered a key aspect of the process of 'putting' as per Davies' definition above.
The policy formation process in UK's local authorities is conducted by policy and scrutiny committees, council members and the cabinet. Figure 2 is the authors' summary of the decision making processes described in the City of Westminister's website (City of Westminister, 2014). The policy and scrutiny committees provide insight on major issues for policy formation by conducting research and making recommendations. The policy and scrutiny committees are considered the main informative avenue in the policy formation processes of local authorities. The council members and cabinet then typically agree upon and approve policy frameworks.
Figure 2: Public policy process at local authorities (City of Westminister, 2014)
As such, as more and more schools have become academised, so the role of local authorities in the policy process has been reduced to mostly involved in planning academies and co-sponsoring them (Ball, 2007); "significantly, local authorities are now involved in planning, and recently established academies must now follow core aspects of the national curriculum. However, academies still remain independent of local authorities and there is a more diverse range of academies in terms of sponsors" (Gunter, 2011, p.5). As Al Hallami (2013) noted, the Conservative Party has mainly supported the academies program, being the ideological leader behind its introduction. As a result, academisation has been more accepted by boroughs where the Conservative political party is the winning majority, compared with boroughs that are led by other political parties (this point is explored in further detail below).
|* Describe the current policy making process in your local authority in general?|
According to the word frequency query, the most frequently mentioned words and their synonyms used to describe current policy practices at each authority were 'work', 'group', and 'community'. Once these words were observed, the software allowed us to see these concepts in context, to make sense out of the observation, and, correspondingly, conclusions could be drawn from the analysis results. Each of the three terms set out above provided concepts that guided the following observations.
Firstly, in relation to the work local authorities do, it could be seen within the data that their perception regarding their main role is to: (i) bring stakeholders together through "involving different groups together", and "working towards a partnership approach" when it comes to education policy; and (ii) commission "collective decisions" by devising "strategies and policies" as a group. Hence, the authorities play a directive role within the local community that directs, monitors, and implements decisions related to schools and education.
Secondly, both the words 'group' and 'community' have appeared alongside one another as concepts that relate, and are also used frequently when it comes to describing current policy processes. Further analysis revealed that 'group' and 'community' referred to multiple stakeholders, including families, schools, parents, and teachers. These various groups function under the umbrella of the local authority and the multiple boroughs. They are an integral part of the local system that works together as part of their regular practice. As a result, we are able to use the data to define the role of local authorities as: 'a directive body that plans, monitors, and facilitates local education policy whilst also being an executive body that maintains a certain role in policy making locally'. Local authorities involve and work alongside multiple stakeholder groups when making policy related decisions, and act as a communal entity that forms policies that aim to enhance its performance nationally (Al Hallami, 2013).
Local authorities have much less freedom than they used to, but still we have a certain level of freedom to decide how we want to work with a school. (Camden representative, 2013)At all three boroughs investigated, it is clear that the academies have created a fragmented local school structure and a concommitant decrease in formal power at the meso level. As stated by the Camden representative, there is much less freedom in how the authority works with schools today. Currently, local authorities have no formal power over academies as they are located outside the local borough's jurisdiction. Consequently, concerns related to underperforming academies and the ability of local authorities to intervene emerged. With the shifts of power that academisation has created, little can be done to regain that power, even when needed. However, this change in power relations has also acted as a catalyst for authorities to develop close relationships with academies within their boroughs:
If a school that is an academy is failing, I am afraid, no, we can't actually intervene. It is the state's responsibility. (Kensington and Chelsea representative, 2013)
As a local authority, government is specific; you don't have the formal power as the maintained sector. (Hackney representative, 2013)
We want to retain the partnership between local education and what we call the family of schools. Instead of schools going on their own and competing against each other our idea is we will cooperate with each other to achieve the best for children. (Camden representative, 2013)The approach that all three authorities undertook in the face of their shifting powers towards academy schools centres upon a strong foundation of partnership. Retaining partnerships within local academies creates an informal relationship wherein local authorities are able to intervene and monitor schools indirectly. However, each of the boroughs explored has developed certain approaches with regard to how they partner with schools. For instance, both Camden and Kensington and Chelsea's approach is to retain partnerships between local education and schools through a communal relationship. Hackney, on the other hand, builds upon the community approach to include moral authority. This moral authority creates responsibility where everyone is held accountable for his or her actions.
We have very close relations with our school, we are not called local authorities because now we are communities. (Kensington and Chelsea representative, 2013)
We have elected representatives and the moral authority. Our approach is to have moral responsibility to make sure that academies are performing well enough and are serving well. (Hackney representative, 2013)
While authorities are adjusting to the shifting powers that the academies act has created, it is important to keep in mind that for all authorities student education is stated as being of the utmost importance. Yet, some authorities more than others are supportive of the witnessed change. For instance, a Kensington and Chelsea respondent stressed that the structure or type of school does not matter: "we want high standards [but] we don't care who delivers the high standards". In other words, the authority believed that academies have the potential to drive performance positively and, as such, are supportive of them. The converse was represented by Camden's comments which noted that academies were "not the way we are going in Camden". This is because Camden attributed the success of its schools to the communal relationship that the authority has with their schools: "we think it does show the way we work can achieve very good results". It would seem, therefore, that academies' influence on local authorities has introduced changes to the local policy process. Yet, each authority is creating approaches to deal with these changes and sustain good performance across their schools.
All schools require national league table for SAT and Key Stage. The borough does a lot of work on getting evidence of how children are performing at the different stages. (Camden representative, 2013)Some authorities more than others go beyond this statutory duty for data, and use research as a method to enhance performance. In Camden, for instance, the initial data collected from schools is extended to include as much information as possible on individual student groups' performance. This type of information is then used to compare and analyse performances to help the authority address the issue more accurately.
I think providing that data for schools is important. What the data shows are the groups that are not doing so well, and increasingly the group that is not doing well is not the ones we assumed will underperform. (Camden representative, 2013)However, in addition both Camden and Hackney promoted the use of research, and facilitated it amongst their schools. In Camden, the data collected as part of the authorities' statutory duties was distributed to each individual school for the purpose of encouraging schools to engage with it and compare performances. As a result, schools are becoming more engaged with data and indicators as a method to monitor improve their overall performance. Camden believed that they are in the best position to provide these data, and have specialised people who work with the information. In addition, above and beyond this, Hackney also promoted the use of research tools within their schools. Nonetheless, while these authorities were playing a further role in engaging with research, school engagements with their efforts are voluntary:
Another element of our process is recognising that schools are themselves independent institutions and all our schools are independent. We don't direct them to do anything in particular, so we see our role as kind off encouraging them to understand that these particular research practices we know to have high impact and relatively low cost. (Hackney representative, 2013)While research plays various roles within the local authorities explored, it may be seen as limited when it comes to its role in policy making. However, the authorities seen to encourage the use of research, and be further engaged with it, do use research in policy shaping. Camden, for example, believed that its use of research in the development of its policies was becoming better than it previously might have been. Hackney, on the other hand, believed that it is building a tradition where the authority is keeping a close eye on research to inform policy.
To further understand the role that research plays in informing policy within local boroughs, a word frequency query was undertaken to analyse and define this role according to the interview data. This approach can suggests terms with which the role we are exploring can be better explained.
|* What is the role of research and evidence in informing policy development?|
Another word frequency query was undertaken to allow us to develop further observations of connecting concepts to explain the role of research in informing policy development. Amongst the most frequent words used to express the role of research in informing policy development, words related to change occurred most frequently. Words such as 'implementation', 'inform', and 'improve' were all associated under the umbrella of change. This allows us to suggest that while research might inform policy, authorities that use extensive research connect it to notions of 'change', and so present it as a method that promotes positive changes in the local community, schools, and boroughs.
In particular, it should be noted that the case studies presented two contrasting scenarios that illustrated the influence of these ideologies on each authority's engagement with evidence. Two of the three authorities explored are led by the Labour Party, and one by the Conservative Party. The perspectives of the parties in each case act as the base of the ideological grounds that the policy-makers act upon. For both Camden and Hackney which are led by the Labour Party, engagement and advocacy of research and evidence within their boroughs and schools were prominent. The two scenarios that local authorities face when engaging with evidence are illustrated in Figure 3.
Figure 3: Scenarios of local authority engagement with evidence
We explain Figure 3 as follows. Our research has shown that the central government is involved in engaging with evidence at the national level through its statutory requirement of data collection that it imposes on local authorities. The statutory information collected by local authorities for the central government presents a form of interaction between authorities and evidence. However, local authorities also decide how to interact with the evidence (and related evidence-informed 'solutions') that central government presents and pushes down to them; which may or may not relate to their context and/or the evidence they have passed up to central government (e.g. see Moss, 2013 in relation to the national strategies). Hence, authorities have two options when dealing with evidence: accept all the evidence as it is, or accept partial evidence and engage in further research and data locally. The full acceptance of existing evidence that is supplied by the central government restricts the engagement of authorities with further evidence. These authorities use the evidence provided to them, and do not seek further research and investigation. On the other hand, authorities that partially accept central government evidence, are seen to engage thoroughly with research.
In the case of the three authorities explored, Camden and Hackney could be identified under the partial acceptance scenario of evidence. They engage with research and data on a local level, and promote evidence usage across their boroughs. However, the Kensington and Chelsea authority is identified under the full acceptance of evidence scenario. Its engagement with further evidence and research locally could be described as less intense compared to the other two boroughs. An explanation of that may be linked to the ideological beliefs of the central government that are similar to Kensington and Chelsea authority as both are led with the Conservative Party ideology. Hence, we suggest that authorities with similar ideologies to that of central government are more likely to fully accept evidence (or evidence-informed solutions) pushed down by the government, and engage in less local research. As such we argue that (at least in relation to the enactment of the academy act) the influence of ideology on the use of research in local policymaking is apparent.
The issue with frequent changes is that once a change or policy is implemented, it is hard for it to be reversed. If local authorities give up their role in local education, it will likely be a challenging prospect for them to again become formally involved. Currently, authorities are practising what they refer to as 'soft power' to make sure they are still involved in ensuring that schools are not failing their students. Informing education policy remains a matter of immense complexity as many ideologies, groups, people, and other stakeholders attempt to shape it. Yet, while the role of evidence and research in informing local policies may not be great, it presents a method by which authorities may seek to subvert (or not) the national policy agenda, whilst also seeking to positively influence school performance.
Al Hallami, M. (2013). Informing education policy: An exploratory multi-case study of policy practices at three local education authorities in London boroughs. Unpublished Masters dissertation, Institute of Education, University of London, United Kingdom.
Anderson, J. E. (1990). Public policymaking: An introduction. Boston: Houghton Mifflin.
Ball, S. (1994). Education reform: A critical and post-structural approach. Buckingham: Open University Press.
Ball, S. (2006). Education policy and social class. London: Routledge.
Ball, S. (2008). The education debate. Bristol: The Policy Press.
Ball, S. (2012). Global Education Inc: New policy networks and the neoliberal imaginary. London: Routledge.
Barber, M. & Mourshed, M. (2007). How the world's best performing school systems come out on top. London: McKinsey and Company.
BBC News (2000). 'City academies' to tackle school failure. BBC News, 15 September. http://news.bbc.co.uk/2/hi/uk_news/education/925378.stm
Briggs, A. R. J., Coleman, M. & Morrison, M. (Eds) (2012). Research methods in educational leadership and management. Los Angeles: SAGE.
Brown, C. (2011). Exploring the concepts of knowledge adoption and conceptual impact: Implications for educational research submissions to the Research Excellence Framework (2014). Education, Knowledge and Economy, 5(3), 137-154. http://dx.doi.org/10.1080/17496896.2012.673938
Brown, C. (2013). Making evidence matter: A new perspective for evidence-informed policy making in education. London: Bedford Way Papers.
Bullock, R., Mountford, J. & Stanley, R. (2001). Better policy making. London: Centre for Management and Policy Studies, Cabinet Office.
Burstein, P. (1991). Policy domains: Organization, culture, and policy outcomes. Annual Review of Sociology, 17, 327-350. http://www.jstor.org/stable/2083346
Campbell, S., Benita, S., Coates, E., Davies, P. & Penn, G. (2007). Analysis for policy: Evidence-based policy in practice. London, HM Treasury. http://www.civilservice.gov.uk/wp-content/uploads/2011/09/Analysis-for-Policy-report_tcm6-4148.pdf
City of Westminister (2014). Policy and scrutiny. https://www.westminster.gov.uk/policy-and-scrutiny. The Leader and the Cabinet. https://www.westminster.gov.uk/cabinet
Dale, R. (1989). The state and education policy. Milton Keynes: Open University.
Davies, H., Nutley, S. & Smith, P. (2000). What works? Evidence-based policy and practice in public services. Bristol: The Policy Press.
Davies, P. (2004). Is evidence-based government possible? Jerry Lee Lecture, 4th Annual Campbell Collaboration Colloquium, Washington DC, 19 February 2004.
Department for Children Schools and Families (currently renamed Department of Education) (2009). Analysis and Evidence Strategy 09-10. London: DCSF.
Department for Children Schools and Families (2010). Science and analysis review of the Department for Children, Schools & Families. London, DCSF. http://webarchive.nationalarchives.gov.uk/20121212135622/
Duncan, S. (2005). Towards evidence-inspired policy making. Social Sciences, 61, 10-11.
Dye, T. (1992). Understanding public policy. 7th ed. New Jersey: Prentice-Hall.
Gilbert, C., Husbands, C., Wigdortz, B. & Francis, B. (2013). Unleashing greatness: Getting the best from an academised system. London: Pearson.
Gladwell, M. (2000). The tipping point: How little things can make a big difference. London: Little Brown.
Gough, D. (2004). Systematic research synthesis. In G. Thomas & R. Pring (Eds), Evidence-based practice in education. Buckingham: Open University Press.
Gunter, H. (2011). The state and education policy: The academies programme. London: Continuum.
Haddad, W. & Demsky, T. (1995). Education policy-planning process: An applied framework. Paris, UNESCO. http://www.unesco.org/education/information/pdf/11_200.pdf
Hannaway, J. & Mittleman, J. (2011). Education politics in an era of evidence. In D. Mitchell, R. Crowson & D. Shipps (Eds), Shaping education policy. New York: Routledge.
Hargreaves, D. (2010). Creating a self-improving school system. National College for Leadership of Schools and Chidren's Services. http://dera.ioe.ac.uk/2093/1/download%3Fid%3D133672%26filename%3Dcreating-a-self-improving-school-system.pdf
Hargreaves, D. (2012). A self-improving school system in international context. National College for School Leadership. http://www.southendeducationtrust.co.uk/media/
Haynes, L., Service, O., Goldacre, B. & Torgerson, D. (2012). Test, learn adapt: Developing public policy with randomised controlled trials. London: Cabinet Office. https://www.gov.uk/
Hill, M. (2003). Understanding social policy. Oxford: Basil Blackwell.
Huberman, M. (1993). Linking the practitioner and researcher communities for school improvement. School Effectiveness and School Improvement, 4(1), 1-16. http://dx.doi.org/10.1080/0924345930040101
Jones, C. (1970). An introduction to the study of public policy. California: Duxbury.
Landry, R., Lamari, M. & Amara, N. (2003). The extent and determinants of utilization of university research in government agencies. Public Administration Review, 63(2), 192-205. http://dx.doi.org/10.1111/1540-6210.00279
Levin, B. (2008). Thinking about knowledge mobilization. Paper prepared for an invitational symposium sponsored by the Canadian Council on Learning and the Social Sciences and Humanities Research Council of Canada, 15-18 May, 2008. http://www.ccl-cca.ca/pdfs/OtherReports/LevinDiscussionPaperEN.pdf
Lipsky M. (1980). Street-level bureaucracy: The dilemmas of the individual in public services. New York: Russell Sage Foundation.
Luke, A. & Hogan, D. (2006). Redesigning what counts as evidence in educational policy: The Singapore model. London: World Yearbook of Education.
Moss, G. (2013). Research, policy and knowledge flows in education: What counts in knowledge mobilisation. Contemporary Social Science, 8(3), 237-248. http://dx.doi.org/10.1080/21582041.2013.767466
Nutley, S. M., Davies, H. T. O. & Walter, I. (2002). Conceptual synthesis 1: Learning from the diffusion of innovations. Edinburgh: University of St Andrews. https://www.kcl.ac.uk/
OECD/CERI (2007). Evidence in education: Linking research and policy. Paris: OECD.
Palumbo, D. J. (1988). Public policy in America. New York: Harcourt Brace Jovanovich.
QSR International (2013). Find and analyze text using queries. http://help-nv9-en.qsrinternational.com/
Rizvi, F. & Lingard, B. (2010). Globalizing education policy. London: Routledge.
Rosenbaum, D. P. (1991). The pursuit of "justice" in the United States: A policy lesson in the war on crime and drugs? Canadian Police College Journal, 15(4), 239-256.
Sebba, J. (2007). Enhancing impact on policy making through increasing user engagement in research. In L. Saunders (Ed), Educational research and policy making. London: Routledge.
Simon, B. (1988). Bending the rules: The Baker 'reform' of education. London: Lawrence and Wishart.
Stake, R. E. (1995). The art of case study research. Thousand Oaks, CA: SAGE.
Taylor, S., Rizvi, F., Lingard, B. & Henry, M. (1997). Educational policy and the politics of change. London: Routledge.
Trowler, P. (2003). Education policy. London: Routledge.
Weimer, D. & Vining, A. (2004). Policy analysis: Concepts and practice. 4th ed. New Jersey: Prentice-Hall.
Weiss, C. H. (1982). Research in the context of diffuse decision-making. The Journal of Higher Education, 53(6), 619-639. http://www.jstor.org/stable/1981522
Yin, R. K. (1994). Case study: Design and methods. Newbury Park, CA: SAGE.
|Authors: Mariam Al Hallami is a doctoral student at the Institute of Education currently researching on evidence-based policymaking in education. She also is the Manager of Research and Evaluation at a philanthropic organisation in the United Arab Emirates. Al Hallami also holds a Master's degree in Education Leadership from the Institute of Education, University of London.|
Chris Brown is a John Adams Research Fellow at the Institute of Education, University of London. Currently working within the London Centre for Leadership in Learning, Chris leads various projects which seek to help practitioners identify and scale up best practice and and has also worked with a number of international governments to examine how research can better impact on policy-making. Formerly a civil servant, Chris has held roles in government research and in policy-making. His most recent positions include Head of Research for the Training and Development Agency for Schools, and Senior Policy Advisor for the Ministry of Justice.
Chris completed his DPhil at the University of Sussex in 2011. With a long standing interest in how evidence can aid decision-making and the development of policy, Chris has written several papers on the subject and has presented on it at a number of international conferences both in Europe and North America.