IIER logo 4
Issues in Educational Research, 2015, Vol 25(4), 442-459
[ Contents Vol 25 ] [ IIER Home ]

Teachers' evaluation of professional development in support of national reforms

Tuba Gökmenoğlu
European University of Lefke, Cyprus

Christopher M. Clark
Arizona State University, USA

As in many other nations, the Turkish education system has undergone many significant curricular and structural reforms in the last decade. This study was designed to learn from teachers about the quality of professional development programs that were designed to support national reforms. Ten years into a period of intensive national reform, teachers reported that professional development activities only moderately satisfied them. We speculate that 'reform fatigue' may be partially responsible for relatively low teacher enthusiasm for mandatory, centrally designed training. Failure to adjust teacher development designs to meet established teacher preferences could undermine ambitious and expensive programs of national education reform.


Introduction

The effectiveness of centrally mandated reform initiatives have long been criticised by teacher educators and classroom teachers. Extensive research on school effectiveness and on improving instructional practices, with special attention to teacher professional development, has taught us a great deal about the possibilities and limitations of this centralised approach to retooling teachers to meet the demands of an innovation (McConnell, Delate & Newlon, 2011; Shah, Sultana, Hassain & Ali, 2011; Smith, 2005). The research literature confirms that professional development programs not only help teachers in enhancing and deepening their knowledge and skills, but also in creating a frameshift in their minds, which generally have a resistance to educational change (Taylor, Yates, Meyer & Kinsella, 2011). Researchers conclude that teachers who took in-service training implemented more effective instructional practices, and gave more importance to teaching practices (Guskey, 1985; de Jager, Reezigt & Creemers, 2002; Kealey, Peterson, Gaul & Dinh, 2000). The literature of educational change has firmly established that significant reform movements on a national level will either succeed or fail depending on the quality, quantity and timing of the professional development support provided to teachers (Guskey, 2003; Newmann, King & Youngs, 2000).

Despite the importance of individual teachers in reform processes, teachers rarely take part in helping to design and frame the educational change process (Torres, 1996). Mostly, educational reforms are put into action and then teachers are informed via short seminars or in-service training courses (Avalos, 2000; Van Driel, Beijaard & Verloop, 2001). Certainly, if curriculum reforms are to be achieved with the help of teachers, teacher education needs to be reviewed and renewed in the light of reform initiatives, as teachers and local school leaders are the keys to successful education reform (Avalos, 2000; Minor, Onwuegbuzie, Witcher & James, 2002). Similarly, Guskey (2002: 381) highlighted the importance of professional development, stating that "High-quality professional development is a central component in nearly every modern proposal for improving education". Since professional development programs are the key elements in the success of any educational reform, the quality of learning opportunities for school staff is a key topic in both policy debates and educational research (Wilson & Berne, 1999). In sum, since individual school teachers are the ultimate implementers of school reforms, they must be both well prepared and continually supported if the reforms are to be implemented in a meaningful way.

A reform may look wonderful on paper or in a national plan, but its eventual success depends on faithful and creative interpretation and successful adaptation to constraints and opportunities present in each school and classroom.

The situation in Turkish education

The past two decades have been turbulent for primary school education in Turkey (Grossman, 2013). Beginning in 1997, Turkish primary and secondary education has undergone several massive, nationally mandated and centrally administered reforms. In 1997, the primary school system was changed from a 5-year compulsory education system to an 8-year compulsory education system. This dramatic reform, along with a campaign to eliminate dropouts and encourage girls to get at least a primary school education, was intended to create a new Turkish generation with better skills, qualified to fill 21st Century jobs.

In 2003, the national mathematics and science curricula were replaced with completely new maths and science curricula based on a constructivist theory of learning, in response to disappointing PISA (Programme for International Student Assessment) and TIMSS (Trends in International Mathematics and Science Study) exam results (Gur, Celik & Ozoglu, 2012; Koc, Isiksal & Bulut, 2007). In response to suggestions and requirements of the World Bank and the European Union, and to low results in these international science and mathematics examinations, serious modifications have been made to education in Turkey. The educational paradigm has shifted toward constructivism (in contrast to rote learning), influenced by the strong winds of liberty, individualism, and postmodernism movements (Aksit, 2007; Incikabi, 2012). The role of the Turkish teacher has shifted from being director and lecturer towards serving as learning guide and coach. Pupils play more active roles in classroom activities. Teachers now seek to situate students' academic experiences in authentic contexts and promote their critical thinking. Use of technology and concrete curriculum materials, and highlighting the needs of learners in instructional planning have emerged as new trends in Turkish primary school education, as they have in many other parts of the world.

In 2012, the structure of K-12 schooling was again changed, from an 8-year compulsory education system to a 4+4+4 system. As well, massive infusions of classroom technology have been introduced nationally, with 12,800 primary and secondary school students using tablet computers in pilot schools that also have smart boards installed in each classroom. Following a nationwide pilot study, the Turkish Ministry of National Education (MNE) plans to distribute and install 570,000 smart boards and provide one tablet computer to every student (12 million total) in Turkey (MNE, 2013).

To ensure the adaptation and adoption of the reformed curricula and methods by teachers, the number of professional development programs offered by MNE increased significantly. While there were 450 in-service training activities with 17,808 participants offered in 1996, this number was increased to 21,128 activities with 479,436 participants in 2011 (MNE, 2013). However, the quality of the professional development programs is crucial as well as the scale. There are few studies of the effectiveness of in-service training programs in Turkish literature. Most of this research uses case study design. Until now, much of the research on the effectiveness and appropriateness of in-service training programs in Turkey has concentrated on particular courses or seminars. There are no published studies that focus on characteristics and qualities of effective teacher in-service training programs in Turkey during a time of dramatic national reform in a comprehensive way (e.g. Catmali, 2006; Oztaskin, 2010; Yalin, 2004).

The SWOT analysis (strengths, weaknesses, opportunities and threats) conducted by the MNE In-Service Training Department confirmed that the weaknesses of the department include: inadequacy of needs assessment; lack of follow-up and assessment studies; not presenting motivating features for teachers and directors; lack of short, medium and long-term planning due to the lack of communication among ministry departments; and lack of effective training implementation because of the density of target populations and insufficiency of resources. Additionally, the threats to the department's ability to achieve its mission include size of target population, limitation of economic and human resources, lack of coordination among institutions, the perception of in-service training programs as a 'holiday' by target populations, geographical conditions and difficulty of transportation to training centres (MNE, 2011). Thus, there is plenty of room for improvement in the design and delivery of centrally mandated professional development activities in Turkey.

Effectiveness of professional development programs

According to the professional development literature, the characteristics of effective in-service training programs include: engaging collective participation; involving participants from the same department and grade; compliance with standards; meaningful assessment component; continuing with improvement efforts; cooperation between post-secondary institutions and school districts; intensive; sustained and job-embedded; and content focus on teachers' knowledge of subject matter (Borko, 2004; Cohen & Hill, 1998; Desimone, Porter, Garet, Yoon & Birman, 2002; Desimone, 2009; Eylon & Bagno, 1997; Yoon, Duncan, Lee, Scarloss & Shaply, 2007; Varela, 2012). In addition, it is generally known that highly effective in-service training programs are the ones that are grounded in teachers' needs (Avalos, 2011; Darling-Hammond & McLaughlin, 1995; OECD, 2010). Promoting the professional autonomy of teachers while designing a professional development model is one of the most important characteristics of successful training programs. Ball (1996) and Clark (1992) concluded that teacher determination of the shape and the course of their own professional development is essential in the design of any training model. Furthermore, the American Society for Training and Development endorsed a list of essential qualities for effective training and development, including integrating appropriate technology into programs; encouraging connectivity and collaboration; presenting content not as a discrete event – should be an event exemplified by real classroom cases; focusing on learning styles and preferences; and meeting teachers' training needs (Arneson, Rothwell & Naughton, 2013).

Any professional development program aiming to improve teacher and student learning and performance requires an evaluation process as an integral part (Delvaux et al., 2013; Linn, Gill, Sherman, Vaughn, & Mixon, 2010). Owing to its importance and necessity, there are a significant number of studies in the literature that focus on the evaluation of in-service teacher training programs. Some studies documented strong critiques of typical professional development programs, but the most widely cited result was the need to enhance teachers' content knowledge and pedagogical knowledge (Guskey, 2003). Wade's (1985) meta-analysis that reviewed 91 journal articles concluded that professional development programs were moderately effective in terms of effect levels, reaction, learning, behaviour, and results. By contrast, Kealey, Peterson, Gaul and Dinh (2000) concluded that implementation failure was still a common problem. In sum, the consensus of the literature is that in-service training programs often fail to achieve their objectives and fail to be delivered as planned (Guskey, 1986; Fullan, 1991).

The literature on the effectiveness and importance of teacher professional development programs documents two strong but contradictory claims. On the one hand, teachers surveyed find much to criticise about typical in-service programs. On the other hand, strong, well-designed and well-implemented programs of professional development are rated positively by participating teachers. To illustrate, it was found by Grieve and McGinley (2010) that successful completion of a continuing professional development program in Scotland led teachers to rate the program favourably in terms of improving learning, integrating theory into practice, and increasing pedagogical skills. Birman, Desimone, Porter and Garet (2000) surveyed more than 1000 teachers who participated in a US Federal Government sponsored in-service training program. Results indicated that in-service training activities, including study groups, teacher networks, research projects, and teacher resource centres, were more effective as they were more compact, consistent and they encouraged active participation. In addition, collective participation is regarded as an essential feature of an effective training program by teachers; participation of teachers from the same department, subject or grade level was more likely to be evaluated positively because these features foster active involvement and interconnection among teachers (Birman et al., 2000). In this context, in-service training courses focusing on a specific subject area rather than generic content were rated as effective by teachers (Birman et al., 2000; Borko, 2004; Cohen & Hill, 1998; Desimone, 2011). Similarly, professional development activities that were consistent with established policies and teachers' professional experiences were found to be more effective and memorable (Birman et al., 2000).

However, Barnett (2002) reported that inadequacy of follow-up programs after in-service training courses was one of the most crucial problems of professional development programs, and suggested follow-up activities for one- or two-day seminars, which are not sufficient to improve teacher knowledge and skills by themselves. Furthermore, a study of 207 teachers in 30 schools, in 10 districts in 5 states in the USA indicated that professional development elements such as those involving activities that were aligned with standards, assessing teacher outcomes, continuing improvement efforts, and cooperation between post-secondary institutions and school districts, were strongly associated with higher quality by teachers (Desimone et al., 2003). In the Yoon et al. (2007) review, in-service training programs that were evaluated as positive had these defining qualities: they were perceived as intensive, sustained, job-embedded, and focused on the teachers' subject matter knowledge. Similarly, Eylon and Bagno (1997) concluded that to be effective, the duration of the in-service training should be sufficient to acquire mastery of the innovation. From another perspective, Joyce and Showers (1980) pointed out that "to be most effective, training should include theory, demonstration, practice, feedback and classroom application" (p. 379). An action research study conducted to document the use of professional development knowledge in classroom settings showed that there was limited evidence of teachers' use of the ideas acquired in in-service training programs in their classroom settings (Linn et al., 2010). Sparks (2002) asserted that no one could promote school effectiveness without focused and un-fragmented professional development efforts. Furthermore, Day (1997) claimed that there were no systematic, coordinated, and conceptualised training programs in many countries of the European Union (EU). Additionally, he criticised the programs as top-down and short-course dominated. In 2007 the European Union (EU, 2007) also reported that only 11 states offered systematic in-service training programs, and most of these had problems in terms of coherence and continuity.

Purpose

Synthesising ideas from the literature on professional development of teachers, the influence of professional development on teacher change and reform success has been well documented (Johnson & Fargo, 2010; Guskey, 2003; Opfer & Pedder, 2011). It is possible to conclude that teacher satisfaction and student learning increase, hence, the possibility of success for reforms increased if professional development activities are close to home, meet needs and expectations, and contribute to teachers' curricular understanding and self-efficacy (Avalos, 2011). Parallel with this, it was found that intensive, inquiry-based in-service training programs had positive effects on teacher attitudes towards reform, their skill at adapting to reform, and their understanding and use of reform-based practices (Supovitz, Mayer & Kahle, 2000).

In the light of the findings gleaned from the international and the Turkish literature, the present study aimed to document the thoughts and opinions of Turkish primary teachers on the broad range of in-service training programs that they have attended compulsorily. The analysis of existing conditions and learner judgments was regarded as an essential step in improving and aligning with the design of Turkish teacher professional development programs, which could play a crucial role in the improvement of conditions, adapting the educational reforms and reform effectiveness using limited resources. In addition, by documenting the views of teachers about in-service training programs, this study aimed to reach more general conclusions about in-service training programs in Turkey in a time of dramatic educational reform. As Avalos (2011) stated, despite the fact that there are different cases and country specifications, teacher development processes are similar across different national contexts. This study is expected to contribute to the international literature of professional development, with special relevance for nations implementing comprehensive reforms of large primary education systems. Two research questions guided the study:

  1. How do primary teachers in Turkey rate the appropriateness of nationally mandated in-service program content, instructors, organisation, training centres, participants, and assessment methods?
  2. How do primary teachers' reports of nationally mandated in-service training program appropriateness and effectiveness vary by gender, subject area, and teaching experience?

Method

The study reported here is based on a national survey of 1,730 Turkish teachers conducted in Spring 2012. The primary aim of the study was to learn about teachers' judgments of the quality and sufficiency of the in-service professional development programs that they have participated in during a time of dramatic system-wide reforms. To achieve the purpose of the research, a survey method was adopted.

Sample

The target population comprised K-8 teachers who were Classroom teachers (primary grades), or Mathematics, Science and Technology, Social Studies, Turkish, or English teachers in public schools in Turkey. The sample was selected by using cluster random sampling. A sampling procedure was performed within this target population in two steps: There are 81 cities under 26 statistical regions in Turkey which are defined according to the SR criteria of the European Union regional classification, in order to make the socio-economic analyses of regions (MNE, 2011). One city from each statistical region was selected randomly to represent the region's overall population. The number of participant schools was determined by dividing the total number of schools by 40 in each city (40 represents the lowest number of schools, from Tunceli city). Thus, 352 primary schools from 26 cities were selected through simple random sampling. Six volunteer teachers from each randomly selected school were invited to participate, comprising a sample of 2,112 teachers in total. Ultimately 1,730 teachers participated in the study with a return rate of 81.9%. The characteristics of participants are presented in Table 1.

Table 1: Demographic characteristics of teachers
(categories do not add to 100% owing to some incomplete responses)


N%
GenderFemale90752.4
Male75843.8
Teaching
experience
0-5 years72441.8
6-1045526.3
11-1526315.2
16 and more25915.0
BranchClassroom Teacher (Primary)45926.5
Math24814.3
Science and Technology24714.3
Turkish26015.0
English24414.1
Social Sciences24214.0

Data collection instrument

Data were collected through a scale entitled Opinions on In-Service Training Programs, which was developed by the researchers as a 5-point Likert style scale ranging from 'not valid for any in-service training programs' to 'valid for all in-service training programs.' The total number of the items is 50. The following steps describe the development of the survey.

First, previously conducted studies on in-service teacher training and other related resources were reviewed, including MNE Training Courses 2011 Catalogue (MNE, 2011), Teachers' General Competencies Booklet of MNE (MNE, 2011), and teacher education courses defined by the Higher Education Council (2011). At the same time, pilot interviews with ten K-8 teachers were conducted to determine basic themes and an item pool of the questionnaire. Second, items were categorised, and headings were specified based on previous studies and preliminary interviews. Two sections were determined: demographic information, and opinions on in-service training programs. To ensure the face and content validity, expert opinions were taken from seven academicians from Curriculum and Instruction, Elementary Mathematics and Science, Educational Leadership and Administration, and Educational Evaluation and Measurement departments, and seven teachers from classroom Mathematics, Science and Technology, Social Studies, and English areas, two experts from the In-service Teacher Training Department of MNE Board of Education, and two district directors of the MNE. After getting expert opinions on the test items and the general appearance of the instrument, 50 items were determined in the opinions on the in-service training programs section. Last, after obtaining necessary permissions from Middle East Technical University Human Subjects Ethics Committee, and MNE, the questionnaire was piloted with 460 primary school teachers.

An exploratory factor analysis was performed on pilot test data from 460 volunteer teacher respondents. The factor analysis yielded six factors that accounted for 79% of the variance in opinions on in-service trainings programs. The six factors were labelled: instructors, training centres, assessment of training, contents of training, participants and organisation. Including all items that had item loadings greater than .30 yielded 42 items. Each factor was analysed separately to ensure the reliability of the scores. The Cronbach's alpha coefficients ranged between .90 and .98. The Cronbach's alphas for the subscales of the instrument have been found to be exceptionally high. Since this is not a routinely expected result for an initial development of a measurement tool, the rationale behind such high alphas needs to be elaborated. At first we suspected a consistent answering bias/tendency. However, when reversed items were considered, we concluded that there was no way for participants to avoid dealing with the inconsistency of reversed items. Reversed items increase the inter-item reliability of the scales (Nunnally & Bernstein, 1994).

Data collection and analysis

Following the pilot study, the project proposal was submitted to the MNE Educational Research and Development Department (ERDD) to get support for the study. ERDD agreed to sponsor by mailing questionnaires in sealed envelopes to each participant in the sample. Gathered data were screened for missing values and for incorrect data entry. Second, to provide construct validation evidence for the scale, a factor analysis was conducted, resulting in six factors with item loadings greater than .30, which account for 67% of the variance. The Cronbach's alpha coefficients of factors ranged between .70 and .97.

Teachers' reports of their opinions about in-service training programs are presented as rating means and standard deviations. The range of the scale (which was 4) was divided by the number of scale points (which was 5), and the range of each unit in the scale was .80. Therefore, the 5-point scale was interpreted so that a rating from 1-1.80 indicates 'not valid for programs', 1.81-2.60 indicates 'valid for few training programs', 2.61-3.40 indicates 'valid for half of training programs', 3.41-4.20 indicates 'valid for most of training programs', and 4.21- 5.00 indicates 'valid for all training programs'.

To document the effect of gender, teaching experience and subject area on a teacher's ratings of the appropriateness of in-service training programs, a 2x4x6 Factorial MANOVA was employed. To prevent the excessive inflation of Type I and Type II error rates, multivariate analysis was preferred to univariate analysis (Haase & Ellis, 1987). The present study involves seven dependent variables namely participants (who participated in), content (what was taught), instructors (who taught), organisation, training centres, and assessment of in-service training programs, and three independent variables which are gender (female, male), teaching experience (0-5 years, 6-10 years, 11-15 years, 16 and more years), and subject area of teachers (Classroom, Mathematics, Science and Technology, Social Studies, Turkish, English).

Table 2: Opinions of teachers on in-service training programs



MSD
ParticipantsThey were highly motivated.3.241.01
They were active during the training.3.011.05
The participants taught common subject matters.2.661.08
ContentContent was understandable for me.3.56.99
Content was up to date.3.241.06
Contributed to my professional performance.3.111.05
Demand was within acceptable levels.3.11.96
Met participants' theoretical needs.3.091.05
Exemplified by real classroom situations.3.001.06
Increased teachers' interest in and attention to the subject.2.931.07
Content provided personal benefits besides teaching.2.831.14
Content met teachers' needs for application.2.731.03
InstructorsPresented content clearly.3.181.03
Gave satisfactory answers to content related questions.3.111.04
Used time effectively.3.081.05
Incorporated participants' pre-existing knowledge.3.051.04
Used appropriate materials.3.041.07
Set an appropriate pace.3.031.02
Encouraged participants' evaluation.2.991.16
Employed appropriate teaching methods.2.941.02
Had effective classroom management skills.2.921.06
Encouraged active participation.2.911.04
Increased participants' learning interest.2.821.05
Made connections between subject area and course content.2.821.04
OrganisationAnnouncements of program participation were done on time.3.421.15
The places of the courses were appropriate.3.251.14
Announcements done throughout programs were appropriate.3.121.06
The dates of the courses were appropriate.3.051.16
The information given before the courses was sufficient.2.911.12
Some necessities were provided like notebook, pen, computer etc.2.491.24
Training
centres
Halls were appropriate for learning in terms of lighting.3.181.12
Halls were appropriate for learning in terms of width.3.171.13
Coordinators' approach to problems was appropriate.3.081.02
Coordinators provided effective execution of training.3.041.02
Halls were appropriate in terms of technical equipment.2.911.08
Halls were appropriate for learning in terms of temperature.2.821.19
Food and refreshments offered during the courses were enough.1.911.13
AssessmentAssessment was fair.3.341.15
In the assessment, questions were clear.3.171.07
In the assessment, questions covered whole content.3.131.06
Learning was assessed at the end of the course.3.121.16
More than one assessment method was used.2.741.11

Results

To answer the research question "How do teachers rate the appropriateness and effectiveness of in-service program content, instructors, organisation, training centres, participants, and assessment methods?", teachers' ratings of their training experiences are presented in terms of means and standard deviations. Participating teachers rated all factors as 'valid for half of the training programs'. The means of each factor are: participants (M=2.97), content (M=3.07), instructors (M=2.99), organisation (M=3.04), training centres (M=2.87), and evaluation (M=3.10). Teachers' opinions on previous training programs are presented in Table 2.

According to the teachers, for more than half of the training programs content was understandable for teachers (M=3.56, SD=.99) and participation announcements were done on time (M=3.42, SD=1.15), but in only few training programs, some necessities like notebook, pen, computer were provided (M=2.49, SD=1.24), and food and refreshments offered during the courses were sufficient (M=1.91, SD=1.13). Teachers rate all other items as 'valid for half of the training programs'.

To explore differences in the opinions of teachers with respect to gender, subject area, and teaching experience a MANOVA was employed. The MANOVA analysis results in significant multivariate main effects for gender; Pillai's trace=.02, F(6, 1481)=3.92, p<.05, and teaching experience Pillai's trace=.04, F(18, 4449)=3.30, p<.05, but no significant multivariate main effect for subject area, Pillai's trace=.03, F(30, 5926)=1.40, p=.07, and the interaction effects of gender and subject area, Pillai's trace=.02, F(30, 7425)=1.17, p=.24; gender and experience Pillai's trace=.02, F(18, 4449)=1.47, p=.09; subject area and experience Pillai's trace=.04, F(90, 8916)=.73, p=.97; and gender, subject area and experience Pillai's trace=.06, F(90, 8916)=1.01, p=.45. Owing to significant multivariate test results, a further examination of univariate test results for each dependent variable was made. Although there are some significant effects of gender and teaching experience, they do not rise to the level of practical significance as the effect sizes are small.

Discussion

A prominent feature of the education discourse relating to educational change and reforms has been the call for heightened attention to professional development of teachers. As Fullan (1991: 315) indicated, “Continuous development of all teachers is the cornerstone for meaning, improvement, and reform", scholars agree on the place of in-service training programs and supporting teacher knowledge and skills, and developing the capacity in undertakings of large-scale educational reforms (Fullan & Hargreaves, 1992; Guskey, 2003; Newmann et al., 2000). Professional development supports reforms through two strategies: strategies for instructional change and organisational change (Fullan, 1991). How well professional development programs activate new strategies, how much they are responsive to teachers' needs, and how much they build on and extend teachers' existing knowledge and skills are major focuses of nation-wide reform initiatives.

The Turkish Government's endorsement of educational reforms as a vehicle for curricular and pedagogical change to achieve desired student performance in international examinations is generally consistent with international patterns. However, the findings of this study also indicate only moderate satisfaction by teachers. In the literature, the inclusion of participants from the same subject area and holding similar positions are identified as effective professional development program characteristics (Birman et al., 2000). For half of the Turkish training programs, participants are highly motivated, and they are from the same subject area. The literature proposes that effective professional development programs have some common features: practicality associated with their classroom activities (Fullan & Miles, 1992), having follow-up tests after program completion (Ball, 1996), focusing on teacher needs, involving real life problems (Vukelich & Wrenn, 1999), promoting active participation (Desimone et al., 2002), integrating feedback mechanisms (Abdal-Haqq, 1996; Varela, 2012). According to the respondents, half of the training programs' contents were rated as up-to-date, contributed in a professional sense, were at an acceptable challenge level, met the Turkish teachers' theoretical needs, were presented in real classroom situations, increased teachers' interest and attention to the subject, provided teachers with personal benefits besides teaching, and met teachers' practice needs. In general, the results show that Turkish participants mostly rate positive program descriptors as 'valid for half of in-service training programs'. That is, for at most half of the rated training programs, teachers expressed positive opinions about their fellow participants, content, instructors, organisation, training centres and evaluation of in-service training programs. A fifty-percent failure rate of expensive training programs is not a satisfying performance standard.

Turkish teachers' opinions about in-service training did not differ with respect to particular demographic characteristics. Results show that teaching experience has a statistically significant effect on teachers' opinions about participants, content, instructors, organisation, training centres, and evaluation of previous training programs. However, the effect size of teaching experience on teacher judgments is very small. Similarly, gender has a statistically significant effect only on the organisation and evaluation of training programs, and also a small effect. Subject area did not have a statistically significant effect on teachers' judgments. In the light of these findings, it is possible to conclude that there is a consensus of opinion of Turkish teachers about the evaluation of in-service training programs which does not differ by years of teaching experience, gender or subject matter specialisation.

The present study contributes to our understanding of in-service teacher training in several ways. First, in the literature, much of the research on in-service teacher education focused on the relationship between features of professional development programs and their outcomes for teachers who participated voluntarily - volunteers who were highly motivated to learn or to change (Supovitz & Zeif, 2000). Thus, the outcomes of studies of in-service training programs that are compulsory for teachers were unclear (Bobrowsky, Marx & Fishman, 2001). In Turkey, teachers generally attend compulsorily and 68.7% did not participate willingly in professional development programs designed by MNE (Ozer, 2001; 2004). Thus, this study makes a case for getting information about in-service training programs from teachers who did not attend programs voluntarily.

It was important to further explore what supports need to be in place to sustain national reform efforts. As another contribution to the literature, this study aimed to shed light on the reported efficiency of in-service training programs. While the Turkish context is distinctive, the broader claim that engaging classroom teachers respectfully and significantly in planning and implementing major educational reforms stands as a universally helpful reminder, even a cautionary note, since without high quality in-service teacher education, it is difficult to attain success with educational reforms and to overcome educational deficits (Guskey, 2003; Opfer & Pedder, 2011). The present study presents a menu of ways in which large scale, mandatory, reform-driven in-service training programs can be made more effective (e.g., grouping teachers by subject matter taught, don't oppress the teachers by requiring in-service training during school vacation time). These findings help central authorities to understand from participants' ratings what teachers need or prefer before, during, and after in-service training programs. Therefore, knowledge about teachers' experiences gained from this study can provide useful guidance to others designing effective and appropriate in-service teacher education program models that are intended to support large-scale national reforms.

Conclusion

To sum up, several waves of national education reform have transpired over the past 10-20 years, and each one of these reform initiatives changed teachers' classroom roles and practices. Despite all these efforts and changes, the findings of this study indicated that the quality of in-service training programs is well below the desired standard. The Turkish government may have a well-planned set of educational reforms on paper. However, without considering the needs of teachers, and placing teacher concerns at the centre, the reform efforts will remain incompletely implemented. The findings suggest that the outcomes of massive national educational reforms will be limited or even undermined to the extent that teacher judgments about the quality of in-service training are not taken into account.

"... across the world, educational reform is itself a huge priority" Hargreaves (2000). As a result of several research projects, educational researchers have learned more about processes of system-wide changes, effective implementation strategies, and necessary actions to be taken and investments to be made to reach success. Scholars acknowledge that success of large-scale reforms depends heavily on the quality of teacher professional development and of teachers' receptivity to in-service training program designs. It is also known that exploration of the concerns of primary school teachers involved in implementing innovations is essential for designing and evaluating professional development programs. In short, although in-service training is seen as a major mechanism to support reforms, and despite the fact that a significant amount of time, money, and effort are devoted to in-service training and education in Turkey, only the half of the training programs were rated by teachers as adequate. This should trigger a serious strategic reassessment by the Turkish MNE and by other central education authorities in nations implementing large scale reforms of primary education. Given the large public investment in professional development and educational reforms, there is much to gain from taking teachers' judgments seriously.

References

Abdal-Haqq, I. (1996). Making time for teacher professional development. (ERIC Digests). Washington, DC: ERIC Clearinghouse on Teaching and Teacher Education. http://files.eric.ed.gov/fulltext/ED400259.pdf

Aksit, N. (2007). Educational reform in Turkey. International Journal of Educational Development, 27(2), 129-137. http://yoksis.bilkent.edu.tr/pdf/files/10.1016-j.ijedudev.2006.07.011.pdf

Arneson, J., Rothwell, W. & Naughton, J. (2013). Training and development competencies redefined to create competitive advantage. Alexandria VA: ASTD.

Avalos, B. (2011). Teacher professional development in Teaching and Teacher Education over ten years. Teaching and Teacher Education, 27(1), 10-20. http://dx.doi.org/10.1016/j.tate.2010.08.007

Avalos, B. (2000). Policies for teacher education in developing countries. International Journal of Educational Research, 33(5), 457-474. http://dx.doi.org/10.1016/S0883-0355(00)00029-X

Ball, D. L. (1996). Teacher learning and the mathematics reforms: What we think we know and what we need to learn? Phi Delta Kappan, 77(7), 500-508. https://www.questia.com/library/journal/1G1-18540869/teacher-learning-and-the-mathematics-reforms-what

Barnett, M. (2002). Issues and trends concerning electronic networking technologies for teacher professional development: A critical review of the literature. Paper presented at the American Educational Research Association, New Orleans, LA.

Birman, B. F., Desimone, L., Porter, A. C. & Garet, M. S. (2000). Designing professional development that works. Educational Leadership, 57(8), 28-33. http://www.ascd.org/ASCD/pdf/journals/ed_lead/el200005_birman.pdf

Bobrowsky, W., Marx, R. & Fishman, B. (2001, March). The empirical base for professional development in science education: Moving beyond volunteers. Paper presented at the annual meeting of the National Association of Research on Science Teaching, St. Louis, MO. http://www.umich.edu/~hiceweb/papers/2001/BobrowskyNARST2001.pdf

Borko, H. (2004). Professional development and teacher learning: Mapping the terrain. Educational Researcher, 33(8), 3-15. http://dx.doi.org/10.3102/0013189X033008003

Catmali, M. (2006). Gelecek ićçin eğitim hizmetići eğitim kursunun değerlendirilmesi. [Evaluation of in-service education program Teach to the future]. Unpublished Master's thesis. Balűkesir University, Turkey.

Clark, C. M. (1992). Teachers as designers in self-directed professional development. In A. Hargreaves & M. G. Fullan (Eds.), Understanding teacher development. New York: Teachers College Press, pp. 75-84.

Cohen, D. K. & Hill, H. C. (1998). Instructional policy and classroom performance: The mathematics reform in California (RR-39). Philadelphia: Consortium for Policy Research and Education. http://repository.upenn.edu/cpre_researchreports/4/

Darling-Hammond, L. & McLaughlin, M. W. (1995). Policies that support professional development in an era of reform. Phi Delta Kappan, 76(8), 597-604.

Day, C. (1997). In-service teacher education in Europe: Conditions and themes for development in the 21st century. Journal of In-Service Education, 23(1), 39-54. http://dx.doi.org/10.1080/13674589700200002

De Jager, B., Reezigt, G. J. & Creemers, B. P. M. (2002). The effects of teacher training on new instructional behaviour in reading comprehension. Teaching and Teacher Education, 18(7), 831-842. http://dx.doi.org/10.1016/S0742-051X(02)00046-X

Delvaux, E., Vanhoof, J., Tuytens, M., Vekeman, E., Devos, G. & van Petegem, P. (2013). How may teacher evaluation have an impact on professional development? A multilevel analysis. Teaching and Teacher Education, 36, 1-11. http://www.sciencedirect.com/science/article/pii/S0742051X13001078

Desimone, L. M. (2011). A primer on effective professional development. Phi Delta Kappan, 92(6), 68-71. http://dx.doi.org/10.1177/003172171109200616

Desimone, L. M. (2009). Improving impact studies of teachers' professional development: Toward better conceptualizations and measures. Educational Researcher, 38(3), 181-199. http://dx.doi.org/10.3102/0013189X08331140

Desimone, L. M., Garet, M. S., Birman, B. F., Porter, A. C. & Yoon, K. S. (2003). Improving teachers' in-service professional development in mathematics and science: The role of postsecondary institutions. Educational Policy, 17(5), 613-649. http://dx.doi.org/10.1177/0895904803256791

Desimone, L. M., Porter, A. C., Garet, M. S., Yoon, K. S. & Birman, B. F. (2002). Effects of professional development on teachers' instruction: Results from a three-year longitudinal study. Educational Evaluation and Policy Analysis, 24(2), 81-112. http://dx.doi.org/10.3102/01623737024002081

Desimone, L. M., Smith, T. M. & Ueno, K. (2006). Are teachers who need sustained, content-focused professional development getting it? An administrator's dilemma. Educational Administration Quarterly, 42(2), 179-215. http://dx.doi.org/10.1177/0013161X04273848

EU (European Union) (2007). EU on teacher education. [viewed at http://www.eaea.org/news.php?aid=13883&hakustr=teacher%20education#a_13883; not found 22 Nov 2015]

Eylon, B. & Bagno, E. (1997). Professional development of physics teachers through long term in-service program: The Israeli experience. In E. F. Redish & J. S. Rigden (Eds.), The changing role of physics departments in modern universities: Proceedings of ICUPE, The American Institute of Physics (pp. 299-326).

Fullan, M. G. (1991). The new meaning of educational change. USA: Teachers College Press.

Fullan, M. G. (1995). The limits and the potential of professional development. In T. R. Guskey & M. Huberman (Eds.), Professional development in education: New paradigms and practices (pp. 253-268). New York: Teachers College Press.

Fullan, M. G. & Hargreaves, A. (1992). Teacher development and educational change. In M. Fullan & A. Hargreaves (Eds.), Teacher development and educational change (pp. 1-9). London, UK: The Falmer Press.

Fullan, M. G. & Miles, M. B. (1992). Getting reform right: What works and what doesn't. Phi Delta Kappan, 73(10), 745-752.

Grieve, A. M. & McGinley, B. P. (2010). Enhancing professionalism? Teachers' voices on continuing professional development in Scotland. Teaching Education, 21(2), 171-184. http://dx.doi.org/10.1080/10476210903281482

Grossman, G. M. (2013). Developing social capital through national education. In I. R. Haslam, M. S. Khine & I. M. Saleh (Eds.), Large scale reform and social capital building (pp. 127-140). USA: Routledge.

Gur, B. S., Celik, Z. & Ozoglu, M. (2012). Policy options for Turkey: A critique of the interpretation and utilization of PISA results in Turkey. Journal of Education Policy, 27(1), 1-21. http://dx.doi.org/10.1080/02680939.2011.595509

Guskey, T. R. (1985). The effects of staff development on teachers' perceptions about effective teaching. The Journal of Educational Research, 78(6), 378-382. http://dx.doi.org/10.1080/00220671.1985.10885634

Guskey, T. R. (1986). Staff development and the process of teacher change. Educational Researcher, 15(5), 5-12. http://dx.doi.org/10.3102/0013189X015005005

Guskey, T. R. (2002). Professional development and teacher change. Teachers and Teaching: Theory and Practice, 8(3), 381-391. http://dx.doi.org/10.1080/135406002100000512

Guskey, T. R. (2003). What makes professional development effective? Phi Delta Kappan, 84(10), 748-750. http://dx.doi.org/10.1177/003172170308401007

Haase, R. F. & Ellis, M. V. (1987). Multivariate analysis of variance. Journal of Counseling Psychology, 34(4), 404-413. http://psycnet.apa.org/doi/10.1037/0022-0167.34.4.404

Hair J. F., Black, W. C., Babin, B. J., Anderson, R. E. & Tatham, R. L. (2006). Multivariate data analysis. New Jersey: Pearson Prentice Hall.

Hardy, I. (2008). Competing priorities in professional development: An Australian study of teacher professional development and practice. Asia-Pacific Journal of Teacher Education, 36(4), 36, 277-290. http://dx.doi.org/10.1080/13598660802364135

Hargreaves, A. (2000). Editor-in-Chief's introduction representing educational change. Journal of Educational Change, 1(1), 1-3. http://dx.doi.org/10.1023/A:1010008518807

Higher Education Council (2011). Faculty of Education Teacher Education Programs. http://www.yok.gov.tr/web/guest/icerik/-/journal_content/56_INSTANCE_rEHF8BIsfYRx/10279/49875

Incikabi, L. (2012). After the reform in Turkey: A content analysis of SBS and TIMSS assessment in terms of mathematics content, cognitive domains, and item types. Education as Change, 16(2), 301-312. http://dx.doi.org/10.1080/16823206.2012.745758

Johnson, C. & Fargo, J. (2010). Urban school reform enabled by transformative professional development: Impact on teacher change and student learning of science. Urban Education, 45(1), 4-29. http://dx.doi.org/10.1177/0042085909352073

Joyce, B. & Showers, B. (1980). Improving in-service training: The messages of research. Educational Leadership, 37(5), 379-385.

Kealey, K. A., Peterson, A. V., Gaul, M. A. & Dinh, K. T. (2000). Teacher training as a behavior change process: Principles and results from a longitudinal study. Health Education and Behavior, 27(1), 64-81. http://dx.doi.org/10.1177/109019810002700107

Koc, Y., Isiksal, M. & Bulut, S. (2007). Elementary school curriculum reform in Turkey. International Education Journal, 8(1), 30-39. http://files.eric.ed.gov/fulltext/EJ841629.pdf

Linn, G. B., Gill, P., Sherman, R., Vaughn, V. & Mixon, J. (2010). Evaluating the long-term impact of professional development. Professional Development in Education, 36(4), 679-682. http://dx.doi.org/10.1080/19415250903109288

McConnell, K. J., Delate, T. & Newlon, C. L. (2011). Impact of continuing professional development versus traditional continuing pharmacy education on learning behaviors. Journal of American Pharmacists Association, 52(6), 742-752. http://dx.doi.org/10.1331/JAPhA.2012.11080

MNE (Ministry of National Education, Turkey) (2013). Application no: 730010. Message posted 21 January to http://us-mg6.mail.yahoo.com/neo/launch?.rand=19dgf7p5lt28u#mail

MNE (2013). FATIH Project. http://fatihprojesi.meb.gov.tr/tr/english.php

MNE (2012). Milli Eğitim Bakanlığında Mesleki Gelişim [Professional Development in Ministry of National Education]. http://oyegm.meb.gov.tr/www/tanitim/icerik/7

MNE (2011). In-service Training in MoNE. http://hedb.meb.gov.tr/net/

MNE (2011). National Education Statistics 2010-2011. http://sgb.meb.gov.tr/meb_iys_dosyalar/2012_12/06021014_meb_istatistikleri_orgun_egitim_2010_2011.pdf

MNE (2011). Professional development in Ministry of National Education. http://oyegm.meb.gov.tr/www/tanitim/icerik/7

MNE (2011). Teachers' general competencies. http://otmg.meb.gov.tr/YetGenel.html

Minor, L. C., Onwuegbuzie, A. J., Witcher, A. E. & James, T. L. (2002). Preservice teachers' educational beliefs and their perception of characteristics of effective teachers. The Journal of Educational Research, 96(2), 116-127. http://dx.doi.org/10.1080/00220670209598798

Newmann, F. M., King, M. B. & Youngs, P. (2000). Professional development that addresses school capacity: Lessons from urban elementary schools. American Journal of Education, 108(4), 259-299. http://www.jstor.org/stable/1085442

Nunnally, J. C. & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.

OECD (2010). Overview of country results in TALIS: Turkey. http://www.oecd.org/edu/school/43072807.pdf

Olson, C. L. (1979). Practical considerations in choosing a MANOVA test statistic: A rejoinder to Stevens. Psychological Bulletin, 86(6), 1350-1352. http://psycnet.apa.org/doi/10.1037/0033-2909.86.6.1350

Opfer, V. & Pedder, D. (2011). The lost promise of teacher professional development in England. European Journal of Teacher Education, 34(1), 3-24. http://dx.doi.org/10.1080/02619768.2010.534131

Ozer, B. (2001). Ortaöğretim ogretmenlerinin mesleki gelisime yaklasimi [Secondary school teachers' approach to professional development]. Unpublished research report, Eskisehir.

Ozer, B. (2004). In-service training of teachers in Turkey at the beginning of the 2000s. Journal of In-service Education, 30(1), 89-100. http://www.tandfonline.com/doi/pdf/10.1080/13674580400200238

Oztaskin, O. B. (2010). Identifying the in-service training needs of the social studies teachers within the context of lifelong learning. Procedia Social and Behavioral Sciences, 2, 3036-3042. http://dx.doi.org/10.1016/j.sbspro.2010.03.460

Shah, S. M. H., Sultana, N., Hassain, K. S. & Ali, I. (2011). Impact of professional development on the performance of secondary school principals: A managerial perspective. International Journal of Management, 28(4), 92-105. https://www.questia.com/library/journal/1P3-2505650641/impact-of-professional-development-on-the-performance

Smith, K. (2005). Teacher educators' expertise: What do novice teachers and teacher educators say? Teaching and Teacher Education, 21(2), 177-192. http://dx.doi.org/10.1016/j.tate.2004.12.008

Sparks, D. (2002). Designing powerful professional development for teachers and principals. Oxford, OH: National Staff Development Council. http://www.friscoisd.org/docs/default-source/professional-development/designingpowerfulprofessionaldevelopmentforteachersandprincipals_000.pdf

Supovitz, J. A., Mayer, D. P. & Kahle, J. B. (2000). Promoting inquiry-based instructional practice: The longitudinal impact of professional development in the context of systemic reform. Educational Policy, 14(3), 331-356. http://dx.doi.org/10.1177/0895904800014003001

Supovitz, J. & Zief, S. G. (2000). Why they stay away: Survey reveals invisible barriers to teacher participation. Journal of Staff Development, 21(4). http://learningforward.org/publications/jsd/jsd-blog/jsd/2000/09/01/jsd-fall-2000-teacher-quality

Tabachnick, B. G. & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Boston: Allyn and Bacon.

Taylor, M., Yates, A., Meyer, L.H. & Kinsella, P. (2011). Teacher professional leadership in support of teacher professional development. Teaching and Teacher Education, 27(1), 85-94. http://dx.doi.org/10.1016/j.tate.2010.07.005

Torres, R. M. (1996). Without the reform of teacher education there will be no reform of education. Prospects, 26(3), 447-467. http://dx.doi.org/10.1007/BF02195051

Van Driel, J. H., Beijaard, D. & Verloop, N. (2001). Professional development and reform in science education: The role of teachers' practical knowledge. Journal of Research in Science Teaching, 38(2), 137-158. http://dx.doi.org/10.1002/1098-2736(200102)38:2<137::AID-TEA1001>3.0.CO;2-U

Varela, A. M. (2012). Three major sins of professional development: How can we make it better? Education Digest: Essential Readings Condensed for Quick Review, 78(4), 17-20.

Veenman, S., Van Tulder, M. & Voeten, M. (1994). The impact of in-service training on teacher behaviour. Teaching and Teacher Education, 10(3), 303-317. http://dx.doi.org/10.1016/0742-051X(93)E0001-8

Vukelich, C. & Wrenn, L. C. (1999). Quality professional development: What do we think we know? Childhood Education, 75(3), 153-160. http://dx.doi.org/10.1080/00094056.1999.10522003

Wade, R. K. (1985). What makes a difference in in-service teacher education? A meta-analysis of research. Educational Leadership, 42(4), 48-54. http://www.ascd.org/ASCD/pdf/journals/ed_lead/el_198412_wade.pdf

Wilson, S. & Berne, J. (1999). Teacher learning and the acquisition of professional knowledge: an examination of research on contemporary professional development. Review of Research in Education, 24, 173-209. http://www.jstor.org/stable/1167270

Yalın, H. İ. (2001). Hizmetiçi eğitim programlarının değerlendirilmesi. Milli Eğitim Dergisi, 150.

Yoon, K. S., Duncan, T., Lee, S. W. Y., Scarloss, B. & Shapley, K. (2007). Reviewing the evidence on how teacher professional development affects student achievement (Issues & Answers Report, REL 2007-No.033). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southwest. https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=REL2007033

Authors: Dr Tuba Gökmenoğlu is an Assistant Professor in the Dr Fazıl Küçük Faculty of Education, European University of Lefke, Cyprus. She was awarded her PhD from Middle East Technical University in Turkey on Curriculum and Instruction program in 2012. Dr Gökmenoğlu undertook a research fellowship program in Arizona State University 2011-2012.
Email: tgokmenoglu@eul.edu.tr

Dr Christopher M. Clark is a researcher at the Center for Advanced Studies in Global Education at Arizona State University and Professor Emeritus, Michigan State University. His awards include a Fulbright Research Fellowship; Spencer Fellowship; Visiting Professorships at Teachers College, Columbia, at the University of Alberta and at the University of Haifa; and AERA awards for original research and for relating research to the practice of teaching. Dr Clark was Director of the School of Education at the University of Delaware 2000-2006.
Email: christopher.michae.clark@asu.edu
Web: https://webapp4.asu.edu/directory/person/985301

Please cite as: Gökmenoğlu, T. & Clark, C. M. (2015). Teachers' evaluation of professional development in support of national reforms. Issues in Educational Research, 25(4), 442-459. http://www.iier.org.au/iier25/gokmenoglu.html


[ PDF version of this article ] [ Contents Vol 25 ] [ IIER Home ]
© 2015 Issues In Educational Research. This URL: http://www.iier.org.au/iier25/gokmenoglu.html
Created 21 Dec 2015. Last revision: 21 Dec 2015.
HTML: Roger Atkinson [rjatkinson@bigpond.com]