Utility of a conceptual framework within doctoral study: A researcher's reflections
University of New England
The author of this paper provides an example of a conceptual framework that supported her doctoral study and written dissertation in the field of educational psychology. The study was carried out prior to the more recent explicit emphasis on conceptual frameworks in postgraduate research texts and academic literature. The instigation for the development of an explicit conceptual framework was a change in supervisors and their need to be meaningfully included in the journey of the student. The conceptual framework supported the explanation of the multiple theoretical frameworks and literature base, as well as the professional educational context of the study. It also explicitly responded to the literature search in each area, clearly articulating theory and practice in school curriculum and assessment. This in turn supported the definition of research themes and research questions, and the research methodology (data gathering and analysis), from which the meaningful consideration of results could emerge and be shown to be conceptually coherent. The conceptual framework served as a reference for the supervisory relationship, and an organising structure for the written thesis. It also provided a tool of metacognition for the author as she actively constructed and used her conceptual framework to support her doctoral study.
Following on this need, an evolving conceptual framework allowed a spotlight on specific aspects as they became the focus of the learning process. The focus moved from the key concepts and theoretical frameworks underpinning the professional context of the study, to the research themes and questions, to the methodology, to the implementation of the study, to the results and analysis, and finally to the conceptual and practical outcomes of the study. These aspects of the study were all highlighted in different diagrammatic versions of the conceptual framework that became embedded in the dissertation.
The use of the conceptual framework in this study resulted in significant artifacts that became integral to the writing of the dissertation; a series of diagrams. However, when it was time to justify the conceptual framework, there was little guidance available in the literature at the time and a vague definition from the health sciences that a conceptual framework contains descriptive categories within relationships (Minichiello, Axford Greenwood & Sullivan, 1999) was relied on. Despite the scarcity of literature on the role and function of conceptual frameworks in educational psychology at the time (the late 1990s), the actual contribution to the study was profound in supporting a successful instance of doctoral research. The experience presented in this paper reflects and illustrates the now accepted vitality around conceptual frameworks that is evident in core texts for doctoral students (Cooksey & McDonald, 2011; Maxwell & Loomis, 2003; Punch, 2009; Savin-Baden & Howell Major, 2010) and in peer reviewed papers (Leshem, 2007; Leshem & Trafford, 2007; Shields & Tajalli, 2006).
Criteria for the consideration of the utility of conceptual frameworks in doctoral study have been provided by Smyth (2004) and Leshem and Trafford (2007) and are summarised in Table 1. There are some commonalities and some distinctions in these features of conceptual frameworks in doctoral studies. In my reflection on the utility of my conceptual framework I have drawn on factors from both sources, and reconceptualised these into four processes. These four processes provide clear definition of the activity required of a student and can act as a scaffold for new doctoral students and their supervisors. The conceptual framework in this study was useful in: (i) defining the research problem; (ii) establishing theoretical coherence; (iii) organising research design and implementation; and (iv) framing conceptual conclusions.
|||provision of a common language||||unity within appropriate theories|
|||a set of guiding principles against which judgements and predictions could be made||||direction to research design and accompanying fieldwork|
|||a set of reference points from which to locate the research questions within contemporary theorising||||coherence between empirical observations and conceptual conclusions|
|||a structure for organisation of the content of the research and for framing of conclusions for the research|
|Smyth (2004)||Leshem & Trafford (2007)|
The development of the research problem was based on the need for the content of the curriculum and defined educational purposes to inform the use of assessment approaches. The study was thus to follow the development, implementation and evaluation of curriculum-based dynamic assessment. Therefore, all of these distinct fields of practice (school curriculum, conventional and alternative educational assessment, and educational decision making that is informed by assessment) needed to be synthesised into a single conceptual framework. The relationships between the distinct dimensions of the study were then evident. This initial framework is shown in Figure 1.
Figure 1: Initial conceptual framework: Assessment of mathematical learning
(For more information about the study see Berman & Graham, 2002)
The content of assessment was defined by the school curriculum. It was necessary to select a part of that curriculum (for the purposes of the study), and in this case the mathematics curriculum was targeted. The nature of school mathematical learning was thus extracted from the relevant curriculum documents and related literature and is encapsulated in the shaded box. The literature about the purpose of assessment in schools provided four types of educational decision-making that were supported by assessment information. These are listed in the framework, with the three that were relevant for classroom and school psychology assessment shaded to highlight their role in the definition of the research problem.
The literature clearly provided a dichotomy between conventional assessment that has a basis in psychometric theory of assessment reflecting the behaviourist theory of learning and alternative assessment based on social constructivist theories of learning that reflect the social context of learning. Within the two assessment theories, it was essential to consider those approaches appropriate for the professional context of the study. My role as a school psychologist providing clinical assessment to complement classroom assessment was built in through this specification. The assessment approaches were drawn from academic literature and professional practice and one alternative assessment approach, dynamic assessment, that was to be the specific approach to be studied, was shaded in the diagram.
The relationships between assessment content, purpose and approach are shown by bars. The links are not further elaborated in the diagram, but they flag the focus area for the study: that is to examine the support provided by curriculum-based assessment to the three key educational decisions. The shaded boxes clearly defined the aspects of the content, purpose and assessment approach which are focused on in this study.
This initial conceptual framework provided the common language needed in the study and in the supervisory interactions (Smyth, 2004). The language was drawn from the professional setting and from the academic literature that housed the theoretical frameworks that were drawn upon. The language thus articulated became a common language, a vehicle for sharing meaning between me and my supervisors. The explicit tabling of language was particularly vital in this case, since neither of the supervisors was engaged with the key research area of dynamic assessment in their academic work. I therefore had to work hard to ensure that any written work and discussions were based on common understandings of the terminology of dynamic assessment.
In this study, the conceptual framework provided a definition of reality that included the professional context from which the research problem was derived and the language to be used around the context and problem. The derivation of a research problem from within a professional educational setting is common in doctorates in educational practice, and was the case here.
The conceptual framework provided a reference for the members of the learning alliance, the supervisors and the student. Independent supervised study, as required in doctoral study, is generally individual learning within the context of a supervisory relationship with one or more supervisors. It is learning on the social and individual planes as described by Vygotsky (1978), but it is more fluid than the process that occurs in schools, where the social plane has one more competent other (the teacher) who is intellectually distinct from the child learners. In this setting, at the doctoral level, the social plane involves interplay between the supervisors and the student that has a level of distance, but it can also involve complementary competence in that the student becomes the more competent other in some aspects of the study.
Halse and Malfroy's theorising about research supervision assumes a level of supervisor scholarly expertise in the field that is being researched by the student. In my case the supervisors brought significant scholarly expertise, but, because of the circumstances, they needed to draw on the specific educational psychology expertise of the student who was required to construct a representation of her thinking that would be accessible by this new learning alliance. In essence, this continuous development of the scholarly expertise of the supervisors is recognised as a key outcome for supervisors in a doctoral supervision relationship (Halse & Malfroy, 2010). All members of the team change and are changed by their involvement. They also all act as more competent others at times, since all are self-driven, they have a common goal and focus and can activate intentional interpsychological interdependence and intrapsychological outcomes (i.e., individual cognitive development) (Vygotsky, 1987).
My contribution to the supervisory learning alliance was essential in this case. It acted to support the development of expertise in the field of dynamic assessment for the supervisors so they could support my progress in my research. This situation, where the student leads the learning in some dimensions of the study, occurs in most doctoral journeys as the student's knowledge becomes increasingly developed and exceeds that of the supervisors in some areas. It is then essential for the student to frame and filter this specific knowledge in order to take the supervisors along with him/her. The supervisors, as the more competent scholars, are essential to allow the testing and substantiating of new knowledge in the social and cultural context of doctoral study. In my specific situation the conceptual framework provided a firm reference for the process, that elucidated the reality and the language of the knowledge development and supported clear communication (Halse & Malfroy, 2010).
My initial conceptual framework was an explicit mapping of the reality of the research problem in terms of the nature of a specific domain of the school mathematics curriculum, of the purposes of curriculum-based assessment in schools and the actions that follow assessment, and of the conventional and contemporary theory and practice of assessment by teachers and school psychologists. This initial diagram (Figure 1) provided a clear structural overview of the research study, it provided terminology to be used, and it became the culmination of the first chapter in the written dissertation in which the research problem was defined.
The theoretical basis of dynamic assessment was also explored and articulated in the second version of the conceptual framework in Figure 2.
Figure 2: Second iteration: Dynamic assessment in the conceptual framework
The theory that presented opportunities for review in this study was concerned with curriculum-based assessment, and in particular, dynamic assessment of school learning. My study thus explored the Vygotskian theoretical underpinnings of dynamic assessment and the ability of such assessment to respond to the purposes of assessment, and to the nature of school mathematical learning.
Figure 3: Conceptual framework with the specific domain of mathematical learning
The substantial research basis for the nature of learning in the selected mathematical domain, enables the exploration of the substantial research basis for the nature of learning in the selected mathematical domain, enables the exploration of understanding as a domain that was considered pivotal to school mathematics. understanding as a domain that was considered pivotal to school mathematics. The outcomes of the study would therefore have significant implications for mathematical teaching and learning. The diagram accompanying this phase of the study is shown in Figure 3.
It specifies the domain of mathematical learning as defined through cognitive research, what this learning looks like and what tasks could be used in a dynamic assessment procedure. The definition of understanding of place-value numeration is now included in the content box in this framework.
This iteration of the conceptual framework clearly established the theoretical coherence of the research problem, articulating and drawing together the multiple theories involved in the reality and providing a basis for theorising (Smyth, 2004) and for generating new knowledge. There were multiple layers of theory in this study. They include theory concerning assessment purposes, school mathematical curriculum and the nature of learning, assessment of learning, and a focus on theories underlying dynamic assessment. All of these needed to be meaningfully tied together to represent the reality of the practice of assessment in schools. This is an acknowledgement and articulation of the multiple theoretical perspectives inherent in the reality of the study but is not a "unity" of theory as postulated by Leshem & Trafford (2007). Perhaps the unity is to come in the new knowledge and intermediate theory that is the outcome of the study. The multiple perspective foundation for the study was clearly shown in the three conceptual framework diagrams, that were then developed to articulate the research themes and questions that underpinned the methodology of my study.
The research themes and questions were mapped onto the conceptual framework, showing how they related to the issues and theoretical bases of the reality. From these the research design and the plan were then derived. Since the research questions were explicit in the framework, it was easy to respond to these systematically and to consider the implications of the data analysis in relation to the questions. This included the selection of specific representative assessment tools for classroom and clinical assessment, development of an alternative dynamic assessment tool and consideration of how the data generated by these instruments could be compared. I felt that the study should use both conventional and alternative assessment procedures and evaluate them with respect to the specified educational decision-making.
Figure 4: Conceptual framework with research themes and questions
The research themes articulated the relationships between the theoretical and practical dimensions that were to be the focus of my study. This first theme investigated the conventional and alternative assessment procedures with respect to the purposes of assessment. Theme 1 generated three questions, one for each of the educational decisions relevant to a classroom context. The subsequent themes focused on the theory and practice of the alternative assessment procedure, dynamic assessment. The second theme, with three research questions, considered the theoretical foundation of dynamic assessment, that is, Vygotskian theory of learning and assessment, in terms of its contribution to assessment of school mathematical learning. Two questions considered the contribution of the Vygotskian theory to assessment as well as the contribution of this study to Vygotskian theory. The third question in this second theme sought to evaluate the conceptual framework developed during this thesis. The third theme considered the practical issues associated with the use of dynamic assessment, since these have been presented as reasons for the low level of acceptance of dynamic assessment in schools.
The research design was then developed in response to these research questions. An excerpt of the conceptual framework showed the process for data collection, that was the administration of conventional and alternative assessment. The selected assessment procedures (DMT, KeyMath-R and STOPV-DA) were then shown in an excerpt of the framework in Figure 5 that articulated the plan for the administration of the assessment to be used in the study.
Figure 5: Plan for the administration of the selected assessment instruments
Analysis of the data was also anchored to the conceptual framework and is shown in the next iteration of the conceptual framework in Figure 6. The classroom pencil and paper test (DMT), the individual clinical procedure (KeyMath-R) and the alternative assessment (STOPV-DA) all had defined procedures for scoring, analysis and interpretation. In order to be able to compare the results of these conventional and alternative assessments, it was vital to establish the reliability, construct validity and item difficulty for the three instruments, and this was done in a preliminary analysis using Item Response Theory and Rasch analysis.
Figure 6: Preliminary data analysis (Rasch analysis) within the conceptual framework
The study involved the development and implementation of a clinical assessment procedure for assessment of a specific domain of school mathematics. The domain was defined by cognitive research and in reference to the curriculum, and tasks were selected to be used in the assessment. The tasks reflected the range and complexity of the content of the specific domain. The assessment procedure involved a pre-test, mediation and post-test within an individual interview setting between a student and the assessor. The mediation established and maintained an optimum relationship, and focused on the structure of the content of the domain, i.e., children's understanding of place-value of numbers. Interpretation of student learning occurred both during and after the assessment. It involved exploration of the student's actual and assisted achievement and the context of these two levels of functioning. The resulting information from the assessment was reported in terms of the specific cognitive, general cognitive, social and emotional aspects of functioning as well as specifications of intensity and types of teaching needed.
Ultimately, the conceptual framework was available as a reference for the conceptual and practical conclusions of the study (Smyth, 2004). The explicit coherence between the study and the conclusions was thus evident within the framework and the interpretation of the data with respect to the theory and reality of the study was meaningful. In particular, the key research outcome, a framework for the development of curriculum-based dynamic assessment, was presented within the initial conceptual framework (Figure 7), providing a context for it, and thus conceptual validity. This final framework has become an intermediate theoretical framework, that is ready for further testing. It retains its conceptual coherence that would need to be accounted for in future research.
Figure 7: Dynamic assessment procedure for a specific domain of school mathematical learning
As well as this explicit set of research outcomes, there is a subsidiary outcome. The conceptual framework that was derived from my school psychology practice and that contextualised the research problem, is in fact a framework that can be used in other contexts to guide assessment decisions (Wong, Graham, Hoskyn & Berman, 2008). Any assessment needs to take into account the content and purpose of the assessment, and this will inform the selection of assessment procedures. Although this conceptualisation was derived from practice in school psychology assessment it is a much wider and profound framework that can be applied to any assessment situation.
The conceptual framework provided a strong structure for organisation of the content of the research (Smyth, 2004) and thus the dissertation, in which it was adapted to respond to the changing focus of the chapters. The framework also allowed the clear identification of limitations of this study as well as the potential opportunities for future research. The outcomes, within the initial framework, are kept bounded by the limitations of the study. Yet, the potential for translating into other contexts, with different curriculum areas is present, and available for others to test.
Although the dissertation presents the use of the conceptual framework as orderly and sequential, it was not. Each of these phases was visited many times and adjustments were made as the learning journey progressed. However, the framework supported the sequential presentation of the doctoral study, in order for the examiners to make sense of it.
|Defining the research problem|
|Establishing theoretical coherence|
|Organising research design and implementation|
|Framing conceptual conclusions|
Shields and Tajalli (2006) and Leshem (2007) have demonstrated that the explicit development of conceptual frameworks supports enhanced higher degree study and in my case it proved to be a vital aspect of the presentation of a written dissertation for other academics who have not been on the learning journey with me, but who needed to be convinced of the new learning and of the validity of the process through which this knowledge was being generated. Because doctoral studies are prolonged, it is vital to reduce that intense and extensive process into a few hundred thousand words that is accessible by others for the purpose of examination and for future readers. The expression of my framework within a conceptual diagram also allowed brevity, to encapsulate the complexity of the realities of the study, acting as a scaffold for the process (Wisker, 2005).
The derivation of the key concepts, the theoretical perspectives and the practical context within which the study was developed are clear in the first three iterations of the framework. The research themes and questions along with the methodology are outlined in the subsequent framework diagrams. Outcomes of the research with respect to the development of a curriculum-based dynamic assessment procedure, are elaborated in the final framework, and the overall framework is valid for considering many different contexts for assessment of school learning.
The development and reference to this series of frameworks throughout the process of this doctoral journey supported the inclusion of two new supervisors through an easily accessible conceptualisation and a common language. While pages of written words are necessary, with reference to the literature base, the frameworks in this study allowed the clear articulation of the key concepts and their relationship, as well as a focus on individual areas as needed in the various phases of the study and the relevant chapters. It provided a reference for continued literature searches, as well as for the logistics of the study. The use of the conceptual framework supported the higher order thinking that is essential in doctoral study. It resulted in generation of new knowledge that was, in turn, presented within the conceptual frameworks, demonstrating conceptual coherence or conceptual validity.
As other doctoral journeys are shared, it is clear that students develop and articulate their conceptualisations in different ways that meet their particular needs. These can be in the form of matrices (Smyth, 2004) or metaphors (Kelly, 2011; Mackenzie & Ling, 2009) or, as in this case, articulated through diagrams. My conceptual diagrams that evolved throughout the journey explicitly supported the research process, provided the shared meaning necessary within the supervisory relationship and became significant artifacts in the written thesis. My experience illustrates the power of conceptual frameworks as a vital aspect of doctoral study, and I present the reflective conceptualisation for use by other students and their supervisors.
Berman, J. & Graham, L. (2002). School counsellor use of curriculum-based dynamic assessment. Australian Journal of Guidance and Counselling, 12(1), 21-40.
Cooksey, R. & McDonald, G. (2011). Surviving and thriving in postgraduate research. Prahan, VIC: Tilde University Press.
Gredler, M. & Shields, C. (2008). Vygotsky's legacy: A foundation for research and practice. New York: Guildford Press
Grigorenko, E. & Sternberg, R. J. (1998). Dynamic testing. Psychological Bulletin, 124(1), 75-111.
Halse, C. & Malfroy, J. (2010). Retheorizing doctoral supervision as professional work. Studies in Higher Education, 35(1), 79-92. http://dx.doi.org/10.1080/03075070902906798
Haywood, C. & Lidz, C. (2007). Dynamic assesment in practice: Clinical and educational applications. Cambridge: Cambridge University Press.
Kelly, F. (2011). 'Cooking together disparate things': The role of metaphor in thesis writing. Innovations in Education and Teaching, 48(4), 429-438. http://dx.doi.org/10.1080/14703297.2011.617088
Knight, S. & Cross, D. (2012). Using contextual constructs model to frame doctoral research methodology. International Journal of Doctoral Studies, 7, 39-62. http://ijds.org/Volume7/IJDSv7p039-062Knight234.pdf
Knight, S., Halkett, G. & Cross, D. (2010). The context and contextual constructs of research. In 5th Conference on Qualitative Research in IT, Brisbane. http://espace.library.curtin.edu.au/R?func=dbin-jump-full&local_base=gen01-era02&object_id=154947
Leshem, S. (2007). Thinking about conceptual frameworks in a research community of practice: A case of a doctoral programme. Innovations in Education and Teaching International, 44(3), 287-299. http://dx.doi.org/10.1080/14703290701486696
Leshem, S. & Trafford, V. (2007). Overlooking the conceptual framework. Innovations in Education and Teaching International, 44(1), 93-105. http://dx.doi.org/10.1080/14703290601081407
Lidz, C. (1987). Dynamic assessment: An interactional approach to evaluating learning potential. New York: Guilford.
Mackenzie, N. & Ling, L. (2009). The research journey: A Lonely Planet approach. Issues in Educational Research, 19(1). http://www.iier.org.au/iier19/mackenzie.html
Maxwell, J. & Loomis, D. (2003). Mixed methods design: An alternative approach. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioural research (pp. 241-272). Thousand Oaks, CA: Sage.
Punch, R. (2009). Introduction to research methods in education. London: Sage.
Ross, S. (1986). The development of children's place-value numeration concepts in grades two through five. Paper presented at the Annual Meeting of the American Educational Research Association (67th, San Francisco, CA, April 16-20, 1986). http://www.eric. ed.gov/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=ED273482
Savin-Baden, M. & Howell Major, C. (2010). New approaches to qualitative research: Wisdom and uncertainty. London: Routledge.
Shields, P. & Tajalli, H. (2006). Intermediate theory: The missing link in successful student scholarship. Journal of Public Affairs Education, 12(3), 313-334. http://www.jstor.org/stable/40215744
Vygotsky L. S. (1987). The collected works of L. S. Vygotsky Vol. 1: Problems of general psychology. R. W. Rieber & A. S. Carton (Eds.) & N. Minick, (Trans.). New York: Plenum Press.
Wisker, G. (2005). The good supervisor: Supervising postgraduate and undergraduate research for doctoral theses and dissertations. London: Palgrave Macmillan.
Wong, B., Graham, L., Hoskyn, M. & Berman, J. (2008). The ABCs of learning disabilities. Second Edition. Burlington, MA: Elsevier Academic Press.
|Author: Jeanette Berman has just been appointed senior lecturer in educational psychology at Massey University in New Zealand, after a number of years lecturing in special education and educational psychology at the University of New England, Australia. Her research interests include psycho-educational assessment, curriculum based dynamic assessment, learning difficulties and mathematical learning.|
Please cite as: Berman, J. (2013). Utility of a conceptual framework within doctoral study: A researcher's reflections. Issues in Educational Research, 23(1), 1-18. http://www.iier.org.au/iier23/berman.html