An instrument for investigating Chinese language learning environments in Singapore secondary schools
Siew Lian Chua
St Andrew's Junior College, Singapore
Angela F. L. Wong & Der-Thanq Chen
Nanyang Technological University, Singapore
This paper describes how a new classroom environment instrument, the 'Chinese Language Classroom Environment Inventory (CLCEI)', was developed to investigate the nature of Chinese language classroom learning environments in Singapore secondary schools. The CLCEI is a bilingual instrument (English and Chinese Language) with 48 items written in both English and Chinese. The English version of the CLCEI was customised from the What is happening in this class? (WIHIC) questionnaire (Fraser, Fisher, & McRobbie, 1996) and its Chinese version was modified from the Taiwanese Chinese version of the WIHIC questionnaire (Huang & Fraser, 1997). A rigorous 5 stage translation process involving 7 different focus groups with the application of the 4 step Nominal Focus Group technique (Moore, 1987; Stewart & Shamdasani, 1990) was adopted to develop the CLCEI. The CLCEI was validated with a sample of 1460 secondary three (grade 9) students from 50 express stream (above average academic ability) classes in 25 secondary schools in Singapore. The validation results indicated that each of the scales exhibited high internal consistency reliability and satisfactory discriminant and factorial validity. The validation results also indicated that each scale of the CLCEI had the ability to differentiate between perceptions of students from different Chinese language classes. The detailed results were reported in Chua, Wong and Chen (2006). The purpose of this paper is to share the detailed procedures of each translation stage and the outcome obtained from each stage. A discussion of the findings of the translation process is also provided.
A distinctive feature of a classroom learning environment instrument is that it usually consists of a number of scales for assessing different dimensions of a classroom learning environment. These scales have always been classified into one of the three basic dimensions according to Moos' (1974) scheme of scale classification. The three basic dimensions as defined by Moos (1974) are as follows.
The Singapore government has always emphasised the importance of learning the Chinese language (http://www.moe.gov.sg/). Since the research literature had indicated that there were associations between students' cognitive and affective learning outcomes and their perceptions of psychosocial characteristics of their classroom environment (Fraser, 1986, Haertel, Walberg, & Haertel, 1981), an understanding of the nature of the Chinese language classroom learning environments would be beneficial to both teachers and students in the teaching and learning of the language.
In order to carry out the investigation in Chinese language classroom, the instrument, the Chinese Language Classroom Environment Inventory (CLCEI), was developed for use in this study. It is a bilingual instrument with 48 items written in both English and Chinese, adapted from the What is happening in this class? (WIHIC) questionnaire (Fraser, Fisher, & McRobbie, 1996). It has been validated and found to exhibit high internal consistency reliability and satisfactory discriminant and factorial validity (Chua, Wong & Chen, 2006). For the student-actual form, the Cronbach alpha coefficients ranged from .82 to .91 when the individual student's score was used as the unit of analysis and from .87 to .96 when the class mean was used as the unit of analysis. The discriminant validity is described as the extent to which a scale measures a unique dimension not covered by the other scales of the instrument. The results indicated that the mean correlation of a scale with the other five scales ranged from .44 to .52 for the student-actual form of the CLCEI and from .56 to .68 for the student-preferred form when using the individual student's score as the unit of analysis. The factor structure of the CLCEI under a factor analyses (with varimax rotation) showed that all 48 items of both the student-actual form and student-preferred form loaded neatly into their six a priori scales with all items having factor loadings greater than 0.40 on their respective scale (Chua, 2004; Chua, Wong & Chen, 2006).
The WIHIC was chosen for this study to develop the CLCEI for two reasons. Firstly, The WIHIC had been developed with the best features of the existing instruments to include the salient scales of these instruments. Secondly it allowed the exclusion of irrelevant scale(s) to suit any classroom environment under study without affecting the reliability and validity of the instrument (Fraser, Fisher, & McRobbie, 1996). The English version of the CLCEI was adapted from the WIHIC and it was translated into Singapore Chinese for use in this study. Although there is a Taiwanese Chinese version of the WIHIC questionnaire (Huang & Fraser, 1997), it was not suitable for use in the Singapore context because of differences in culture and language use. The Taiwanese version was written in traditional Chinese characters whereas simplified Chinese characters are used in Singapore. Therefore the Taiwanese Chinese version of the WIHIC questionnaire was only used as a reference in the translation process.
The back translation technique was used by Huang and Fraser (1997) to translate the WIHIC from English to Taiwanese Chinese and by Okan (2008) from English to Turkish. However, the translation procedures reported in these studies did not indicate the involvement of focus groups and Nominal Group Techniques which were also used in translating the WIHIC to the CLCEI. These procedures were included to help ensure that the original meaning and integrity of each item in WIHIC remained intact in the Singapore Chinese version. Thus, the purpose of this paper is to share the experience of using 'back translation', focus groups and the NGT when translating instruments from one language to another. The back translation, in this context, is a translation of a translated Chinese item back into the English language of the original item, made without reference to the original item. As the validation results showed that the CLCEI is a valid bilingual instrument written in both English and simplified Chinese characters developed through the 5 stage rigorous translation procedure, sharing of the results from each stage would also help those researchers from making similar mistakes when using the back translation method. In addition, the 4 step focus group technique used in this study, that is, the Nominal Group Technique (NGT) suggested by Moore (1987) and Stewart and Shamdasani (1990) has its significant application in decision making using focus groups.
The 5 stage back translation process is as follows.
Stage 1: customising and drafting of Chinese items from the original WIHIC
Stage 2: focus group validation of the Chinese draft
Stage 3: back translation of the validated Chinese draft to English
Stage 4: appraisal of back translated English version with the original English version
Stage 5: redrafting the inappropriately phrased Chinese item(s)
The purpose of planning this 5 stage translation procedure was to decrease the likelihood of items being inappropriately translated and to increase the accuracy, reliability and readability of the bilingual CLCEI (Chinese Language Classroom Environment Inventory). After completing stage 1, the whole translation process was repeated from stage 2 to stage 5 until all the 48 items were satisfactorily modified. The 5 stage back translation procedure was slightly modified during the second and third cycles to improve the effectiveness of the translation. Detailed procedures of each stage are described in the following paragraphs.
Firstly, the original English version (Fraser, Fisher, & McRobbie, 1996) was customised for use in Chinese Language classrooms by adding the phrase 'Chinese Language' where appropriate. Secondly, the 'Investigation' scale was excluded because the 8 items in the 'Investigation' scale were more suitable for use in the scientifically related experimental environments rather than for language classroom environments. Thirdly, the Chinese characters in the Taiwanese Chinese version were converted from the traditional Chinese form to the simplified form because the latter form is used in Singapore. Fourthly, each English item was translated by using one of the two translation approaches, translate literally and translate with meaning, as suggested by Newmark (1988). Translate literally means direct translation of the English phrase to a Chinese phrase without changing the sentence structure whereas translate with meaning means that the translation focuses mainly on maintaining the original meaning of the item but the sentence structure may be altered.
In order to standardise the translation approach, Chinese language experts were consulted regarding the method of translation with reference made to the Taiwanese Chinese version of the WIHIC (Huang & Fraser, 1997). Each item was then translated literally if no grammatical error and misrepresentation of meaning was evident. Otherwise, the item was translated with meaning.
The outcome at the end of stage 1 was a preliminary draft Singaporean Chinese version of the WIHIC questionnaire. It consisted of six eight-item scales, namely 'Student Cohesiveness', 'Teacher Support', 'Involvement', 'Task Orientation', 'Cooperation' and 'Equity', with a total of 48 Chinese items and their corresponding original English items.
The four step NGT procedures were carried out at this stage with another focus group comprising five effectively bilingual educators. They had not been involved in stage 2 of the translation process. The outcome of this stage was a finalised back translated English version which would be used in stage 4.
Appendix 1 summarises the 5 stage translation procedure described above. Appendix 2 depicts the whole development process, which underwent three cycles of the 5 stage translation procedures.
Over interpretation meant that the original English item was translated using a Chinese phrase with 'stronger' meaning than its original. As a result, the back translated English version of the Chinese draft was different in meaning and probably different in sentence structure from the original English version. For example, the meaning of the original item 15, was 'The Chinese language teacher moves about the class to talk with me' and its back translated English item, read 'The Chinese language teacher will talk to me during his inspection of the class.' Apparently, the Chinese translation over interpreted and altered the meaning of 'move about' to 'inspection'.
The second and third errors were due to over interpretation when the translator underwent the English back translation. For example, the original item 1 'I develop friendship with the students in this Chinese language class' was translated appropriately into Chinese as The English back translation reads, 'I can develop friendship with the students in this Chinese language class.' The error of the back translation is rather obvious because 'can' is not used in the Chinese version. This error was probably due to the intention of providing a literal English translation of the Chinese version. However, by so doing, the meaning of the sentence is slightly altered. This is one of the limitations of the back translation method. In this particular case, even though the Chinese was translated appropriately, due to the error introduced in the back translation, the item had to be re-examined. In other words, the back translation method is too strict to accept certain items.
The analyses of the overall procedures and findings indicated that these errors occurred mainly because inadequate instructions were given to the focus group members before they carried out their assigned tasks. Therefore, the ten inappropriately drafted Chinese items were redrafted and went through the second cycle of the translation procedure from stage 2 to stage 5 again.
Firstly, the back translation method used in the modification process was useful and desirable as some inappropriately translated Chinese items were not detected before comparing the meanings of the back translated English item with its corresponding original English item. Therefore, the use of the back translation method for counter checking the accuracy of the translations was useful.
Secondly, over interpreting the original meaning of an English item and neglecting the differences in the usage of the two languages would lead to an inappropriate translation of the items as was seen in this development of the CLCEI.
Thirdly, detailed verbatim instructions were essential when carrying out the four step Nominal Group Technique (NGT) used in the development of the CLCEI. Reading out a set of detailed verbatim instructions to the focus group members before they carried out the assigned tasks helped them understand clearly what had to be done so as to avoid making similar procedural mistakes that would incur unnecessary repetitions of the translation procedures.
Fourthly, the readability level of the CLCEI was found to be high, as students did not have problems in understanding the meanings of the items during the implementation stage. It could be because the items of the CLCEI were presented in both English and Chinese. At the end of the survey, students were asked about the usefulness of having the CLCEI presented in two languages. Many students reported that reading both versions helped them to understand the meaning of the items better. As a result the data obtained could be more reliable. This also indicated the usefulness of using a bilingual instrument in this study.
However, there are some limitations in the translation procedures used in the development of the CLCEI. For example, although the use of the back translation technique helps to make the translation process more rigorous as indicated at the end of the second cycle, appropriate Chinese translated items may be rejected due to poor English back translation rather than poor Chinese translation. It may have caused erroneous results requiring the implementation of the next cycle when it was actually not necessary.
There is also another concern regarding the structural and conceptual equivalence between two languages as suggested by Potaka and Cochrane (2004). Structural or semantic equivalence refers to the degree to which one language shares similar grammatical constructions with another language and contains words or phrases with similar or identical meaning. Whereas conceptual equivalence relates to the extent to which concepts and ideas are transferable between cultures. Concepts relevant to one culture may not apply to another culture, or may be thought of differently. The translation of instruments could be more effective if the researchers also take into consideration these structural and conceptual differences between two languages used by the two instruments concerned by conducting a pilot study to ensure instruments are culturally valid. Lastly, practical challenges such as adequate bilingual experts and technical reviewers may continue to be a problem for implementing the translation procedures suggested in this study. Overall, although the 5 stage translation procedures suggested in this study was found to be lengthy and required substantive manpower, the validation results (Chua, Wong, & Chen, 2006) do provide support for the confident future use of this translation procedure to translate questionnaires for use in cross country studies where the educational cultures of the countries involved are quite different.
In summary, the development of the CLCEI has made several contributions to the educational research field. Firstly, the use of a bilingual instrument to investigate the nature of a particular language classroom learning environment could be useful because it could help students to cross-check the meanings of items when they are in doubt and this could help raise the integrity of the data collected. Secondly, the simplified Chinese form of the CLCEI could be used in other countries which use the simplified form of the Chinese language, for example China and Malaysia, to investigate the nature of classroom learning environments. Thirdly, the 5 stage translation procedure used in this study could be used for future cross country studies as a reference for translating questionnaires written in one country's language to another language used in another country. Fourthly, the whole process of developing the CLCEI has exposed the importance of achieving structural and conceptual equivalence (Potaka & Cochrane, 2004) in the translation process by taking into consideration the language and cultural differences at every stage of the development cycle. Lastly, the modified 4 step Nominal Group Technique (NGT) used in this study had also been proven as an effective focus group technique for use in making decisions.
Chua, S. L., Wong, F. L., & Chen, D. (2006). Validation of the 'Chinese Language Classroom Learning Environment Inventory' for investigating the nature of Chinese language classrooms. Issues in Educational Research, 16(2), 139-151. http://www.iier.org.au/iier16/chua.html
Fisher, D. L., & Fraser, B.J. (1983). Use of work environment scale (WES) to assess science teachers' perceptions of school environment. European Journal of Science Education, 5, 231-233.
Fraser, B. J. (1986). Classroom environment. London: Croom Helm.
Fraser, B. J. (1998). Science learning environments: Assessment, effects and determinants. In B. J. Fraser, & K. G. Tobin (Eds.), International handbook of science education (pp.527-564). Dordrecht, The Netherlands: Kluwer.
Fraser, B. J. (2002). Learning environments research: Yesterday, today and tomorrow. In S. C. Goh, & M. S. Khine (Eds.), Studies in educational learning environments: An international perspective (pp. 1-25). Singapore: World Scientific.
Fraser, B. J. (2007). Classroom learning environments. In S. K. Abell, & N. G. Lederman (Eds.). Handbook of research on science education (pp. 103-124). Mahwah, NJ: Lawrence Erlbaum.
Fraser, B. J., Anderson, G. J., & Walberg, H. J. (1982). Assessment of learning environments: Manual for Learning Environment Inventory (LEI) and My Class Inventory (MCI) (3rd Ed.). Bentley, WA: Western Australian Institute of Technology.
Fraser, B. J., Fisher, D. L., & McRobbie, C. J. (1996). Development, validation and use of personal and class forms of a new classroom environment instrument. Paper presented at the annual meeting of the American Educational Research Association, New York.
Fraser, B. J., & Walberg, H. J. (1991). Educational environments: Evaluation, antecedents and consequences. Pergamon, London.
Haertel, G. D., Walberg, H. J., & Haertel, E. H. (1981). Social-psychological environments and learning: A quantitative synthesis. British Educational Research Journal, 7(1), 27-36.
Huang, I. T. C., & Fraser, B. J. (1997). The development of a questionnaire for assessing student perception of teaching environments in Taiwan and Australia. Paper presented at the annual meeting of the National Association for Research in Science Teaching, Chicago.
Kim. H. B., Fisher, D. L., & Fraser, B. J. (2000). Classroom environment and teacher interpersonal behavior in secondary school classes in Korea. Evaluation and Research in Education, 14, 3-22.
Margianti, E. S., Fraser, B. J. & Aldridge, J. M. (2001). Classroom environment and students' outcomes among university computing students in Indonesia. Paper presented at the annual meeting of the American Educational Research Association, Seattle, WA.
Ministry of Education, Singapore. (2004). Report of the Chinese Language Curriculum and Pedagogy Review Committee. [viewed 6 Apr 2009]. http://www.moe.gov.sg/media/speeches/2004/sp20041124.htm
Moore, C. M. (1987). Group techniques for idea building. Newbury Park, CA: Sage.
Moos, R. H. (1974). The social climate scales: an overview. Consulting Psychologist Press, Palo Alto, CA.
Newmark, P. (1988). A textbook of translation. NJ: Prentice Hall.
Okan, Z. (2008). Computing laboratory classes as language learning environments. Learning Environment Research, 11(1), 31-48.
Potaka, L., & Cochrane, S. (2004). Developing bilingual questionnaires: Experiences from New Zealand in the development of the 2001 Maori Language Survey. Journal of Official Statistics, 20(2), 289-300.
Stewart, D. W., & Shamdasani, P. N. (1990). Focus groups: Theory and practice (3rd ed.). Newbury Park, CA: Sage.
CLCEI: Chinese Language Classroom Environment Inventory
|Tasks||Cycle 1||Cycle 2||Cycle 3|
customising and drafting of items
Focus group validation of the draft
Back translation of the validated draft to English
Appraisal of back translation version with customised English version
Redrafting the inappropriately phrased items
|*NGT technique (Moore, 1987; Stewart & Shamdasani, 1990). The four-steps are (a) silent generation of ideas in writing, (b) round robin recording of ideas, (c) serial discussion of the list of ideas, and (d) voting.|
|Authors: Dr Siew Lian Chua is a Lecturer in Computer Science at St. Andrew's Junior College. She is a member of AARE and was one of the awardees for the 'Travel Award, 1997' given by the Australian Association of Research in Education. The awarded thesis entitled 'Computer Anxiety: A meta-analyses' was extracted from her Master Degree Thesis. Email: email@example.com
Dr Angela F. L. Wong is an Associate Professor at the National Institute of Education, Nanyang Technological University, Singapore.
Dr Victor Der-Thanq Chen is an Associate Professor at the National Institute of Education, Nanyang Technological University, Singapore.
Please cite as: Chua, S. L., Wong, A. F. L. & Chen, D.-T. (2009). An instrument for investigating Chinese language learning environments in Singapore secondary schools. Issues In Educational Research, 19(2), 100-113. http://www.iier.org.au/iier19/chua.html