Validation of the 'Chinese language classroom learning environment inventory' for investigating the nature of Chinese language classrooms

The Chinese Language Classroom Environment Inventory (CLCEI) is a bilingual instrument developed for use in measuring students' and teachers' perceptions toward their Chinese Language classroom learning environments in Singapore secondary schools. The English version of the CLCEI was customised from the English version of the 'What is happening in this class? (WIHIC)' questionnaire (Fraser, Fisher, & McRobbie, 1996) and its Chinese version was modified from the Taiwanese Chinese version of the WIHIC questionnaire (Huang & Fraser, 1997).

The CLCEI consists of six 8-item scales examining 6 different dimensions of the Chinese language classroom learning environments, namely, Student Cohesiveness, Teacher Support, Involvement, Cooperation, Task Orientation and Equity. It was validated with a sample of 1460 secondary three (grade 9) students from 50 express stream (above average academic ability) classes in 25 secondary schools in Singapore. Various statistical procedures were undertaken to examine the factor structure, validity and reliability of the scales of the CLCEI. The validation results indicated that each of the scales exhibited high internal consistency reliability and satisfactory discriminant and factorial validity. The validation results also indicated that each scale of the CLCEI had the ability to differentiate between perceptions of students from different Chinese language classes.

Background

Among these instruments, the Learning environment inventory (LEI) (Fraser, Anderson & Walberg, 1982) and the Classroom environment scale (CES) (Fisher & Fraser, 1983) are the two pioneer environment instruments and many of their scales have been adapted or modified for use in other instruments developed at a later date. For example, the MCI (Fisher & Fraser, 1981) was simplified from the LEI. The CUCEI (Fraser & Treagust & Dennis, 1986) adapted items from both the LEI and CES. The What is happening in this class (WIHIC)? questionnaire (Fraser, Fisher & McRobbie, 1996) was developed using the best features of the existing instruments, adapting their salient scales. It also included new scales that accommodated contemporary educational concerns. The final version of the WIHIC consisted of seven eight-item scales, namely Student Cohesiveness, Teacher Support, Involvement, Investigation, Task Orientation, Cooperation and Equity where the first six scales were adapted from the existing instruments and the Equity scale was introduced to address new educational concerns of gender equality. The WIHIC questionnaire has also been translated into various languages, such as Taiwanese Chinese (Huang & Fraser, 1997), Korean (Kim, Fisher & Fraser, 2000) and Indonesian (Margianti, Fraser & Aldridge, 2001) for use in different countries. Reviews of classroom environment studies also indicated that there were associations between students' perceptions towards their classroom learning environments and their cognitive and affective learning outcomes (Fraser, 1986; Haertel, Walberg & Haertel, 1981).

In Singapore, associations between learning outcomes and classroom learning environments were found in Chemistry classes (Wong & Fraser, 1995), Geography classes (Teh & Fraser, 1995) and Mathematics classes (Goh, Young & Fraser, 1995). However, such a study had not been conducted for Chinese language classrooms in Singapore. It was therefore the primary objective of this study to develop an appropriate instrument to assess the psychosocial characteristics of Chinese language classroom learning environments in Singapore secondary schools for the purpose of investigating its associations with students' motivation in learning the Chinese language. With this in mind, the Chinese language classroom environment inventory (CLCEI), was developed (Chua, Wong & Chen, 2000). It is a bilingual instrument with 48 items written in both English and Chinese. The English version of the CLCEI was customised from the What is happening in this class? (WIHIC) questionnaire (Fraser, Fisher & McRobbie, 1996) and its Chinese version was modified from the Taiwanese Chinese version of the WIHIC questionnaire (Huang & Fraser, 1997). The development process involved a specifically designed 5-stage translation procedure that involved 29 people spread over 7 different focus groups (Chua, Wong & Chen, 2000). The focus group technique used was the application of the 4-step Nominal Focus Group technique (Moore, 1987; Stewart & Shamdasani, 1990). The translation procedure was repeated twice before the CLCEI was ready for validation with the Singaporean sample.

The objectives of this paper are: (1) to describe the validation procedures of the CLCEI; (2) to report the validation results obtained from various statistical procedures; and (3) to compare the validation results with those of the original WIHIC questionnaire (Fraser, Fisher & McRobbie, 1996).

Methodology

Sample

Instrument: The CLCEI questionnaire

The CLCEI, like the WIHIC questionnaire, is presented in 4 different forms. In the WIHIC questionnaire, the four forms are named personal-actual, personal-preferred, class-actual and class-preferred forms. For easy identification of the respondents of each form, the four forms of the CLCEI were renamed as student-actual, student-preferred, teacher-actual and teacher-preferred forms respectively. The student-actual and student-preferred forms were used to assess students' perceptions toward their actual and preferred Chinese language classroom learning environment, whereas the teacher-actual and teacher-preferred forms were used to assess teachers' perceptions toward their actual and preferred Chinese language classroom learning environment. Table 2 gives an example of an item in its four different forms.

Procedures of data collection

The researcher administered the CLCEI to 1460 students personally. The period for data collection was in the third term of the school year (July-September). In order to ensure a smooth data collection process, the preparation for data collection was carried out comprehensively. This included, for example, writing of verbatim instructions for administering the questionnaires to ensure uniformity of instructions given to all the respondents across the 25 participating schools and seeking the approval from the Ministry of Education (MOE), Singapore and the principals of the 25 selected secondary schools etc. During data collection, Mandarin was used as the language of communication for this survey. The researcher explained the purpose and the method of the survey using the prepared verbatim instructions. At the end of the survey, when students submitted the completed forms, the researcher and the Chinese language teachers performed a manual check of all the forms. Students were asked to fill in any missing entries, as necessary. This was to reduce the number of missing entries. At the end of the data collection, all data captured forms were scanned by using the Optical Mark Reader (OMR) in the sequence of the school codes. Raw data was captured in 25 computer files which were later imported into the SPSS statistical software to produce workable SPSS formatted data files for data analysis.

Procedures of statistical analysis

The student-actual and student-preferred forms of the CLCEI were validated using the student sample stated above. The data was then stored in various SPSS files to perform all the statistical analyses on the internal consistency reliability, discriminant validity, the scales' ability to differentiate between students' perceptions in different classrooms and the factorial validity of the CLCEI.

The Cronbach alpha reliability coefficient was calculated for each of the six scales as a measure of internal consistency reliability for each scale, using two units of analysis (the individual and the class mean). In addition, the discriminant validity for each CLCEI scale was also obtained by computing the mean correlation of that scale with the other five scales of the CLCEI using the individual student as the unit of analysis.

The ability of the CLCEI scales (student-actual) to differentiate between perceptions of students in different classes was also examined using a series of one-way analyses of variance (ANOVA). For this ANOVA, class membership was used as the independent variable and the set of the six scales of the CLCEI in the student-actual form was used as the dependent variable. The results of this analysis were presented in terms of the eta² statistic that is the ratio of between to total sums of squares.

Furthermore, principal components factor analysis with varimax rotation was conducted using the perception scores obtained from the student-actual form of the CLCEI (ie, analysed based on students' perceptions of the actual classroom learning environments). The individual student's score was used as the unit of analysis. The purpose of this factor analysis was to test whether the 48 items of the CLCEI would load into six a priori scales for assessing six different dimensions of the Chinese language classroom learning environment. Only items with factor loadings greater than 0.40 were considered in deciding the factor structure of each scale of the CLCEI. The factor structure that evolved was then compared with that reported previously for the original WIHIC questionnaire, to examine the extent to which it was replicated. The validation results of the CLCEI are reported and discussed in the next section.

Results and discussion

Environment Scale	Unit of Analysis	Alpha Reliability (alpha)		Mean Correlation with Other Scales		ANOVA Results (eta2)
		Actual	Preferred	Actual	Preferred	Actual	Preferred
Student Cohesiveness	Individual student	.90	.90	.44	.68	.13**	.13**
	Class Mean	.94	.96
Teacher Support	Individual student	.82	.93	.52	.60	.12**	.10**
	Class Mean	.94	.96
Involvement	Individual student	.87	.91	.51	.59	.08**	.09**
	Class Mean	.87	.96
Task Orientation	Individual student	.89	.93	.45	.58	.13**	.12**
	Class Mean	.96	.97
Cooperation	Individual student	.88	.93	.51	.56	.11**	.11**
	Class Mean	.94	.97
Equity	Individual student	.91	.93	.50	.63	.11**	.10**
	Class Mean	.96	.97
**p <.001      Sample Size: 1460 students from 50 Chinese language classes

The eta² statistic (which is the ratio of between to total sums of squares) represents the proportion of variance explained by class membership.

The statistics of the Cronbach alpha reliability coefficient and the mean correlation of each scale with the other five scales were generated as indices of scale internal consistency reliability and discriminant validity respectively.

Results in Table 3 shows that the internal consistency of each of the six scales of the CLCEI for both the actual form and the preferred form was highly satisfactory. For the student-actual form of the CLCEI, the Cronbach alpha coefficients ranged from .82 to .91 when the individual student's score was used as the unit of analysis and from .87 to .96 when the class mean was used as the unit of analysis. For the student-preferred form of the CLCEI, the Cronbach alpha coefficients ranged from .90 to .93 when the individual student's score was used as the unit of analysis and from .96 to .97 when the class mean was used as the unit of analysis.

The results also indicated that when the class mean was used as the unit of analysis, the values of the Cronbach alpha were higher than when the individual student's score was used as the unit of analysis for all the six scales for both the actual and preferred forms. In general, the internal consistency of the CLCEI in student-actual form was found to be higher than for the WIHIC questionnaire. When the original WIHIC (Fraser, Fisher & McRobbie, 1996) was validated, the Cronbach alpha coefficients ranged from .77 to .89 when the individual student's score was used as the unit of analysis and from .67 to .88 when the class mean was used as the unit of analysis.

The discriminant validity is described as the extent to which a scale measures a unique dimension not covered by the other scales of the instrument. Table 3 indicates that the mean correlation of a scale with the other five scales ranged from .44 to .52 for the student-actual form of the CLCEI and from .56 to .68 for the student-preferred form when using the individual student's score as the unit of analysis. These results were comparable to the validation results when the original WIHIC was validated using an Australian sample (Fraser, Fisher & McRobbie, 1996). The reported mean correlations for the Australian study for five out of the six scales namely, Student Cohesiveness, Teacher Support, Involvement, Task Orientation, and Cooperation were in the range of .42 to .45, whilst the Equity scale had an extremely low value of .06 for its mean correlation with the other five scales. Whereas, when the CLCEI was validated using the Singaporean sample, the mean correlation of the Equity scale with the other five scales was .50. The higher value for the Equity scale of the CLCEI could be related to cultural factors. For instance, when the Equity scale of the original WIHIC was validated using the Taiwanese sample, its mean correlation with the other five scales was .59 (Huang & Fraser, 1997) whereas, it was .27 when validated using an Indonesian sample (Margianti & Fraser, 2000). These results indicated that the way Equity is interpreted seems to vary depending on the cultural context in which the study is undertaken. A more accurate interpretation of such a finding would require further exploration in future studies.

In general, the overall results showed that the six scales of the CLCEI have adequate discriminant validity although the scales assess somewhat overlapping aspects of the Chinese language classroom learning environment.

The ability of the CLCEI to differentiate between the perceptions of students in different classes was investigated using a one-way ANOVA. The findings of the one-way ANOVA showed that each scale of the CLCEI in the student form was able to differentiate significantly between the perceptions of students from 50 different Chinese language classrooms (p < .001). The eta² statistic, which represents the proportion of variance accounted for by class membership, ranged from .08 to .13 for the student-actual form and ranged from .09 to .13 for the student-preferred form, when the individual student's score was used as the unit of analysis.

The 1460 students' responses to the 48 items of the student-actual form and 48 items of the student-preferred form of the CLCEI were subjected to separate principal components factor analyses (with varimax rotation) involving the individual student's score as the unit of analysis. The factor loadings for all the six scales are presented in Table 4, in which factor loadings with values less than 0.40 were omitted. The factor structure of the CLCEI which evolved showed that all 48 items of the student-actual form and all 48 items of the student-preferred form loaded neatly into their six a priori scales with all items having factor loadings greater than 0.40 on their respective scale. The factor structure for the six scales of the CLCEI was similar to that of the 56-item version of the WIHIC questionnaire (Chionh & Fraser, 1998).

Table 4 shows that item 8, which reads "In this Chinese language class, I get help from other students" loaded into two scales, the Student Cohesiveness scale and the Task Orientation scale with equal factor loadings of 0.40 each, whereas in the original WIHIC questionnaire, item 8 was an item in the Student Cohesiveness scale. In addition, item 23, which reads "Students discuss with me how to go about solving problems" in the Involvement scale also loaded in the Cooperation scale with factor loadings .46 and .42 for the actual form and preferred form respectively.

In summary, besides the double-loading pattern for the items 8 and 23 on two different scales explained above, all the other items loaded neatly into their six original a priori scales, and hence, the results lend support to the factorial validity of the CLCEI.

Conclusion

Item No	Student Cohesiveness		Teacher Support		Involvement		Task Orientation		Cooperation		Equity
	Actual	Preferred	Actual	Preferred	Actual	Preferred	Actual	Preferred	Actual	Preferred	Actual	Preferred
1.	.67	.77
2.	.67	.79
3.	.69	.77
4.	.74	.77
5.	.69	.71
6.	---	.42
7.	.63	.58
8.	.40	.49					.40
9.			.68	.71
10.			.74	.71
11.			.74	.67
12.			.69	.65
13.			.67	.68
14.			.71	.72
15.			.61	.70
16.			.47	.49
17.					.68	.72
18.					.77	.74
19.					.47	.64
20.					.72	.69
21.					.61	.63
22.					.66	.72
23.					.49	.51			.46	.42
24.					.60	.65
25.							.67	.72
26.							.74	.74
27.							.70	.74
28.							.69	.68
29.							.75	.73
30.							.69	.69
31.							.70	.69
32.							.66	.67
33.									.62	.65
34.									.60	.66
35.									.64	.67
36.									.68	.69
37.									.67	.67
38.									.73	.71
39.									.65	.66
40.									.56	.61
41.											.62	.66
42.											.70	.68
43.											.73	.73
44.											.73	.71
45.											.75	.71
46.											.73	.71
47.											.59	.60
48.											.70	.60
% of variance	7.78	10.37	9.56	10.65	9.01	10.70	5.10	11.87	9.60	11.48	10.16	11.87
The amount of variance was 56.72% (actual form) and 66.13% (preferred form)

The results of the principal components factor analyses (with varimax rotation) showed clearly that the 48 items of the CLCEI loaded neatly into the six scales which replicated the factor structure of the original WIHIC questionnaire. The loading pattern is appropriate and therefore, the factorial validity of the CLCEI is comparable to that of the original WIHIC questionnaire.

In conclusion, the validation results obtained and the factor structure that emerged showed that the CLCEI is a reliable bilingual instrument that is ready for use for assessing the nature of the Chinese language classroom learning environment in Singapore secondary schools. In addition, the validation results indicated that the adaptation of the six scales from the original WIHIC questionnaire and the omission of the Investigation scale in the WIHIC questionnaire for developing the CLCEI were appropriately carried out (Chua, Chen & Wong, 2000). It also further supported the claim made by the developers of the WIHIC questionnaire that the WIHIC allowed the exclusion of irrelevant scale(s) to suit any classroom environment under study without affecting its reliability and validity (Fraser, Fisher & McRobbie, 1996). Thus the CLCEI contributes to the field of classroom environment as the first English-Chinese bilingual instrument for use in Chinese language classrooms.

References

Chua, S. L., Wong, F. L., & Chen, D. Q. (2000). The development of a variation to the Chinese "What is happening in this class" (WIHIC) for use in cross-cultural studies. Paper presented at the Second Conference on Science, Mathematics and Technology Education, Taiwan, January 2000.

Fisher, D. L. & Fraser, B.J. (1981). Validity and use of my class inventory. Science Education, 65, 145-156.

Fisher, D. L., & Fraser, B. J. (1983). Use of work environment scale (WES) to assess science teachers' perceptions of school environment. European Journal of Science Education, 5, 231-233.

Fraser, B. J. (1986). Classroom environment. London: Croom Helm.

Fraser, B. J. (1990). Individualised classroom environment questionnaire. Australian Council for Educational Research, Melbourne, Australia.

Fraser, B. J. (1998). Science learning environments: Assessment, effects and determinants. International Handbook of Science Education, 527-564.

Fraser, B. J. (2000). Improving research on learning environments through international cooperation. Keynote address presented at the Second Conference on Science, Mathematics and Technology Education, Taiwan, January 2000.

Fraser, B. J., Anderson, G. J. & Walberg, H. J. (1982). Assessment of learning environments: Manual for learning environment inventory (LEI) and my class inventory (MCI) (3rd Ed.). Bentley, WA: Western Australian Institute of Technology.

Fraser, B. J., Fisher, D. L. & McRobbie, C. J. (1996). Development, validation and use of personal and class forms of a new classroom environment instrument. Paper presented at the annual meeting of the American Educational Research Association, New York.

Fraser, B. J., Giddings, G. J. & McRobbie, C. J. (1993). Development and cross-national validation of a laboratory classroom environment instrument for senior high school science. Science Education, 77, 1-24.

Fraser, B. J. & O'Brien, P. (1985). Student and teacher perceptions of the environment of elementary school classrooms. Elementary School Journal, 85, 567-580.

Fraser, B. J., Treagust, D. F. & Dennis, N. C. (1986). Development of an instrument for assessing classroom psychological environment at universities and colleges. Studies in Higher Education, 11, 43-54.

Fraser, B. J. & Walberg, H. J. (1991). Educational environments: Evaluation, antecedents and consequences. Pergamon, London.

Goh, S. C., Young, D. J. & Fraser, B. J. (1995). Psychosocial climate and student outcomes in elementary mathematics classrooms: A multilevel analysis. Journal of Experimental Education, 64, 29-40.

Haertel, G. D., Walberg, H. J. & Haertel, E. H. (1981). Social-psychological environments and leatning: A quantitive synthesis. British Educational Research Journal, 7(1), 27-36.

Huang, I. T. C. & Fraser, B. J. (1997). The development of a questionnaire for assessing student perception of teaching environments in Taiwan and Australia. Paper presented at the annual meeting of the National Association for Research in Science Teaching, Chicago.

Kim, H. B., Fisher, D. L., & Fraser, B. J. (2000). Classroom environment and teacher interpersonal behavior in secondary school classes in Korea. Evaluation and Research in Education, 14, 3-22.

Margianti, E. & Fraser, D.L. (2000). Learning environment, mathematical ability and students' outcomes in university computing courses in Indonesia. Paper presented at the Second Conference on Science, Mathematics and Technology Education, Taiwan.

Moore, C. M. (1987). Group techniques for idea building. Newbury Park, CA: Sage.

Moos, R. H. & Trickett, E. J. (1987). Classroom environment scale manual (rev. ed.). Consulting Psychologists Press, Palo Alto, CA

Rentoul, A. J. & Fraser, B. J. (1979). Conceptualization of enquiry-based or open classroom learning environments. Journal of Curriculum Studies, 11, 233-245.

Stewart, D. W. & Shamdasani, P. N. (1990). Focus groups: Theory and practice (3rd ed). Newbury Park, CA: Sage.

Taylor, P. C., Fraser, B. J. & Fisher, D. L. (1997). Monitoring constructivist classroom learning environments. International Journal of Educational Research, 27, 293-302.

Teh, G., & Fraser, B. J. (1995). Development and validation of an instrument for assessing the psychosocial environment of computer-assisted learning classrooms. Journal of Educational Computing Research, 12, 177-193.

Walberg, H. J. & Anderson, G.J. (1968). Classroom climate and individual learning. American Educational Research Journal, 5, 163-169.

Wong, F. L. & Fraser, B.J. (1995). Cross-validation in Singapore of the science laboratory environment inventory. Psychological Reports, 76, 907-911.

Wubbels, T. & Levy, J. (Eds.). (1993). Do you know what you look like: Interpersonal relationships in education. Falmer Press, London.

Authors: Dr Chua Siew Lian is a Computer Science Lecturer at St. Andrew's Junior College, Singapore. Email: tee_siew_lian@moe.edu.sg Dr Angela F. L. Wong is an associate professor and Associate Dean at the National Institute of Education, Nanyang Technological University, Singapore. Email: angela.wong@nie.edu.sg Dr Der-Thanq "Victor" Chen. Dr Chen is an associate professor with the National Institute of Education, Nanyang Technological University, Singapore. Email: victor.chen@nie.edu.sg Please cite as: Chua S. L., Wong, A. F. L. & Chen, V. D.-T. (2006). Validation of the 'Chinese language classroom learning environment inventory' for investigating the nature of Chinese language classrooms. Issues In Educational Research, 16(2), 139-151. http://www.iier.org.au/iier16/chua.html