QJER logo 2
[ Contents Vol 16, 2000 ] [ QJER Home ]

Validation and use of a short form of the university-level environment questionnaire

Jeffrey P. Dorman
This article reports research on the institutional environment perceived by Australian academics. Using a sample of 489 academics from 18 universities, a short form of the University-Level Environment Questionnaire was validated. Exploratory factor, confirmatory factor and item analyses indicated that the seven scales of this instrument (Academic Freedom, Concern for Undergraduate Learning, Concern for Research and Scholarship, Empowerment, Affiliation, Mission Consensus and Work Pressure) function reliably. Comparisons of environment in different types of universities and departments revealed some differences, especially between the long-established universities and universities established in the past decade.


Higher education in Australia has undergone major changes during the past decade with dramatic changes in the size, structure, funding arrangements and focus of higher education institutions (Department of Education, Training and Youth Affairs: DETYA, 1998b). The binary system which separated established universities from institutes of technology and colleges of advanced education was abolished in 1987 in favour of a Unified National System (UNS). By 1994, amalgamations and mergers had resulted in 36 publicly-funded universities in the UNS. During the past decade, the Australian tertiary sector has experienced extraordinarily high growth with total student numbers increasing from 394,000 in 1987 to 659,000 in 1997 (DETYA). Universities have been corporatised and compete with each other for students in a climate of decreased public funding for universities. That students are viewed as clients is firmly embedded in the language and culture of contemporary Australian tertiary education.

One important aspect of any educational institution is the quality of the psychosocial environment - the atmosphere, climate or ambience that pervades the institution. Few recent research studies have investigated the overall or institutional environment of colleges and universities. The present article reports the validation and use of a short form of an instrument called the University-Level Environment Questionnaire (ULEQ) which was developed by Dorman (1998) to assess institutional environment in colleges and universities. Typically, short forms of instruments have fewer items compared to the original long form. As both respondents and researchers face work intensification, short forms of instruments are becoming very popular research tools. The usefulness of the ULEQ (Short Form) for comparing environment in universities is illustrated in a later section of this article.

The present research builds upon and extends international research conducted over the past 30 years in the learning environment field. These studies, conducted mainly in primary and secondary schools in the USA and Australia, have provided strong and convincing evidence that the quality of the learning environment is an important determinant of student learning (e.g., Fraser, 1994, 1998a). According to Anderson (1982), studies in secondary schools have established positive links between school-level environment and student cognitive and affective outcomes (Barker, 1963; Brookover et al., 1978; Ellett & Walberg, 1979), student values (Vyskocil & Goens, 1979) and student personal growth and satisfaction (Bailey, 1979; Coyne, 1975). A longitudinal study involving a sample of London schools established strong relationships between school processes and student achievement, attendance and behaviour (Rutter, Maughan, Mortimore, Ouston, & Smith, 1979). Other studies in schools have found that school environment influences classroom environment (Fisher, Fraser & Wubbels, 1993; Fraser & Rentoul, 1982). A very large body of research evidence has shown that classroom environment is a potent predictor of student cognitive and affective outcomes in primary and secondary schools (Fraser, 1998b).

Assessments of university-level environments should be an integral part of quality assurance mechanisms. In its review of teaching in British universities, the Polytechnics and Colleges Funding Council (1990) concluded that the ethos of the whole institution largely determines the student's experience and that this notion of institutional ethos is an important contributing factor to quality in higher education institutions. In Australia, the Higher Education Council (1992) suggested that the learning environment is likely to have an important effect on how students learn and that many aspects of this environment can be influenced by government and institutional action to the benefit of educational quality. However, quite apart from the view that positive environments are associated with enhanced student learning, it needs to be accepted that high-quality environments in universities are intrinsically desirable goals. The present article focuses attention on the quality of the institutional-level environment as an antecedent to outcomes of universities.

While some university studies have investigated classroom environment in small group settings (e.g. tutorials, workshops, and science laboratories), few research studies have investigated overall or institutional-level environment (Fraser, 1998a, 1998b; Fraser & Treagust, 1986, Yarrow, Millwater & Fraser, 1997). The College and University Classroom Environment Inventory was developed by Fraser and Treagust to assess seven psychosocial dimensions of small groups: Personalisation; Involvement; Student Cohesiveness; Satisfaction; Task Orientation; Innovation; and Individualisation. Cross-national environment research in university science laboratory classes using the Science Laboratory Environment Inventory was conducted in the early 1990s (Fraser, Giddings & McRobbie, 1992). A number of existing classroom environment scales used in secondary schools could be modified for use in university classrooms, but a detailed review of these instruments is outside the scope of this article (see Fraser, 1998a).

Thistlethwaite (1962, 1965) conducted one of the few university environment-outcome studies and concluded that students' aspirations and dispositions to seek advanced training were positively related to environmental factors like intellectualism and positive evaluations by academics but negatively related to compliance and expectations of conformity and status. Genn's (1975) research in one Australian university linked satisfaction among 137 students with department environment as assessed by a modified form of Pace and Stern's (1958) College Characteristics Index. Using an eight-scale course perceptions questionnaire, Ramsden (1979) investigated links between student perceptions of department environment and their approaches to learning in a British university. Independent research conducted by Gaff, Crombag & Chang (1976) and van Rooijen (1986) in The Netherlands focused on the importance of department environment in European universities. The study by van Rooijen employed a 58-item instrument called the University Learning Environment Evaluation (ULEE) to assess students' perceptions of their department's learning environment and instructional processes.

In a departure from much prior research in Europe and the USA, Dippelhofer-Stiem (1986) conceptualised a model with five hierarchically-ordered levels (national, university, subject, courses, and the individual person). This longitudinal study in five universities in the Federal Republic of Germany involved the assessment of students' perceptions of these five levels of environment and students' evaluations of these environments. Recently, Clarke (1995, 1998) used a semi-structured, open-ended instrument called the Perceptions of Learning Environments Questionnaire to gather student perceptual data on activities and behaviours in one Queensland university. Clarke (1998) concluded that different learning environments produced different learning outcomes and that students are capable of discriminating between what they consider to be good and bad educational experiences.

Previous studies (e.g., Gaff, Crombag & Chang, 1976) highlight conceptual differences between Europe and the United States in the study of university environments. Whereas American studies have focused on the overall or institutional environment with instruments often tapping academics' perceptions, most European studies have assessed students' perceptions of the faculty or department environment. It can be argued legitimately that studies of department environment which do not have institutional-level dimensions are not assessing university-level environment per se.

The present research focused on the university-level environment, a field that was pioneered in the USA in the 1950s and 1960s by Pace and his colleagues (Pace, 1963, 1969; Pace & Stern, 1958; Stern, 1970). The College Characteristics Index, an instrument that assesses 30 environment dimensions of universities and colleges, was developed in the 1950s, and refined to form the College and University Environment Scales (CUES) in the 1960s. The final version of the CUES has 160 items that assess seven dimensions of environment considered important to universities in the 1960s (Practicality, Community, Awareness, Propriety, Scholarship, Campus Morale and the Quality of Teaching and Faculty-Student Relationships). Warren and Rees (1975) used modified versions of the CUES in research that compared the environment in a college of advanced education and a neighbouring university.

A second instrument developed in the late 1960s is the Institutional Functioning Inventory (IFI: Peterson, Centra, Hartnett & Linn, 1983). This 132-item instrument has 11 scales: Intellectual-Aesthetic Curriculum, Freedom, Human Diversity, Concern for Improvement in Society, Concern for Undergraduate Learning, Democratic Governance, Meeting Local Needs, Self-Study and Planning, Concern for Advancing Knowledge, Concern for Innovation, and Institutional Esprit. Both the CUES and IFI were important developments in the study of university learning environments and the instrument used in the present study draws upon the work of Pace and Peterson and others. Both the CUES and IFI lack scoring economy in that they possess a large number of items. Additionally, scales have high levels of overlap which confound interpretation of results.

To rectify the above weaknesses and modernise instrumentation in this area, Dorman (1998) developed the University-Level Environment Questionnaire (ULEQ) which consists of 42 items assigned to seven scales (Academic Freedom, Concern for Undergraduate Learning, Concern for Research and Scholarship, Empowerment, Affiliation, Mission Consensus and Work Pressure). Validation data reported by Dorman (1998) attest to the ULEQ's sound psychometric structure for use with Australian academics. However, as indicated earlier, a trend in psychosocial environment research is the development of short forms of instruments that will reduce response and scoring time. Accordingly, the present research was geared to validating and using a short form of the ULEQ. This article also reports the use of confirmatory factor analysis - a highly desirable, yet rarely employed technique for the validation of learning environment instruments.

DESIGN OF PRESENT STUDY

Aims of the Research

The research reported in this article had four aims:

Sample

The present study involved a sample of 489 academics drawn from 28 of the 36 government-funded Australian universities. Australia has two private universities and the university sample drawn for this study reflected the diversity in the population of Australian universities (see table 1). Participants included academics from three types of universities: long-established universities with a strong research tradition (Group 1), comprehensive universities and former institutes of technology which have a well-developed research focus (Group 2), and former colleges of education which have a tradition of undergraduate teaching (Group 3). In each university, 20 academics were drawn randomly from three types of departments that reflect the diversity within universities (viz. Education, Biology and English). The sample consisted of 270 male and 219 female academics in 52 departments. The overall response rate was 87 percent.

Table 1: Description of sample by university type and department type

University TypeDepartmentTotal
EducationBiologyEnglish
1. Long established traditional universities51 (6)50 (5)58 (6)159 (17)
2. Comprehensive universities and former
institutes of technology
58 (6)74 (8)45 (5)177 (19)
3. Former colleges of advanced education44 (4)63 (7)46 (5)153 (16)
Total153 (16)187 (20)149 (16)489 (52)
Note: The number of departments is given in parentheses.

Genesis of the University-Level Environment Questionnaire (Short Form)

The short form of the ULEQ was developed from the University-Level Environment Questionnaire (ULEQ: Dorman, 1998). That instrument consists of 42 items with six items allocated to each of seven scales. Academics respond to each ULEQ item using a five-point Likert response format (Strongly Agree = 5, Agree = 4, Not Sure = 3, Disagree = 2, Strongly Disagree = 1). Twelve negatively-worded items are reverse scored. Scale scores are obtained by aggregating the scores for the six items of each scale. For the short form of the ULEQ, 28 tentative items (that is, four items per scale) were identified using psychometric information from the development of the original ULEQ. Table 2 provides a common sense description and a typical item for each scale of the ULEQ (Short Form).

According to Moos (1974), an important characteristic of psychosocial environment instruments is that they assess three general categories: Personal Development (the basic directions along which personal growth and self-enhancement tend to occur), Relationship (the extent to which people are involved in the environment and support and help each other), and System Maintenance and System Change (the extent to which the environment is orderly, clear in expectations, maintains control and is responsive to change). As shown in table 2, the ULEQ (Short Form) has three Personal Development, two Relationship and two System Maintenance and System Change scales.

Table 2: Descriptive information for scales of the university-level environment scales

ScaleScale Description:
The extent to which ...
Typical ItemMoos's
Schema
1. Academic Freedomstaff and students have academic freedom.Staff and students may discuss any topic. (+)P
2. Concern for Undergraduate Learninguniversity processes and teaching approaches emphasise a concern for undergraduate learning.Staff members are sensitive to the interests, needs and aspirations of undergraduates. (+)P
3. Concern for Research and Scholarshipthe university emphasises research and scholarship.Senior academics do not emphasise research as an important institutional purpose. (-)P
4. Empowermentacademics are empowered and encouraged to be involved in decision making.My superiors deal with me in an authoritarian manner. (-)R
5. Affiliationacademics can obtain assistance, advice and encouragement and are made to feel accepted by colleagues.I can rely on my colleagues for assistance if I need it. (+)R
6. Mission Consensusconsensus exists within the staff with regard to the overarching goals of the university.Lecturers agree on the university's overall goals. (+)S
7. Work Pressurework pressure dominates the environment.There is constant pressure on academics to keep working. (+)S
P: Personal Development; R: Relationship; S: System Maintenance and System Change

VALIDATION OF ULEQ (SHORT FORM)

To validate the ULEQ (Short Form), several scale development techniques were used. Initially, estimates of scale internal consistency were calculated using Cronbach's Coefficient Alpha as an index. Second, item-scale correlations were reviewed to establish which items from the long form of the ULEQ could be removed without reducing appreciably that scale's internal reliability. Third, scale-scale correlations were calculated to determine overlap among the scales (that is, discriminant validity). Fourth, exploratory factor analyses were conducted on the data. Finally, the overall structure of the ULEQ (Short Form) was validated using confirmatory factor analysis.

Scale Internal Consistency and Item-Scale Correlations

The internal consistency (Cronbach's Coefficient Alpha) of each of the tentative 7 four-item scales are shown in table 3. These coefficients ranged from .71 for the Concern for Undergraduate Learning and Academic Freedom scales to .85 for the Affiliation scale. Item-scale correlations confirmed that all items had been assigned to the appropriate scale and that each item made an appreciable contribution to that scale's internal consistency.

Discriminant Validity

Table 3 also reports discriminant validity data using the mean correlation of a scale with the remaining six scales as a convenient index. These data indicate that the scales do overlap but not to an extent that would violate the psychometric properties of the ULEQ (Short Form). Additionally, the conceptual distinctiveness of the scales justifies their inclusion in the instrument.

Exploratory Factor Analysis

Exploratory factor analysis using varimax rotation revealed seven factors which accounted for 62 percent of variance in item scores. All items had loadings in excess of .60 with their a priori scale (M = .71, SD = .06) and loadings below .28 on all other scales. With a case/item ratio in excess of 17:1, a high degree of confidence can be placed in the instrument's psychometric structure.

Table 3: Internal consistency (coefficient alpha), discriminant validity (mean correlation), scale means and scale standard deviations for the short form of the University-Level Environment Questionnaire
(n = 489 academic staff in 52 university departments)

ScaleCoeff.
Alpha
Mean
Correlation
MeanStandard
Deviation
1.Academic Freedom.71.3214.522.90
2.Concern for Undergrad. Learning.71.2114.302.81
3.Concern for Research and Sch'ship.73.3014.453.48
4.Empowerment.79.3714.423.63
5.Affiliation.85.3715.433.02
6.Mission Consensus.76.2911.393.10
7.Work Pressure.78.0417.042.69

Confirmatory Factor Analysis

Confirmatory factor analysis is a well-known data analysis technique that investigates the multi-dimensionality of measurement models (Byrne, 1998). It is one important application of the broad data analysis field of structural equation modelling. Briefly, confirmatory factor analysis requires the researcher to postulate a model between observed variables and their underlying constructs (or latent variables) and tests this model statistically for goodness of fit. In the present study, 28 environment items were observed variables for seven first order latent variables (the ULEQ scales). Furthermore, it was hypothesised that variances in these seven latent variables were due to a second order latent variable, University-Level Environment (see figure 1).

To examine this model, indices of model fit, model comparison and model parsimony were calculated. Of the approximately 40 possible goodness of fit indices now available in structural equation modelling, seven salient indices have been calculated: Root Mean Square Error of Approximation (RMSEA), Goodness of Fit Index (GFI), Adjusted Goodness of Fit Index (AGFI), Tucker-Lewis Index (TLI), Relative Noncentrality Index (RNI), Parsimony Goodness of Fit Index (PGFI), and Parsimony Normed Fit Index (PNFI) (Byrne, 1998). To interpret these indices, the following rules of interpretation which are accepted in Structural Equation Modelling literature were employed. RMSEA values should be as small as possible with perfect fit indicated by an index of zero. Values less than .08 indicate at least sound fit while values between .08 and .1 reflect mediocre fit (Byrne, 1998). GFI, AGFI, TLI and RNI range between 0 and 1 with values above .9 indicating good model fit. While PGFI and PNFI values range from 0 to 1, values close to 1 (perfect fit) are not expected. Moreover, indices around .5 are not unexpected in sound-fitting models. The relative strength of each path can be assessed by the standardised regression (or path) coefficient for that path with a t-test conducted to establish the statistical significance of the path.

Results of confirmatory factor analysis for the short form of the ULEQ reveal a sound model fit with RMSEA of .07 and GFI of .96. Additionally model comparison is very good with TLI and RNI values of .93 and .94 respectively. Finally, PGFI (.81) and PNFI (.86) indicate good model parsimony. Overall, these indices provide very strong support for the postulated measurement model for the short form of the ULEQ. Figure 1 shows standardised path coefficients, all of which were statistically significant (p<.001) and almost all quite strong (only the University Environment-Work Pressure path being weak).

Editorial note: Figure 1 was not recorded in the file provided for the web archives.
During 2005 a replacement will be obtained from hard copy versions.

Figure 1: Measurement model for university-level environment

USE OF SHORT FORM OF THE ULEQ

Having established the validity of the short form of the ULEQ, comparisons of environment according to university type, department type and gender of academic were conducted to illustrate the utility of this instrument. Data used to validate this instrument were used to calculate scale scores for each respondent. Accordingly the final data set consisted of 489 scale scores for each of the seven ULEQ (Short Form) scales. Multivariate analysis of variance (MANOVA) was used to establish whether statistically significant differences were evident in population means based on the samples collected in this study (see means in table 4).

In line with the recommendation of Keselman et al. (1998), effect sizes have been calculated so that the reader can assess the practical significance of statistically significant differences (table 5). Cohen's d, the difference in group means divided by the full sample standard deviation, was used as a convenient effect size index (Cohen, 1977). According to Cohen (1977, pp. 24-27), small, medium and large effect sizes are indicated by d values of 0.2, 0.5 and 0.8 respectively. As pointed out by Cohen, the interpretation of effect size is somewhat dependent on the field of research. These d values are consistent with the interpretation of effect size reported in psychosocial environment research during the past 20 years (Dorman, 1999.

Differences Between Academics' Perceptions of Environments in Group 1, Group 2 and Group 3 Universities

To compare environments in the three types of universities identified in table 1 (Groups 1, 2 and 3), a three-way MANOVA with the set of seven environment scales as dependent variables and university type, department type and academic's gender as independent variables was performed. A three-way MANOVA was needed to check for interaction effects. As there were no interaction effects (p<.05), a MANOVA investigating the effect of university type was conducted. This test was significant (p<.001). Univariate F tests for each university environment scale revealed significant differences between university types on five scales: Academic Freedom [F(2, 471) = 25.47 (p <.001)], Concern for Research and Scholarship [F (2, 471) = 89.40 (p <.001)], Empowerment [F (2, 471) = 15.24 (p <.001)], Mission Consensus [F (2, 471) = 5.56 (p <.001)], and Work Pressure [F (2, 471) = 6.38 (p <.001)].

Because these tests involved three types of universities, Tukey's post-hoc procedure was employed to establish those pairs of university types where significant differences occurred. For each of the above five scales, significant differences between Group 1 and Group 3 universities and between Group 2 and Group 3 universities were found (p <.05) (see table 4). Additionally, significant differences were also found for comparisons of Group 1 and Group 2 universities for Concern for Research and Scholarship and Empowerment (p <.05).

Table 4: Mean Scores for Comparisons of University Group, Department and Gender
(N = 489 academic staff in 52 university departments)

ScaleUniversity GroupDepartmentGender
123 Educ.Biol.Engl.MaleFem.
115.6315.4113.51 14.4014.2115.0614.6914.35
213.7914.2014.59 14.3214.2614.4213.8614.99
317.0815.7712.45 14.7514.4214.3314.8214.03
416.0515.0613.28 14.6814.1614.5114.6314.20
515.7515.8915.00 15.7314.9615.6815.4015.53
611.9711.7610.92 11.7311.4411.0911.3011.51
717.6117.4316.54 16.7417.2817.0416.8717.27
For university groups (types) see table 1.
For scale names and descriptions see table 2.

Effect sizes (table 5) for pairwise comparisons of means ranged from 0.31 for the difference between Group 2 and Group 3 universities on the Work Pressure scale to 1.35 for the difference between Group 1 and Group 3 universities on the Concern for Research and Scholarship scale (M = .62, SD = .31). In general, these effect sizes should be interpreted as moderate to large (Cohen, 1977). The general conclusion is that Former Colleges of Advanced Education are significantly and substantially lower than the other two types of university on Academic Freedom, Concern for Research and Scholarship, Empowerment, Mission Consensus and Work Pressure.

Table 5: Effects sizes for pairwise comparisons

ScaleUniversity GroupsDepartmentsMale
vs Fem.
1 vs 22 vs 31 vs 3Educ
vs Biol
Educ
vs Eng
Biol
vs Eng
1.09.66.75 .04.23.27.14
2.05.23.28.01 .09.08.41
3.39.961.35.09 .15.06.31
4.34.43.77.13 .06.07.16
5.02.23.25.20 .01.19.02
6.18.24.42.15 .22.07.04
7.11.31.42.21 .07.14.13
For university groups (types) see table 1.
For scale names and descriptions see table 2.

Differences Between Academics' Perceptions of University Environments in Education, Biology and English Departments

The MANOVA comparison of department was significant (p<.05). Univariate F tests revealed two scales for which there were statistically significant differences: Academic Freedom [F (2, 471) = 3.74 (p <.05)] and Mission Consensus [F (2, 471) = 3.82 (p <.001)]. Tukey's post-hoc procedure indicated only one statistically significant difference between population means - English departments had significantly higher levels of Academic Freedom compared to Biology departments (see table 4), with an effect size of 0.27 (see table 5) which can be interpreted as small (Cohen, 1977). The fact that none of the differences were significant for Mission Consensus (p<.05) highlights the conservative nature of the Tukey test.

Differences Between Male and Female Academics' Perceptions of University Environment

The MANOVA comparison of genders was significant (p <.05). Univariate F tests revealed significant differences (p <.05) between gender scores for one scale only, Concern for Undergraduate Learning [F (1, 471) = 9.19 (p <.01)] (table 4). The associated effect size for this comparison was 0.41 (see table 5) which can be interpreted as moderate.

DISCUSSION

The comparison of university environment in three types of universities indicates that, in general, Group 1 and Group 2 universities have more positive environments than Group 3 universities. These differences are plausible. That Group 1 and 2 universities have higher levels of Concern for Research and Scholarship is consistent with characteristics and performance statistics in recent Australian Government reports (DETYA, 1998a, 1998b). For example, DETYA's (1998a) figures show that the eight Group 1 and 17 Group 2 universities received respectively 57.4 percent and 37.5 percent of the research quantum funding provided by the Government to support research in 1999. The 11 Group 3 universities shared the remaining 5.1 percent of research funds. There appears to be a clear link between Concern for Research and Scholarship and research funding.

The results of the present study are consistent with studies conducted by Streets (1972) and Warren and Rees (1975) in which the perceptions of teacher education students in a university were compared with the perceptions of teacher education students in a college of advanced education. These studies employed modified forms of the College and University Environment Scales (CUES) to show that, whereas university education departments emphasised Scholarship and Awareness, college education departments stressed Practicality and Community. The CUES Awareness scale is similar to the Academic Freedom scale of the ULEQ. As a number of colleges of advanced education with large teacher education faculties became Group 3 universities, it would appear that the differences in the learning environment identified by Warren and Rees (1975) still exist.

The present research also highlights the fact that re-badging colleges as universities does not automatically alter learning environments. While this reality has been self-evident to academics in universities, it was not apparent to the politicians who advocated revolutionary change a decade ago. Moreover, changes in environment are, to some extent, dependent on changes in cultural norms and beliefs which are more likely to occur on an evolutionary rather than a revolutionary basis. As Popkewitz (1983) rightly asserts, institutions are remarkably resilient to change. Although surface changes to practices may occur, underlying patterns of interpretive rules remain unchanged.

As indicated above, the present study found only small differences between academics' perceptions of the university learning environment in Education, Biology and English departments. As there are no recent studies of the effect of department type on institutional-level environment, it is difficult to assess the importance of the results of the present study. This problem is exacerbated by the conceptual differences between Europe and the USA on the study of university environments which have been explained earlier in this article. The present study took the American view that an overall or institutional-level environment exists and can be assessed. The ULEQ was not designed to assess department environments in line with the European approach to university environment research. While not unimportant, students' experiences of their faculty or department are more appropriately located in studies of faculty or department environment.

Two European studies provide evidence of differences in departmental environment. Ramsden (1979) used the Course Perceptions Questionnaire (CPQ) in one university in the United Kingdom to show that students in different departments experience different learning environments. In this case, the Department of Applied Science was considered to be very formal. While the Department of Social Science provided an informal and cooperative learning environment. Gaff, Crombag and Chang's (1976) study of one Dutch university concluded that different departments have different environments. The Chemistry department was pressure-packed with a heavy, prescribed nature. Whereas Medicine was memory-orientated and highly structured, Psychology had a 'free-wheeling' and independent atmosphere. In another study, van Rooijen (1986) used the University Learning Environment Evaluation (ULEE) in several departments but comparisons of those departments' environments were not reported. Taken together, the results of the present study and those reported by Ramsden (1979) and Gaff et al. (1976) suggest that the ULEQ assesses different dimensions from those assessed by these European studies. It would be desirable to establish whether the ULEQ is a useful tool for assessing university-level environment in European universities.

Previous research on the influence of gender on perceptions of university-level has not been reported. Analogous learning environment research conducted in secondary schools has concluded that females perceive environments more positively than males. Dorman, Fraser and McRobbie's (1994) study in coeducational classrooms in Australian secondary schools revealed that female students held more positive perceptions of their classrooms than did their male counterparts. This pattern of gender differences is similar to studies in secondary schools in the USA (Lawrenz, 1987), Singapore (Wong & Fraser, 1995) and Australia (Fraser, Giddings, & McRobbie, 1995). The evidence of the present study does not support fully the view that, compared to male academics, female academics hold more positive perceptions of the learning environment.

This study acknowledges that making firm conclusions on gender comparisons in tertiary education environments is problematic. Primarily this is because the appointment levels of females do not match the appointment levels of males. These differences are well-documented in Australian universities with DETYA (1998b) indicating that, while 36 percent of academics are females, they are disproportionately represented (Academic Level A: 51%, Academic Level B: 41%, Academic Levels D and E: 14%). Accordingly, females are mainly Level A and B academics whose roles emphasise teaching rather than research. Within this context, the finding that, compared to males, females have a greater Concern for Undergraduate Learning but a lower Concern for Research and Scholarship is plausible.

CONCLUSION

This article breaks new ground in learning environment research by reporting the validation and use of a short form of the University-Level Environment Questionnaire (ULEQ) - a seven-scale environment instrument that assesses Academic Freedom, Concern for Undergraduate Learning, Concern for Research and Scholarship, Empowerment, Affiliation, Mission Consensus and Work Pressure in universities. One important feature of this validation is the use of confirmatory factor analysis - a technique rarely employed in the validation of environment instruments. Three comparisons involving the ULEQ demonstrated its usefulness as a research tool for investigating university-level environments. The most important finding from this study is that very clear differences in university-level environment exist among groups of Australian universities. The results of the present study confirm the raft of DETYA (1998a, 1998b) indicators which make clear the disparity within the Australian tertiary sector.

Given the results of this study and the range of DETYA (1998a) statistics, it appears that the former Colleges of Advanced Education have made little progress towards adopting the culture of Australia's well-established, traditional universities. Despite the political imperatives of the late 1980s which suggested that change would be revolutionary, the resilience of institutions to cultural change (Popkewitz, 1983) has ensured that only incremental shifts in culture have occurred. Clearly a differentiated system of tertiary education continues to exist in Australia.

One possible direction for future research involving the ULEQ and its short form derivative would be to monitor the environments of Australian universities over an extended period of time. Such a longitudinal study would provide trend evidence for each university. Will some former Colleges of Advanced Education beat the almost insurmountable odds and get to Group 2 status? Will economic imperatives accelerate a move towards an American style tertiary sector with massive differentiation among its institutions?

Some caution is needed before generalising from the present findings. Results from a sample of Australian universities are not necessarily generalisable to other countries and cultures. Cross-national studies have not been conducted in this area and a desirable direction for future research would be to employ a sample of universities from several countries to validate the ULEQ for different settings and strengthen the generalisability of any findings.

REFERENCES

Anderson, C.S. (1982). The search for school climate: A review of the research. Review of Educational Research, 52, 368-420.

Bailey, M. (1979). The art of positive principalship. Momentum, 10(2), 46-47.

Barker, R.G. (1963). On the nature of the environment. Journal of Social Issues, 19(4), 17-38.

Brookover, W.B., Schweitzer, J.H., Schneider, J.M., Beady, C. H., Flood, P.K. & Wisenbaker, J.M. (1978). Elementary school social climate and school achievement. American Educational Research Journal, 15, 301-318.

Byrne, B. M. (1998). Structural equation modeling with LISREL, PRELIS, and SIMPLIS: Basic concepts, applications and programming. Mahwah, New Jersey: Lawrence Erlbaum.

Clarke, J.A. (1995). Tertiary students' perceptions of their learning environments: A new procedure and some outcomes. Higher Education Research and Development, 14, 1-12.

Clarke, J.A. (1998). Students' perceptions of different tertiary learning environments. Higher Education Research and Development, 17, 107-117.

Cohen, J. (1977). Statistical power analysis for the behavioral sciences (rev. ed.). New York: Academic.

Coyne, R. (1975). Environmental assessment: Mapping for counselor action. Personnel and Guidance Journal, 54, 150-155.

Department of Education, Training and Youth Affairs (DETYA) (1998a). Higher education report for the 1999 to 2001 triennium. Canberra: DETYA. [verified 20 Dec 2004] http://www.dest.gov.au/highered/otherpub/funding1998-2001.pdf

Department of Education, Training and Youth Affairs (DETYA) (1998b). The characteristics and performance of higher education institutions. Canberra: DETYA. [verified 20 Dec 2004] http://www.dest.gov.au/highered/otherpub/characteristics.pdf

Dippelhofer-Stiem, B. (1986). How to measure university environment? Methodological implications and some empirical findings. Higher Education, 15, 475-495.

Dorman, J.P. (1998). The development and validation of an instrument to assess institutional-level environment in universities. Learning Environments Research, 1, 333-352.

Dorman, J.P. (1999). The evolution, validation and use of a personal form of the Catholic School Classroom Environment Questionnaire. Catholic Education, 3, 141-157.

Dorman, J.P., Fraser, B.J. & McRobbie, C.J. (1994). Rhetoric and reality: A study of classroom environment in Catholic and Government secondary schools. In D. L. Fisher (Ed.), The Study of Learning Environments (Vol. 8). Perth: Curtin University of Technology.

Ellett, C.D. & Walberg, H.J. (1979). Principals' competency, environment, and outcomes. In H.J. Walberg (Ed.), Educational environments and effects: Evaluation, policy and productivity. Berkeley: McCutchan.

Fisher, D.L., Fraser, B.J. & Wubbels, T. (1993). Interpersonal teacher behavior and school environment. In T. Wubbels & J. Levy (Eds), Do you know what you look like? Interpersonal relationships in education. London: Falmer.

Fraser, B.J. (1994). Research on classroom and school climate. In D. Gabel (Ed.), Handbook of research on science teaching and learning. New York: Macmillan.

Fraser, B.J. (1998a). Science learning environments: Assessments, effects and determinants. In B.J. Fraser & K.G. Tobin (Eds), International handbook of science education. Dordrecht, The Netherlands: Kluwer.

Fraser, B.J. (1998b). Classroom environment instruments: Development, validity, and applications. Learning Environments Research, 1, 7-33.

Fraser, B.J., Giddings, G.J., & McRobbie, C.J. (1992). Assessment of the psychosocial environment of university science laboratory classrooms: A cross-national study. Higher Education, 24, 431-451.

Fraser, B.J., Giddings, G.J. & McRobbie, C.J. (1995). Evolution and validation of a personal form of an instrument for assessing science laboratory classroom environment. Journal of Research in Science Teaching, 32, 399-422.

Fraser, B.J. & Rentoul, A.J. (1982). Relationships between school-level and classroom-level environment. Alberta Journal of Educational Research, 28, 212-225.

Fraser, B.J. & Treagust, D.F. (1986). Validity and use of an instrument for assessing classroom psychosocial environment in higher education. Higher Education, 15, 37-57.

Gaff, J.G., Crombag, H.F.M., & Chang, T.M. (1976). Environments for learning in a Dutch university. Higher Education, 5, 285-299.

Genn, J. (1975). Students' perceptions of their actual and ideal Diploma in Education learning environments. The Australian University, 13(1), 64-81.

Higher Education Council (1992). Higher education: Achieving quality. Canberra: Australian Government Publishing Services.

Keselman, H.J., Huberty, C.J., Lix, L.M., Olejnik, S., Cribbie, R.A., Donahue, B., Kowalchuk, R.K., Lowman, L.L., Petoskey, M.D., Keselman, J.C. & Levin, J.R. (1998). Statistical practices of educational researchers; An analysis of their ANOVA, MANOVA, and ANCOVA analyses. Review of Educational Research, 68, 350-386.

Lawrenz, F.P. (1987). Gender effects for student perceptions of the classroom psychosocial environment. Journal of Research in Science Teaching, 24, 689-697.

Moos, R.H. (1974). Systems for the assessment and classification of human environments: An overview. In R.H. Moos & P.M. Insel (Eds), Issues in Social Ecology: Human Milieus. Palo Alto: National Press Books.

Pace, C.R. (1963). Differences in campus atmosphere. In W.W. Charters & N. Gage (Eds.), Readings in the social psychology of education. Boston: Allyn & Bacon.

Pace, C.R. (1969). College and university environment scales (3rd ed.). Princeton: Educational Testing Service.

Pace, C.R., & Stern, G.G. (1958). An approach to the measurement of the psychological characteristics of college environments. Journal of Educational Psychology, 49, 269-277.

Peterson, R.E., Centra, J.A., Hartnett, R.T., & Linn, R.L. (1983). Institutional Functioning Inventory: Preliminary technical manual. Princeton, New Jersey: Educational Testing Service.

Polytechnics and Colleges Funding Council (1990). Teaching quality: Report of the Committee of Inquiry appointed by the Council. London: Her Majesty's Stationery Office.

Popkewitz, T.S. (1983). Change and stability in schooling: The dual quality of educational reform. Geelong: Deakin University.

Ramsden, P. (1979). Student learning and perceptions of the academic environment. Higher Education, 8, 411-427.

Rutter, M., Maughan, B., Mortimore, P., Ouston, J. & Smith, A. (1979). Fifteen thousand hours: Secondary schools and their effects on children. Somerset: Open Books.

Stern, G.G. (1970). People in context: Measuring person-environment congruence in education and industry. New York: Wiley.

Streets, R.G. (1972). Career commitment of teacher graduates. Administrators' Bulletin, 3, 1-6.

Thistlethwaite, D.L. (1962). Rival hypotheses for explaining the effects of different learning environments. Journal of Educational Psychology, 3, 310-315.

Thistlethwaite, D.L. (1965). Effects of college upon student aspirations. Nashville: Vanderbilt University.

van Rooijen, L. (1986). Advanced students' adaptation to college. Higher Education, 15, 197-209.

Vyskocil, J.R. & Goens, G.A. (1979). Collective bargaining and supervision: A matter of climate. Educational Leadership, 37, 175-177.

Warren, W.G. & Rees, J.A. (1975). College and university learning environments. Journal of Educational Administration, 13, 46-53.

Wong, A.F.L. & Fraser, B.J. (1995). Cross-validation in Singapore of the Science Laboratory Environment Inventory. Psychological Reports, 76, 907-911.

Yarrow, A., Millwater, J. & Fraser, B. (1997). Improving university and primary school classroom environments through preservice teachers' action research. Practical Experiences in Professional Education, 1(1), 68-93.

ENDNOTE

  1. Copies of the ULEQ (Short Form), scale allocation, and scoring instructions can be obtained from the author.

Author details: Jeffrey P. Dorman is a Senior Lecturer in the School of Education, Australian Catholic University. He specialises in learning environment research.

Please cite as: Dorman, J. P. (2000). Validation and use of a short form of the university-level environment questionnaire. Queensland Journal of Educational Research, 16(1), 31-55. http://education.curtin.edu.au/iier/qjer/qjer16/dorman.html


[ Contents Vol 16, 2000 ] [ QJER Home ]
Created 19 Dec 2004. Last revision: 19 Dec 2004.
URL: http://education.curtin.edu.au/iier/qjer/qjer16/dorman.html