IIER logo 4
Issues in Educational Research, 2014, Vol 24(3), 260-280
[ Contents Vol 24 ] [ IIER Home ]

Does supply always come on the heels of demand? Matches and mismatches in e-learning

Bhavani Sridharan
Deakin University, Australia

Hepu Deng
RMIT University, Australia

Kinshuk
Athabasca University, Canada

Developing sustainable e-learning requires a better understanding of the perceptions and preferences of e-learning providers and e-learners on the four crucial dimensions for e-learning success including pedagogies, technologies, learning resources and management of learning resources. There is, however, little research on evaluating whether these critical dimensions are perceived as critical by e-learning providers and e-learners. To address this issue, this study investigates the gap between e-learners' and e-learning providers' perceptions and preferences on these critical dimensions for e-learning effectiveness. Such an investigation paves the way for developing appropriate measures to reduce the gap between the supply and the demand for sustainable e-learning.


Introduction

E-learning is "instructional content or learning experiences delivered or enabled by electronic technology" (ASTD, 2001). Recently there has been a huge systemic transformation of tertiary e-learning in the global education market. One of the key precursors to this transformation is the inception of massive open online course models. Such models provide learners with infinite opportunities to learn from anywhere, at any time and, free of cost (Karunasena, Deng, & Kim, 2013). As a result, the competition between higher education institutions is becoming increasingly intense worldwide.

To be competitive in such a dynamic environment, higher education institutions have to adequately recognise and consequently respond to the change. This means that the demand of learners (Irvine, Code, & Richards, 2013) needs to be met adequately. Against this backdrop, higher education institutions are rethinking about 'what and how' to scaffold 'teaching and learning' for stepping up the game. The crucial question is, however, whether these changes are aligned with the demand of e-learners and the supply of e-learning providers. In other words, the question is whether higher education institutions have the knowledge about 'what learners want' to reap the full benefits of this change and to cope with the demand of the competition (Jafari, McGee, & Carmean, 2006). In this scenario, the quote by Robert Collier (1926, p. 57), "supply always comes on the heels of demand", is pertinent in many ways owing to the power shift in favour of learners in the competitive higher education market.

Aligning the supply (what e-learning providers facilitate) and the demand (what e-learners want) is crucial for developing sustainable e-learning. There is, however, little research in evaluating whether the critical factors perceived by e-learners and e-learning providers respectively are compatible with each other. To fill this gap, the objective of this paper is to examine whether there is an alignment of the perceptions and preferences between these two stakeholders' on the critical factors influencing the effectiveness of e-learning.

This study builds on the earlier work in evaluating the critical dimensions for the e-learning effectiveness by providing a comparative analysis of the critical factors for sustainable e-learning based on the perceptions of e-learning providers and e-learners (Sridharan, Deng, Kirk, & Corbitt, 2010). Such an analysis helps identify the disparities and similarities between the preferences and perceptions of these stakeholders for sustainable e-learning. The study extends existing research on the critical factors for sustainable e-learning and provides higher education institutions with recommendations on the development of specific strategies and policies for sustainable e-learning.

The paper is organised as follows. Section 2 provides the background to this study. Section 3 presents a review of the related studies. Section 4 explains the research methodology used in this research. Section 5 illustrates the models and the hypothesis for this research. Section 6 presents a comparative analysis of the study. Finally, section 7 discusses the limitations and future research directions.

Background

The massive systemic transformation of higher education in recent years is focused on 'what and how to teach' and 'what and how to assess' (Biggs, 2012). This study is primarily about 'what and how to teach'. To facilitate the study, it is necessary to understand the key concepts evolving from this process.

The purpose of higher education with respect to 'what to teach' is about the transfer of discipline-specific knowledge. The significant shift in the focus towards developing employable graduates is due to the demand of employers and national and international accreditation agencies (Oliver, 2013). Traditionally, there has not been a very explicit focus on taxonomies of learning in tertiary education. This is because the primary focus of higher education is to teach and test the absorption of concepts and theories in a given domain for the purpose of certification. Modern educational reform calls for preparing learners with the development of deep-learning skills to face the real-life challenges and for being 'world-ready', 'future-ready'and career-ready' (Spencer, Riddle, & Knewstubb, 2011).

There has been significant evidence-based research for supporting the criticality of learner-centred teaching and learning (Hall Jr, 2013). To effectively integrate learner-centred teaching and learning, there is a call for the alignment between 'what learners do' and 'what teachers do' in knowledge acquisition (Biggs, 2012). This shows that there are revised expectations from learners, e-learning providers, employers, national quality and standards agencies, and international accreditation agencies. Figure 1 provides a summary of a comparative analysis between traditional and modern higher education.

Figure 1

Figure 1: A comparative analysis between traditional and modern higher education

There has been a paradigm shift in the role of teachers from "sage on stage" to "guide on side" with increased expectations and responsibilities. There are numerous challenges for teachers including innovatively scaffolding pedagogies and technologies for enhancing the learner experience, fostering a sense of community (Garrison, Anderson, & Archer, 2010), creating effective and diverse learning resources, upskilling on the use of the latest tools and technologies (Wong, 2012), participating in profession development sessions (Sun et al., 2008), and developing students' employability in addition to discipline specific knowledge (Oliver, 2013). It is evident that the power of 'teaching presence' is vital for positively impacting students' learning experiences (Marks, Sibley, & Arbaugh, 2005).

The discussion above shows that it is imperative that teachers' and learners' perceptions and preferences be given serious consideration for developing sustainable e-learning. There is, however, a dearth of research in understanding, identifying and matching e-learners' and e-learning providers' perceptions and preferences on the critical factors for e-learning. This study attempts to identify the critical factors as perceived by these two stakeholder groups for sustainable e-learning.

Related studies

The theoretical underpinning of this study is derived from the value theory (Ragowsky, Somers, & Adams, 2005) and the demand and supply theory (Klein, 1983). The value theory specifies the need for recognising what is important for individuals (Ragowsky et al., 2005). In this context, value refers to the 'perceived level of importance' of an item by individuals (Rokeach, 1969). An item refers to a critical factor for sustainable e-learning. Individuals in this study are e-learners and e-learning providers.

The demand and supply theory highlights the importance of matching the supply and the demand for specific items to achieve efficiency in resource allocation in a competitive world in which no individuals can dominate the market. In this study, the demand is related to the factors desired by e-learners, and the supply is about the factors perceived critical by e-learning providers for sustainable e-learning. There are many reasons why this is critical in higher education, such as resource constraint, lack of incentives for academics, rapid changes to information and communication technologies, and globalisation of the higher education sector (Hall Jr, 2013).

There are four inter-related critical dimensions for sustainable e-learning including pedagogies (Alexander & Boud, 2001), technologies (El-Mowafy, Kuhn, & Snow, 2013) , learning resources (Tzeng, Chiang, & Li, 2007), and management of learning resources (Duval, 2006). The development of sustainable e-learning depends on the effective interaction between these dimensions.

Pedagogies refer to the principles and methods of teaching and learning used in the knowledge transfer process. Several pedagogies are acknowledged as critical in e-learning including collaborative learning (Laurillard, 2009), interactive learning (Craighead, 2008), explorative learning (Yi, 2008), adaptive learning (VanLehn, 2006), and concept mapping (Novak, 2010). Collaborative learning refers to instructional methods where learners have an opportunity to work in groups and develop their team working skills by exchanging their ideas (Goodyear & Zenios, 2007). Interactive learning is where learners construct their own knowledge by interacting with subject matter through hands-on activities (Laurillard et al., 2013). Explorative learning is where learners construct their own knowledge by exploring and discovering inconsistencies in understanding within their learning environments (Dalgarno, 2001). Adaptive learning accommodates the differences in levels, styles and preferences of learners in contrast to the 'one-size-fits-all' concept. Concept mapping is a diagrammatic representation of the relationships between concepts in a specific situation (Novak, 2010).

Technologies refer to the use of educational tools and systems to support diverse pedagogies. Some of these technologies include learning management systems (LMS), intelligent tutoring systems (Craighead, 2008), collaborative technologies (Laurillard, 2009), Web 2.0 technologies (Conole, 2013), adaptive learning systems (VanLehn, 2006), concept map technologies (Liu, Chen, & Chang, 2010), semantic technologies (Charlton, Magoulas, & Laurillard, 2012), search and retrieval technologies (Yi, 2008), clicker technologies (Evans & Matthew, 2013) and mobile learning technologies (Wong, 2012).

The critical role of learning resources in scaffolding multiple pedagogies and technologies for enhancing e-learning is widely recognised in the literature. Several types of learning resources such as Web 2.0 resources, open educational resources, massive open online course resources, and interactive multimedia resources have been proposed for facilitating learner-centred pedagogies (Conole, 2013). Various approaches have been developed for adequately utilising existing learning resources. Richards (2007), for example, suggested an active re-use of learning resources for improving e-learning effectiveness. Craighead (2008) shows that using adaptive learning resources to fit individual learners' levels and styles can enhance e-learning. Liu et al. (2010) state that the adoption of diagram-based resources for supporting concept mapping can enhance the comprehension of individual learners.

The deployment of these pedagogies, technologies and learning resources often leads to the generation of a massive number of valuable e-learning resources. This necessitates the effective management of learning resources for overcoming the problem of information overload by filtering relevant and re-usable learning resources (Demidova et al., 2005). Several factors have been identified as critical for the effective management of learning resources (Nonaka & Toyamma, 2003). These factors include effective resources organisation and presentation (Yi, 2008), effective knowledge retrieval and re-use (Huang & Mille, 2006), filtering and pruning e-learning resources (Sridharan, Deng, & Corbitt, 2008), effective retrieval of multimedia objects (El Saddik, Fischer, & Steinmetz, 2001), and re-use of lessons (Ras & Rech, 2009).

Recent developments in the semantic web can augment the creation, extraction, organisation, retrieval and re-use of learning resources in e-learning (Huang, Webster, Wood, & Ishaya, 2006). Two aspects of the semantic web including metadata and ontologies play a significant role in the effective management of learning resources. Metadata is "any data which conveys knowledge about an item without requiring examination of the item itself" (Haase, 2004, p. 204). Ontologies are "the metadata schema providing a controlled vocabulary of concepts" (Maedche & Staab, 2001, p. 72). They define the relationship between concepts (Berners-Lee, Handler, & Lassila, 2006). Ontologies facilitate communication between people and computers through a shared understanding of resources (Davies, Harmelen, & Fensel, 2002).

The four dimensions described above are mutually interdependent (Sridharan, Deng, & Corbitt, 2010). Pedagogies have no value in e-learning without embedding the associated technologies, learning resources and facilitates for retrieval of learning resources. Analogously, instructional technologies and learning resources have limited usefulness without understanding the pedagogical principles behind these dimensions. Figure 2 represents the intertwined nature of the four dimensions for enhancing the e-learning effectiveness.

There are innumerable factors within each dimension that are critical for sustainable e-learning. Identifying the critical factors from the perspective of both demand and supply is vital for e-learning success. There is, however, little research in understanding the compatibility of these critical factors between e-learners and e-learning providers. This study evaluates the similarities and the variations in the perceptions of e-learners and e-learning providers on the critical dimensions and the critical factors for sustainable e-learning. This evaluation leads to a reduced number of factors perceived as critical by both e-learners and e-learning providers, as represented in Figure 3.

The main research question addressed in this study is as follows: Is there a match between the perceptions of e-learners and e-learning providers on the critical dimensions and the critical factors within each dimension for sustainable e-learning?

Research methods

A cross-sectional research design using interviews and surveys is used for realising the purpose of this study. The target population in this study is e-learning providers for the interview and e-learners for the survey. The qualitative data are collected through structured interviews. This process consists of nine steps as shown in Figure 4.

Figure 2

Figure 2: The interconnection between the four dimensions of e-learning

Figure 3

Figure 3: A conceptual framework for identifying the critical factors

The qualitative data collection consists of five phases including identifying the critical factors for each dimension, developing preliminary interview questions, pilot-testing and revising interview questions, conducting interviews, and transcribing and analysing interviews. The interview questions include (a) demographic information, (b) perceptions on the influence of pedagogies, technologies, learning resources and management of learning resources, and (c) perceptions on the critical factors for sustainable e-learning. A total of twenty-nine interviews were conducted, out of which twenty-seven were from five universities in a south eastern city in Australia, and two were with representatives from the Open Universities Australia. Prospective candidates for interviews were identified through an Internet search of university websites and contacts provided by colleagues and interviewees.

Figure 4

Figure 4: The research process

The quantitative data collection includes formulating the survey instrument, pilot-testing questions with domain experts and e-learners, conducting an online survey of e-learners, and cleaning and preparing collected data for analysis. There are seven latent constructs with 62 items in the survey instrument including pedagogies, technologies, learning resources, management of learning resources, metadata ontologies, management effectiveness and e-learning effectiveness. The instrument items are derived from an iterative process of literature review and content analysis of the interview. The online data collection was accomplished over a period of nine months. To prevent errors arising from data collection, specific measures were taken to avoid incorrect data entries, invalid entries and missing responses. This lead to two hundred and ten valid responses being collected.

A content analysis of the transcribed interviews was conducted, resulted in the extension of items, formulation of preliminary hypotheses, and the development of the base model for this study. The analysis was used to see whether or not a hypothesis was possible (Bouma & Atkinson, 1995). Such an analysis would help identify probable factors for sustainable e-learning in developing a base model.

Several measures were taken to ensure the reliability and validity of the findings in this study. To ensure the reliability, this study adhered to Kleven's (2008) consistency checks for guaranteeing the uniformity of measurements. To ensure the validity, this study used Johnson's (1997) framework to evaluate its methodological strengths and weaknesses, with appropriate steps taken to overcome the weaknesses. Researchers' bias and three types of validity (descriptive, interpretive and theoretical) were adhered to. In addition, the internal and external validity (Maxwell, 1992) were applied due to the exploration of the cause and effect relationship and the plausible generalisation of the findings.

The quantitative data analysis consisted of preliminary multivariate analysis, instrumental validity, exploratory factor analysis and confirmatory factor analysis (CFA). KMO measure and Bartlett's test of sphericity were used to validate the sampling adequacy and suitability of data for factor analysis. Mahalanobis distance, skewness and kurtosis measures were used to confirm the normality of the dataset. Instrumental validity was tested through three reliability measures including internal consistency, item reliability and construct reliability and two validity measures including convergent and discriminant validity measures. The use of these measures helped to reduce the number of items that did not fit the recommended guidelines for developing a more parsimonious model.

The primary focus of this study was to test the relationship between the latent constructs in the hypothesised model. A two-step approach to structural equation modelling was considered including estimating the measurement model and the structural model. A CFA was conducted in the measurement model for assessing the contribution of each indicator variable and for measuring the model's adequacy. Three stages were involved in assessing the measurement model including (a) the model specification, (b) the iterative model modification, and (c) the estimation of parameters. The iterative model modification process required refinement and retesting of individual measurement models. This resulted in developing a more parsimonious limited set of items to represent a construct, followed by an estimation of the structural model (Anderson & Gerbing, 1988).

The overall model fitness was evaluated using several measures of the goodness-of-fit (GOF) including the chi-square test (?2), the ratio of ?2 to degrees of freedom (DF), the goodness-of-fit index (GFI), the adjusted goodness-of-fit index (AGFI), the root mean square error of approximation (RMSEA), and the Tucker-Lewis index (TLI). The significance of the path coefficient was assessed using the standard error and the t-value (Holmes-Smith, Cunningham, & Coote, 2006).

A comparative analysis of the perceptions of the stakeholders was conducted that consisted of (a) compilation and recording of key findings from both the study, (b) identification of the critical factors within each dimension from both groups of stakeholders, (c) identification of matching factors, (d) identification of mismatching factors, and (e) identification of the reasons for those differences. The details were recorded and organised using a systematic classification process using Microsoft Excel worksheets.

Models and hypothesis

The base model in this study is derived from a methodical review of the relevant literature before and after the interview. This iterative process resulted in an extension of the number of items from thirty-eight in the interview stage to sixty-two in the survey stage. The development of an iterative measurement model lead to the deletion of multiple items including five items during the instrumental validation phase, thirty-two items during the convergent validity phase, and nine items during the discriminant validity stage. Two constructs were combined into one (technologies and learning resources) due to a lack of discriminant validity. As a result, the most critical factors were identified.

Figure 5 represents the summary of the proposed relationship and hypotheses for the base model and the refined model. The independent latent constructs are represented in blue shaded boxes, and the dependent latent constructs in the green shaded boxes. The base model and the refined model are evaluated using interview results and the survey findings respectively. The base model represents the direct relationships between the six constructs and e-learning effectiveness in six respective hypotheses represented as H1 to H6. The refined model extends the base model to capture both the direct and indirect relationships between the six constructs and the e-learning effectiveness. Consequently, nine hypotheses were proposed for considering the direct effects, and three hypotheses were developed for consideration of the indirect effects.

Figure 5a: Base model and hypothesis
Base model and hypothesis

Figure 5b: Refined model and hypotheses
Refined model and hypotheses

Figure 5: Summary of the relationships in the base model and the refined model

There are specific differences between the base model and the refined model. In the refined model, the first set of hypotheses H1, H2a, H2b and H3 deals with the direct influence of four latent constructs on the e-learning effectiveness. These four hypotheses are the replica of the base model hypotheses of H1, H2, H3 and H5. The second set of hypotheses H4, H5a, H5b and H6 examined the direct influence of four latent constructs on the management of learning resources. The third set of hypotheses H7, H8 and H9 examined the indirect influence of two mediating latent constructs on the management effectiveness and the e-learning effectiveness. Except for H8, the other two hypotheses H7 and H9 are the same as the base model hypotheses represented as H4 and H6.

Results

Interview findings

An analysis of the interview transcripts lead to the identification of new items and the splitting of exiting items for facilitating the development of the survey instrument. This iterative process resulted in adding twenty-four new items. Furthermore it provides valuable insights into e-learning providers' perceptions about the identified critical dimensions and the critical factors.

Most interviewees acknowledged the importance of combining pedagogies for improving e-learning effectiveness. Two widely-used strategies were collaborative learning and explorative learning. There is great variation, however, in the implementation of these strategies. For instance, collaborative learning is adopted from the proactive involvement of academic staff through incorporating fully moderated collaborative forums to unmoderated discussions by students. Explorative learning is used with additional course-related learning resources through external links. However in most instances, the assessment of authenticity, quality and value of exploratory e-learning resources is left to the judgment of individual learners. With respect to interactive learning, adaptive learning and concept maps, interviewees highlighted the importance of embedding them in online courses. Notwithstanding these views, in reality interviews reveal a general under-utilisation of these pedagogies and allied technologies.

The necessity for using appropriate technologies and learning resources in e-learning was extensively recognised by the interviewees. The adoption of appropriate technologies and learning resources for e-learning, however, has not become a reality due to several barriers to their implementation. These barriers include the ineffectiveness of LMS for enhancing a learner-centred learning process, a lack of understanding the underpinning pedagogy behind the use of these technologies, and the efficient use of academics' time and effort and the effect on personal career prospects. The two most popular technologies include LMS and collaborative learning technologies. Two popular learning resources include multi-media resources and external learning resources.

There are mixed views on the management of learning resources and associated dimensions such as metadata ontologies and management effectiveness. The study shows that LMS are not effective particularly in terms of reusability and searchability. It reveals that e-learning resources within LMS are neither reusable nor inter-operable. The key challenges in creating a reusable learning object repository include reusability, shareability, searchability, authenticity, version controls, granularity and copyright issues.

The findings indicated a positive association between pedagogies (H1), technologies (H2) and learning resources (H3), and e-learning effectiveness. This is in contrast to a lack of clear support for the positive influence of management of learning resources (H4), metadata ontologies (H5) and management effectiveness (H6) on the e-learning effectiveness. Interviewees from the library and content management division in one university believed that these factors are critical for the reusability of learning resources. This view, however, was only endorsed by some interviewees. Several interviewees felt that it would be an absolute waste of time and resources to create these repositories which would soon become obsolete.

The interview findings substantiate the literature on the perceived effectiveness of multiple pedagogical approaches for enhancing the e-learning effectiveness. Overall, most interviewees agreed that adopting multiple pedagogical strategies and associated technologies and learning resources can achieve sustainable e-learning success. The interview findings reveal various views on the power of the management of resources to enhance e-learning effectiveness. Further research is required to validate this finding by using surveys to collect data from key stakeholders with a micro-level data analysis.

Survey findings

This section reports the results from the quantitative analysis of the survey data by estimating the structural model based on three measures: the overall GOF measure, the ability to explain the variance in the dependent variables, and the significance of the estimates of model coefficients. The final revised structural model contains technologies and learning resources as a single combined construct because of the lack of discriminant validity. This has resulted in a reduction in the number of hypotheses, from eleven to nine, and reduced the number of constructs from seven to six. Hypotheses H2a and H2b have been coupled and denoted as H2, while hypotheses H5a and H5b have been coupled and denoted as H5. The final structural model shows the path coefficients and the explanatory power (R2) for each dependent construct, and non-significant paths are represented by dotted lines, as shown in Figure 6.

Figure 6 hypothesised refined model

Figure 6 final structural model

Figure 6: A hypothesised refined model (above) and a final structural model (below)

The first criterion for assessing the structural model is the GOF measure. The GOF measure and the recommended value for the structural model suggest that the model displays a good fit for the dataset with a χ2 result of 112.04 with 92 DF. This observation is further endorsed by a χ2/df result of 1.22, and the GFI, AGFI and TLI results with values greater than 0.90 for each of these measures. An RMSEA of 0.03 is also within the recommended value of 0.1.

The second criterion is the explanatory power (R2) of each dependent construct. These R2 results indicate that the model explains 50% of the variance in the e-learning effectiveness, 41% of the variance in the management of learning resources, and 18% of the variance in management effectiveness.

The third criterion is the significance of the model coefficients for all structural paths in the structural model. Table 1 presents the hypothesis results from the structural models including the hypothesis statements, their path coefficient values, the significance of each hypothesis, and the results in terms of the acceptance or rejection of each hypothesis. The findings suggest strong support for H3, H5 and H8, moderate support for H2, H6 and H7, weak support for H9 and no support for H1 and H4. This implies that e-learning effectiveness, as perceived by e-learners, can be explained by the management of learning resources, technologies and learning resources, and the metadata ontologies supporting the management of learning resources. The findings also show the rejection of H1 and H4 indicating that pedagogies have a non-significant effect on the e-learning effectiveness and the management of learning resources. However, the indirect influence of pedagogy can be seen from the elevated total effect represented in parentheses in Table 1.

Table 1: Structural model: hypothesis results

HypothesisPathSignificanceResult
H1:Pedagogies positively influence the e-learning effectiveness-0.08n.sN
H2:Technologies and learning resources positively influence the e-learning effectiveness0.22 (0.37)*Yes
H3: Metadata ontologies positively influence the e-learning effectiveness0.33 (0.42)***Yes
H4: Pedagogies positively influence the management of learning resources0.16n.sNo
H5: Technologies and learning resources positively influence the management of learning resources0.41***Yes
H6: Metadata ontologies positively influence the management of learning resources0.25**Yes
H7: Management of learning resources positively influences the e-learning effectiveness0.29 (0.35)*Yes
H8: Management of learning resources positively influences the management effectiveness of learning resources0.42***Yes
H9: Management effectiveness positively influences the e-learning effectiveness0.15*Yes
*p<0.05; **p<0.01; *** p<0.001

Discussion: A comparative analysis

A comparison of the critical dimensions for sustainable e-learning perceived by e-learning providers and e-learners reveals the disparity at both the dimension level and the factor level. The analysis is organised into two parts: (a) similarities and differences in findings and (b) the reasons for those differences and the challenges in practice.

Table 2 presents a comparative analysis of the perceptions of e-learners and e-learning providers on the influence of key constructs on e-learning effectiveness. With respect to pedagogies, the qualitative study indicates the criticality of using multiple pedagogies to enhance the e-learning effectiveness. There is strong support for all five strategies (interactive, collaborative, adaptive, concept mapping and explorative learning strategies) to enhance e-learning effectiveness. In contrast, the quantitative result suggests that pedagogies per se are not critical from the perspective of e-learners. This result is inconsistent with the finding in the literature on the positive role of pedagogies on influencing e-learning effectiveness (Walker & Fraser, 2005).

The perception that pedagogy per se is not critical is consistent with the view that "pedagogy and interactions are determined by system rather than learners or instructional designers" (Carmean & Brown, 2005, p. 155). Analogous with this view, Brennan (2001, p. 25) report, "there is a disjunction between the reform pedagogy assumptions that policy-makers hold and what actually happens. It is not surprising because in the online environment it is shockingly difficult to get beyond transmission".

Table 2: Matches and mismatches between e-learners and e-learning providers

HypothesisE-learning
providers support
E-learners
support
Match/
mismatch
H1:Pedagogies positively influence the e-learning effectivenessStrongNo SupportMismatch
H2:Technologies and learning resources positively influence the e-learning effectivenessStrongLow SupportMatch
H3: Metadata ontologies positively influence the e-learning effectivenessPartialHigh SupportMatch
H4: Pedagogies positively influence the management of learning resourcesN/A*No SupportN/A*
H5: Technologies and learning resources positively influence the management of learning resourcesN/A*High SupportN/A*
H6: Metadata ontologies positively influence the management of learning resourcesN/A*Medium supportN/A*
H7: Management of learning resources positively influences the e-learning effectivenessPartialLow SupportMatch
H8: Management of learning resources positively influences the management effectiveness of learning resourcesPartialStrong SupportMatch
H9: Management effectiveness positively influences the e-learning effectivenessPartialLow SupportMatch
* N/A - Not applicable

There are some probable reasons that could be attributed to the differences in the perceptions of e-learners and e-learning providers. For e-learners, pedagogies are at the back-end of e-learning systems. They are 'behind the scenes' and are not of any concerns to e-learners. In contrast, a choice of pedagogies is a central issue for e-learning providers, as the design of e-learning courses is pedagogy-based (Carmean & Brown, 2005). E-learners have more expectations on the 'what' side of the coin rather than 'how' aspects of the pedagogy underpinning what is provided to them. However, the 'how' aspects of pedagogies are critical for e-learning providers due to their importance for the design and development of e-learning courses. Further research into the views of e-learners and e-learning providers using an identical survey simultaneously would provide more insights into the criticality of pedagogies for sustainable e-learning.

Both e-learning providers and e-learners perceive technologies and learning resources to be critical for e-learning success. This is consistent with the findings that technology positively influences the e-learning effectiveness (Chandra & Lloyd, 2008). Differences, however, exist on the extent of actual use by e-learning providers. For instance, the qualitative study indicates the lack of a wide use of technologies and learning resources for supporting active learning, visual learning, and explorative learning. In reality one of the most prevalent technologies is the technology supporting collaborative learning integrated in the LMS. Within those collaborative technologies, the qualitative study identifies an underutilisation of the technology. In contrast, the quantitative study confirms the criticality of technologies and learning resources related to concept mapping to enhance e-learning effectiveness. However, the earlier qualitative results suggest that a wide use of concept mapping technologies and associated resources in e-learning is lacking.

The qualitative study identifies many challenges in incorporating the critical factors in the technology dimension. These challenges include the lack of understanding of the theory behind the technologies, the lack of knowledge of the full potential of these technologies (McGill & Klobas, 2009), and the time and effort required to create learning resources. This shows that similarities and differences exist in the perceptions of e-learners and e-learning providers on the influence of the technologies and learning resources on the e-learning effectiveness.

With respect to the management of learning resources, the qualitative findings suggest a lack of unanimous views about its positive influence on enhancing e-learning effectiveness. As a result, the learning resources generated are not transferred to reusable learning resources. In addition, the findings clearly indicate a lack of support for the creation of metadata ontologies to enhance the reusability and searchability of learning resources. This contradicts the findings in the literature that have indicated the positive role of the management of learning resources, metadata ontologies (Hatem, Ramadan, & Neagu, 2005) and management effectiveness (Shaw, Dicks, Venkatesh, Lowerison, & Dai, 2004) in enhancing e-learning effectiveness. In comparison, the quantitative results are consistent with the findings in the literature which identify management of learning resources, including metadata ontologies and management effectiveness, as the critical dimensions for the e-learning effectiveness.

Many reasons can be attributed to the philosophical differences between e-learning providers and e-learners on the influence of the management of learning resources and metadata ontologies in enhancing e-learning effectiveness. For example, e-learning providers may not pay much attention to the management of learning resources due to lack of time and the huge amount of effort required to manage of learning resources and create of metadata ontologies. There may be resistance to share resources, copyright issues, quality, granularity, version control and validation of learning resources and metadata ontologies. In contrast, quick accessibility to relevant and authentic resources is critical for e-learners' knowledge acquisition.

Table 3 summarises the similarities and differences in the perceptions between e-learning providers and e-learners on the e-learning effectiveness based on the qualitative and quantitative findings. It is apparent that both e-learning providers and e-learners view the use of technologies and learning resources as critical for sustainable e-learning. However, many technologies and learning resources are not widely used due to practical difficulties. One of the key obstacles is a lack of LMS for incorporating various technologies supporting identified pedagogies.

Table 3: Comparison of qualitative and quantitative findings

DimensionE-learning providers' perceptionsE-learners' perceptions
PedagogiesPerceived as critical, but not widely practiced in reality.
Specific perceived critical factors are interactive, collaborative, adaptive, concept mapping and explorative learning strategies
Not perceived as critical
Technologies and learning resourcesPerceived as critical, but many obstacles and challenges exist.
Specific identified critical factors are LMS, Collaborative and external links
Perceived as critical. Specific critical factors are concept mapping technologies, push technologies and diagram-based learning resources
Management of learning resources (including metadata ontologies and management effectiveness)Perceived as critical only by a few, but many challenges and obstacles exist.
Specific identified critical factors are reusability, search facilities, keywords, version controls
Perceived as critical. Specific critical factors are search facilities, presentation, metadata details in particular prerequisite and co-requisite learning resources

The comparative analysis indicates both similarities and disparities of the perceptions on the dimensions for sustainable e-learning, as represented in Table 3. The table reveals the difference in the perceptions of two e-learning dimensions: pedagogies and management of learning resources and also shows the similarity in the perceptions of the other two e-learning dimensions: technologies and learning resources.

Conclusion

This study appraises whether there is an alignment between the perceptions of e-learning providers and e-learners on the critical dimensions for sustainable e-learning. Accordingly, this study considers four critical dimensions for sustainable e-learning based on the perceptions of e-learning providers and e-learners. The findings show that the supply does not always follow on the heels of the demand due to the differences in perceptions and the challenges to fulfilling the demand. The findings suggest that major differences between the perceptions of e-learning providers and e-learners exist on two dimensions: pedagogies and management of learning resources. While the use of manifold pedagogies is considered to be critical by e-learning providers, the same is not true of e-learners. The findings indicate the management of learning resources is considered critical by e-learners, but is not strongly perceived by e-learning providers. In regard to technologies and learning resources, the findings indicate synergy between e-learners' and e-learning providers' perceptions for sustainable e-learning. The study indicates the existence of a gap between idealism and reality as most of the dimensions perceived as critical are not being implemented.

This study contributes to the e-learning domain by identifying the necessity for developing more aligned policy measures to unify the critical e-learning dimensions and to eliminate the barriers for sustainable e-learning. The study may assist e-learning providers to implement policy measures which better align the demand with the supply to enhance e-learning effectiveness. Specific measures include providing capacity building sessions to educate academic staff about the effective use and alignment of all four critical dimensions, embedding powerful plug-in technologies to overcome the inherent limitations of LMS, offering incentives for proactive and innovative initiatives of academic staff, and adopting a balanced approach to innovative teaching and the expectation for quality research. The study has a limitation of aggregating and dichotomising huge constructs. To overcome this limitation, future research is proposed by disaggregating and performing a micro-level analysis at each construct level.

Acknowledgments

Vicky White from Cancer Council Victoria (CCV) is acknowledged for allowing the first author to spend time on writing this paper. The researchers would also like to thank the reviewers and the editor for their constructive feedback in significantly improving this paper.

References

Alexander, S., & Boud, D. (2001). Learners still learn from experience when online. In J. Stephenson (Ed.), Teaching and learning online: Pedagogies for new technologies (pp. 3-15). London, UK: Kogan Page.

Anderson, J.C., & Gerbing, D.W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103(3), 411-423. http://psycnet.apa.org/doi/10.1037/0033-2909.103.3.411

ASTD (2001). A vision of e-learning for America's workforce: Report of the commission on technology and adult learning. American Society of Training Directors: Policy and Public Leadership. Alexandria.

Berners-Lee, T., Handler, J., & Lassila, O. (2006). The semantic web. Database and Network Journal, 36(3), 7.

Biggs, J. (2012). What the student does: Teaching for enhanced learning. Higher Education Research & Development, 31(1), 39-55. http://dx.doi.org/10.1080/07294360.2012.642839

Bouma, G.D., & Atkinson, G.B.J. (1995). A handbook of social science research (2 ed.). Oxford, New York: Oxford University Press.

Brennan, R. (2001). One size doesn't fit all: Pedagogy in the online environment (pp. 1-82). Adelaide: NCVER. http://www.ncver.edu.au/wps/poc?urile=wcm:path:/wps/wcm/connect/NCVER_Shared/Publications/965

Carmean, C., & Brown, G. (2005). Measure for measure: Assessing course management systems. Hershey, PA: Information Science Publishing.

Chandra, V., & Lloyd, M. (2008). The methodological nettle: ICT and student achievement. British Journal of Educational Technology, 39(6), 1087-1098. http://dx.doi.org/10.1111/j.1467-8535.2007.00790.x

Charlton, P., Magoulas, G., & Laurillard, D. (2012). Enabling creative learning design through semantic technologies. Technology, Pedagogy and Education, 21(2), 231-253. http://dx.doi.org/10.1080/1475939X.2012.698165

Collier, R. (1926). The secret of the ages. http://newthoughtlibrary.com/collierRobert/secretOfTheAges/

Conole, G. (2013). Designing for learning in an open world. New York: Springer.

Craighead, J. (2008). Distributed, game-based, intelligent tutoring systems - the next step in computer based training? Proceedings of the International Symposium on Collaborative Technologies and Systems, Irvine, CA, USA. http://dx.doi.org/10.1109/CTS.2008.4543938

Dalgarno, B. (2001). Interpretations of constructivism and consequences for computer assisted learning. British Journal of Educational Technology, 22(2), 183-194. http://dx.doi.org/10.1111/1467-8535.00189

Davies, J., Harmelen, F.V., & Fensel, D. (2002). Towards the semantic web: Ontology-driven knowledge management. England: John Wiley & Sons Ltd.

Demidova, E., Ternier, S., Olmedilla, D., Dual, E., Dicerto, M., Stefanov, K., & Sacristan, N. (2005). Integration of heterogeneous information sources into knowledge resource management system for lifelong learning. Proceedings of the TEN Competence Workshop on Service Oriented Approaches and Lifelong Competence Development Infrastructures, Manchester, UK. http://dspace.ou.nl/bitstream/1820/883/1/KRMSforLifelongLearning.pdf

Duval, E. (2006). IEEE standard for learning object metadata (LOM). http://dx.doi.org/10.1109/IEEESTD.2002.94128

El-Mowafy, A., Kuhn, M., & Snow, T. (2013). Blended learning in higher education: Current and future challenges in surveying education. Issues in Educational Research, 23(2), 132-150. http://www.iier.org.au/iier23/el-mowafy.html

El Saddik, A., Fischer, S., & Steinmetz, R. (2001). Reusable multimedia content in web-based learning systems. IEEE Multimedia, 8(3), 30-38. http://dx.doi.org/10.1109/93.939998

Evans, R., & Matthew, A. (2013). A new era; Personal technology challenges educational technology. In Electric dreams: Proceedings ASCILITE Sydney 2013. http://www.ascilite.org/conferences/sydney13/program/papers/Evans.pdf

Garrison, D.R., Anderson, T., & Archer, W. (2010). The first decade of the community of inquiry framework: A retrospective. The Internet and Higher Education, 13(1-2), 5-9. http://dx.doi.org/10.1016/j.iheduc.2009.10.003

Haase, K. (2004). Context for semantic metadata. In Proceedings of the 12th ACM International Conference on Multimedia, New York, USA. http://www.beingmeta.com/pubs/mm2004haase.pdf

Hall Jr, O.P. (2013). Assessing faculty attitudes toward technological change in graduate management education. Journal of Online Learning & Teaching, 9(1), 39-51. http://jolt.merlot.org/vol9no1/hall_0313.htm

Hatem, M.S., Ramadan, H.A., & Neagu, D.C. (2005). E-learning based on context oriented semantic web. Journal of Computer Science, 1(4), 500-504. http://thescipub.com/PDF/jcssp.2005.500.504.pdf

Holmes-Smith, P., Cunningham, E., & Coote, L. (2006). Structural equation modelling: From the fundamentals to advanced topics. Education & Statistics Consultancy, Statsline.

Huang, W., & Mille, A. (2006). ConKMeL: A contextual knowledge management framework to support multimedia e-learning. Multimedia Tools and Applications, 30(2), 205-219. http://dx.doi.org/10.1007/s11042-006-0024-4

Huang, W., Webster, D., Wood, D., & Ishaya, T. (2006). An intelligent semantic e-learning framework using context-aware semantic web technologies. British Journal of Educational Technology, 37(3), 351-373. http://dx.doi.org/10.1111/j.1467-8535.2006.00610.x

Irvine, V., Code, J., & Richards, L. (2013). Realigning higher education for the 21st-century learner through multi-access learning. Journal of Online Learning & Teaching, 9(2), 172-186. http://jolt.merlot.org/vol9no2/irvine_0613.pdf

Jafari, A., McGee, P., & Carmean, C. (2006). Managing courses, defining learning: What faculty, students, and administrators want. EDUCAUSE Review, 41(July/August), 50-71. https://net.educause.edu/ir/library/pdf/erm0643.pdf

Johnson, R.B. (1997). Examining the validity structure of qualitative research. In A. K. Milinki (Ed.), Cases in qualitative research: Research reports for discussion and evaluation (pp. 160-165). Los Angeles, CA: Pyrczak Publishing.

Karunasena, A., Deng, H., & Kim, B. (2013). Structural equation modelling for Web 2.0 based interactive e-learning in Sri Lanka. International Journal of Information Technology and Computer Science, 12(3), 1-8. http://www.ijitcs.com/volume%2012_No_3/Anuradha.pdf

Klein, L. (1983). The economics of supply and demand. Oxford: Basil Blackwell.

Kleven, T.A. (2008). Validity and validation in qualitative and quantitative research. Nordic Educational Research, 28, 219-233. http://www.idunn.no/ts/np/2008/03

Laurillard, D. (2009). The pedagogical challenges to collaborative technologies. International Journal of Computer-Supported Collaborative Learning, 4(1), 5-20. http://dx.doi.org/10.1007/s11412-008-9056-2

Laurillard, D., Charlton, P., Craft, B., Dimakopoulos, D., Ljubojevic, D., Magoulas, G., Masterman, E., Pujadas, R., Whitley, E.A., & Whittlestone, K. (2013). A constructionist learning environment for teachers to model learning designs. Journal of Computer Assisted Learning, 29(1), 15-30. http://dx.doi.org/10.1111/j.1365-2729.2011.00458.x

Liu, P.-L., Chen, C.-J., & Chang, Y.-J. (2010). Effects of a computer-assisted concept mapping learning strategy on EFL college students' English reading comprehension. Computers & Education, 54(2), 436-445. http://dx.doi.org/10.1016/j.compedu.2009.08.027

Maedche, A., & Staab, S. (2001). Learning ontologies for the semantic web. IEEE Intelligent Systems and Their Applications, 16(2), 72-79. http://ceur-ws.org/Vol-40/maedche+staab.pdf

Marks, R.B., Sibley, S.D., & Arbaugh, J.B. (2005). A structural equation model of predictors for effective online learning. Journal of Management Education, 29(4), 531-563. http://dx.doi.org/10.1177/1052562904271199

Maxwell, J.A. (1992). Understanding and validity in qualitative research. Harvard Educational Review, 62(3), 279-299. http://hepg.org/her-home/issues/harvard-educational-review-volume-62,-issue-3/herarticle/_377

McGill, T.J., & Klobas, J.E. (2009). A task-technology fit view of learning management system impact. Computers & Education, 52(2), 496-508. http://dx.doi.org/10.1016/j.compedu.2008.10.002

Nonaka, I., & Toyamma, R. ( 2003). The knowledge-creating theory revisited: Knowledge creation as synthesizing process. Knowledge Management Research and Practice, 1(1), 2-10. http://www.palgrave-journals.com/kmrp/journal/v1/n1/pdf/8500001a.pdf

Novak, J.D. (2010). Learning, creating, and using knowledge: Concept maps as facilitative tools in schools and corporations. NY: Routledge.

Oliver, B. (2013). Graduate attributes as a focus for institution-wide curriculum renewal: Innovations and challenges. Higher Education Research & Development, 32(3), 450-463. http://dx.doi.org/10.1080/07294360.2012.682052

Ragowsky, A., Somers, T.M., & Adams, D.A. (2005). Assessing the value provided by ERP applications through organizational activities. Communications of AIS, 16(18), 1-50. http://aisel.aisnet.org/cgi/viewcontent.cgi?article=3036&context=cais

Ras, E., & Rech, J. (2009). Using wikis to support the net generation in improving knowledge acquisition in capstone projects. Journal of Systems and Software, 82(4), 553-562. http://dx.doi.org/10.1016/j.jss.2008.12.039

Richards, D. (2007). Collaborative knowledge engineering: Socialising expert systems. Proceedings of the 11th International Conference on Computer Supported Cooperative Work in Design, Nice, France. http://dx.doi.org/10.1109/CSCWD.2007.4281510

Rokeach, M. (1969). The role of values in public opinion research. Public Opinion Quarterly, 32(4), 547-559. http://dx.doi.org/10.1086/267645

Shaw, S., Dicks, D., Venkatesh, V., Lowerison, G., & Dai, Z. (2004). An empirical evaluation of topic map search capabilities in an educational context. Proceedings of the 5th International Conference on Information Technology Based Higher Education and Training, Istanbul, Turkey. http://dx.doi.org/10.1109/ITHET.2004.1358242

Spencer, D., Riddle, M., & Knewstubb, B. (2011). Curriculum mapping to embed graduate capabilities. Higher Education Research & Development, 31(2), 217-231. http://dx.doi.org/10.1080/07294360.2011.554387

Sridharan, B., Deng, H., & Corbitt, B. (2008). Evaluating intertwined critical success factors for sustainable e-learning. In Proceedings of the 19th Australasian Conference on Information Systems, Christchurch, New Zealand. http://aisel.aisnet.org/cgi/viewcontent.cgi?article=1099&context=acis2008

Sridharan, B., Deng, H., & Corbitt, B. (2010). Critical success factors in e-learning ecosystems: A qualitative study. Journal of Systems and Information Technology, 12(4), 263-288. http://dx.doi.org/10.1108/13287261011095798

Sridharan, B., Deng, H., Kirk, J., & Corbitt, B. (2010). Structural equation modelling for evaluating the user perceptions of e-learning effectiveness. In Proceedings of the 18th European Conference on Information Systems, Pretoria, South Africa. http://aisel.aisnet.org/ecis2010/59

Tzeng, G.-H., Chiang, C.-H., & Li, C.-W. (2007). Evaluating intertwined effects in e-learning programs: A novel hybrid MCDM model based on factor analysis and DEMETEL. Expert Systems with Applications, 32(4), 1028-1044. http://dx.doi.org/10.1016/j.eswa.2006.02.004

VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3), 227-265. http://iospress.metapress.com/content/AL6R85MM7C6QF7DR

Walker, S., & Fraser, B. (2005). Development and validation of an instrument for assessing distance learning environments in higher education: The distance education learning environments survey (DELES). International Journal of Learning Envrionments Research, 8(3), 289-308. http://dx.doi.org/10.1007/s10984-005-1568-3

Wong, L.-H. (2012). A learner-centric view of mobile seamless learning. British Journal of Educational Technology, 43(1), E19-E23. http://dx.doi.org/10.1111/j.1467-8535.2011.01245.x

Yi, M. (2008). Information organization and retrieval using a topic maps-based ontology: Results of a task-based evaluation. Journal of the American Society for Information Science and Technology, 59(12), 1898-1911. http://dx.doi.org/10.1002/asi.20899

Authors: Dr Bhavani Sridharan is a Lecturer in Learning Innovations with the Faculty of Business and Law, Deakin University. Her areas of research interests include online learning, assurance of learning, rubrics, graduate attributes, learning technologies, authentic assessment, learning analytics, database management and quality of teaching and learning.
Email: bhavani.sridharan@deakin.edu.au

Dr Hepu Deng is a Professor in Information Systems in the School of Business Information Technology and Logistics, RMIT University. His research interests are in the areas of decision analysis, intelligent systems, digital business, knowledge management, electronic government, e-learning, and their applications.
Email: hepu.deng@rmit.edu.au

Professor Kinshuk is Full Professor and Associate Dean of Faculty of Science and Technology at Athabasca University, Canada. He is also the NSERC/iCORE/Xerox/Markin Research Chair for Adaptivity and Personalization in Informatics. His research interests include adaptivity and personalization, learning analytics, mobile and ubiquitous technologies, and cognitive profiling.
Email: kinshuk@athabascau.ca

Please cite as: Sridharan, B., Deng, H. & Kinshuk (2014). Does supply always come on the heels of demand? Matches and mismatches in e-learning. Issues in Educational Research, 24(3), 260-280. http://www.iier.org.au/iier24/sridharan.html


[ PDF version of this article ] [ Contents Vol 24 ] [ IIER Home ]
© 2014 Issues In Educational Research. This URL: http://www.iier.org.au/iier24/sridharan.html
Created 30 Nov 2014. Last revision: 30 Nov 2014.
HTML: Roger Atkinson [rjatkinson@bigpond.com]