You take the high road: National Programmes for the Development of e-Learning in Higher Education

by Terry Mayes, and Derek Morrison in Reflecting Education, Vol 4, No 1 (2008), 6-16

Abstract

The central question addressed by this paper is the effect of national initiatives in e-learning within the Higher Education sector. Two national programmes for the promotion of e-learning in UK higher education are described, and some tentative lessons are drawn from their comparison. One is the English Benchmarking and Pathfinder programme, still ongoing, in which £8M has been distributed widely across over 70 HE institutions, and the other is the £6M Scottish e-learning transformation programme, involving six large-scale collaborative projects. The scale of these two programmes is comparable to the Pew Grant programme in course redesign in USA higher education, which claimed both improved learning and reduced costs through the introduction of technology enhancements. This paper considers how these claims influenced the UK initiatives, and how divergent strategic considerations led the national programmes to be defined differently. A conclusion is that the way the initiatives were framed has influenced their outcomes. However, both programmes have succeeded in building a cross-institutional level of capacity development that offers a policy direction for the future.

Reflecting Education, Vol 4, No 1 (2008), 6-16; You take the high road ... (PDF)

Reflecting Education, Vol 4, No 1 (2008),  Table of Contents

Response to invited peer commentaries by Terry Mayes and Derek Morrison

26 March 2009

The peer commentaries offer some interestingly different perspectives on the arguments advanced in the paper. In considering the points raised we acknowledge the value of this peer-commentary form of publication. There is no doubt that the peer commentaries have forced us to sharpen our thinking about the national programmes described, and to clarify some points that were left vague in the original text.

First, Carol Twigg, Betty Collis and Jeff Moonen, and Jane Plenderleith and Veronica Adamson, have all commented on our comparison of the UK initiatives with the Pew Grant programme. The main argument with our paper is that because of the major differences with the UK initiatives the Pew program does not provide an appropriate comparison. We accept all the points made concerning the differences – that there is no comparable history of such initiatives in the US, that they have quite different goals, that Pew Grant was essentially a taught programme while in the UK initiatives the methods for change are self-determined, and so on.  In fact our main justification for including Pew Grant in the paper was that Twigg’s reports and invited presentations on that initiative had figured prominently in setting the immediate policy context for the Scottish e-Learning Transformation (ELT) Programme. This was not really the case for Benchmarking and Pathfinder, whose policy background lay more in HEFCE’s own previous initiatives, so we were almost certainly on shaky ground in trying to discuss all three programmes together.  Collis & Moonen have proposed a comparison that would seem more relevant than Pew, i.e. with the EU programmes aimed at developing e-learning in HE. We are inclined to agree, although we were disappointed that the report they point to was not more helpful in pulling the (to us) bewildering range of EU funding programmes into some kind of coherent picture for HE. In the end, the attempt to shape arguments around the three programmes we have described, and the insightful peer commentaries on the paper, has clarified for us some key questions that need to be asked of all such initiatives if we are to gain a deeper understanding of their purposes and effects.

1. What is the unit of change?  The three initiatives do indeed differ significantly in their intended targets, from first year courses with large classes (Pew), through individual institutions (Pathfinder), to cross-sectoral collaborations providing exemplars for the entire post-compulsory sector (ELT). Here, we can see that while PCR (Pew Grant Program in Course Redesign) essentially involved demonstrations of how particular interventions can have a particular impact on a particular course, Pathfinder and ELT encouraged institutions to try to shift policy across the board. Even within ELT and Pathfinder the unit of change varied considerably. Some Pathfinders (e.g. University of Leicester, University of Hertfordshire) encouraged a number of PCR-type projects within their own institution. In general, the larger the unit of change the harder it is to pin down the impact, while the smaller the unit the more difficult it is to generalise across subjects and institutional contexts. Pathfinder institutions were encouraged to come to their own decisions about the scale of their changes, and were supported in the process of evaluating the changes. Particular conclusions from Pathfinder projects were distributed across the programme in the form of briefing papers aimed at particular audiences defined by the projects themselves.
2. What kinds of outcomes are sought?  Here we can see that the initiatives differ in the defined specificity of possible changes. On the one hand is a highly prescribed set of measured outcomes for learning achievement and cost of instruction (PCR), with participating courses chosen on the basis of their perceived readiness to meet the stated criteria for achieving those outcomes through course redesign. In complete contrast, Benchmarking/Pathfinder was a process of support for an institution to come up with its own intended outcomes, and then support for it to start to implement its chosen approach to development. A key criterion for selection in this case was simply the demonstrated commitment of the institution to engage with the process. There was no requirement for an institution to adopt centrally prescribed goals – only that the goals selected were consistent with the institution’s overall strategy for enhancement. Somewhere between these two was the Scottish initiative, with a very broad requirement for transformation, but with selected projects that defined and implemented their own versions of what this meant, and to whom it would apply. Despite the suggestion that it was the PCR that seemed out of line with the UK initiatives, in some ways the UK initiatives themselves were equally as different from each other. All three initiatives can be seen as designing interventions aimed at the enhancement of the learner’s experience, although, as Collis and Moonen imply, some of the Pathfinder projects seem further back in the ‘pedagogy chain’. In Pathfinder, informed by benchmarking, the focus seems more on organisational change aimed at institutional procedures, raising the level of awareness of activity-based and technology-enhanced pedagogy across the institution. In the ELT, and in PCR, such awareness was assumed in the project proposal. These differences seem important because they influence the locus of ‘ownership’.
3. To what extent are the outcomes generalisable? In Twigg’s (2003)review of the original ten institutions in the Pew programme, the claim is made that ‘by using technology-based approaches and learner-centred principles in redesigning their courses, these ten institutions have demonstrated a way out of higher education’s historic trade-off between cost and quality’. This outcome of the PCR programme seems so general and so important that an absolutely key question concerns the extent to which the PCR approach will transfer to the different culture of UK HE.

Carol Twigg led the PCR and her comments refer to the continuing important leadership role that the organisation she leads plays in moving beyond the original work. As she herself points out, the Pew program is evolving into something that is being called a ‘movement’ in the US, and is engaging institutions who do not necessarily have the profile of institutional readiness that we emphasised in the paper. We suggest that the PCR ‘movement’ is promoting course redesign in a way not dissimilar from that seen in the Pathfinder programme – towards an activity-based form of pedagogy, where the use of technology is used in a blended way, integrating online learning with face to face methods to a point where the focus is on the learning rather than the ‘e’. Indeed, the PCR is, as Twigg reminds us, not an e-learning transformation programme at all. In all the examples technology is used to scale the pedagogy so that improved learning outcomes can be enjoyed by large numbers of first-year students at reduced cost.

Let us try to examine this more closely. In general, activity-based, enquiry-based pedagogies succeed by ensuring that it is learners themselves who do more of the work. On the face of it, constructivist learning should reduce the need for instruction – shifting some of responsibility for effort from the academic teachers onto the learners themselves. Harvey Mellor raises the interesting question of how we should regard student work. Is it actually desirable that we should try to achieve particular learning outcomes in fewer notional study hours? If we adopt wider goals for learning than simply the passing of content-based assessments then we might argue that a valid educational goal would be to encourage students to study for longer – reaching deeper levels of understanding, exploring the subject more widely, sharing their learning in discussion with peers. There is something of a paradox about learning that makes it different from all other kinds of work – you don’t actually want to make it easier in the usual human factors or automation sense or you lose the very thing that makes it effective – the hard cognitive restructuring that leads to real knowledge (see Mayes & Fowler, 1999). Indeed, there is a groundswell of concern being publicly expressed by some academics in the media that some of the apparent efficiencies offered by technologies may inadvertently be inhibiting, not facilitating, such cognitive restructuring and that this is impacting on more than just the so called “digital natives”, e.g. see the transcript of the recent Analysis programme (Clever.com, BBC Radio 4, 12 March 2009).

Reducing the need for instruction seems less problematic than reducing learner effort. This is still, however, a potentially misleading simplification since it begs the question: what is meant by ‘instruction’?  Kirschner et al (2006) present a case for concluding that ‘constructivist , discovery, problem-based, experiential and inquiry-based teaching’ doesn’t work, and they argue that there will always be a need for the delivery of good expositions of subject matter. They seem to reach that conclusion, however, by considering only versions of constructivist pedagogies that involve unguided discovery learning.  We are aware of few approaches in UK HE that would involve no guidance at all, or even the minimal guidance discussed by Kirschener et al. The ‘instruction’ that is reduced by constructivist approaches is the deliberate delivery of information, not the guidance that should follow from learners’ attempts at self-explanation. So the trouble is that getting students to do more of the activity which should underpin deep learning doesn’t necessarily free up teaching time at all, since activity-based learning places a premium on scaffolding, which in most subjects means tutor feedback.  In fact the normal consequence of adopting enquiry-based methods is an increase in tutor effort, not a decrease at all. It is the highly cost-effective large class lectures that would more typically be reduced, as the academic teacher’s role moves away from the delivery of information towards learner-centred guidance. So it is hard to see how redesign – in the direction of active learning – can possibly lead to reduced costs.

In general, the debates about how to provide feedback in a cost-effective way resolve themselves into two main issues: to what extent can online tutoring (including formative assessment) be ‘smart’ (removing or reducing the need for human intervention), and to what extent can effective scaffolding be achieved through tutors who are not full academic staff, and may be peer tutors? Both are attempts to scale to large classes the need for individual feedback that activity-based pedagogies require. Both possibilities are evidently easier to achieve in some subjects (e.g. mathematics) than in others. Then there is another layer of complexity introduced by the need to provide scaffolding that will be gradually withdrawn as learners advance in their competence and confidence as learners. So the stage of learning is important as well.

Can tutoring really be carried out effectively by slightly more advanced peers? The cognitive science work on tutoring by Chi et al (2001) has demonstrated that a key aim of an effective tutorial should be to help learners generate their own explanations, then for the tutor to respond to those explanations with feedback. This, as Chi has demonstrated, is a demanding task requiring a sophisticated mix of subject expertise and understanding of the learning process itself. The more that we research into the process of tutoring, the clearer it becomes that for quality of learning it matters a great deal how the tutoring is carried out. In the reports generated recently by the Scottish quality enhancement theme for the first-year experience (see Mayes 2009), there is a clear groundswell of opinion against moving further in the direction implied by PCR, where first year students are tutored mainly by inexperienced GTAs (Graduate Teaching Assistants), and where student contact with experienced academics is largely confined to the final years of their course.

In the end we feel that what the funders mean by improved quality becomes a cultural issue. In the rise of the enhancement culture in the UK we see a shift from progression and award statistics towards a wider understanding of what students will get out of their courses, namely, something more like ’academic’ or ‘learning’ literacy. There is increasing recognition of the importance of preparing students more effectively for the HE experience, so that they will more readily engage with their responsibility for active learning. We also see a gradual acknowledgement by the HE institutions that the institutions themselves have a responsibility to try to empower their first year students with the skills, attitudes and confidence that will allow them to get the most from the HE opportunity they have. This more generic and holistic view of outcomes would not align well with the idea that quality in first year classes  is improved when automated testing shows a higher score on specific content knowledge, or that the teaching was more efficient when experienced academics have been substituted by less experienced, and cheaper, staff without producing lower scores on a test.  Twigg (2003) in describing the PCR case study at Penn State commented as follows about staffing substitutions ‘..also used ULAs to grade homework assignments, relieving GTAs of this chore’  This seems to raise the quality issue in a stark way. Within the goals set by the PCR it is perfectly reasonable for a straightforward grading task, where answers to problems can be graded without the need for more personalised feedback, to be carried out by senior undergraduates (in this example it would seem even more appropriate to automate the grading). In some subjects we might even agree that improved grades on such tests would indicate something at least correlated with improved quality of learning, but they would not be addressing the wider issues about quality that now dominate UK debates about the first year experience. While the PCR would be expected to enhance the learning experience through more learner-centred constructivist methods, especially where supported by academics who may have been released to spend more time with individual learners, nevertheless to our minds the enhancement produced by a constructivist redesign is limited by an instrumental first-year culture in which quality is measured by conventional tests.

Finally, we acknowledge the point made by Seb Schmoller – that the real value of these national initiatives may ultimately lie in their unintended consequences. By engaging large numbers of participants with the issues discussed above, the national programmes contribute in a lasting way to an overall shift in the quality culture of HE. A shift towards, as Richard Hall puts it, with characteristic gusto, “a pedagogy of hope: a 21^st century progressive pedagogy that moves us away from constricted, subject-driven views of the curriculum, towards one that captures the student’s non-educational, informal critical development and brings its outcomes head-on into formal education”.

References

1. Kirschner, P.A., Sweller, J. & Clark, R.E. (2006) ‘Why Minimal Guidance During Instruction Does  Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching’, Educational Psychologist, 41:2,75-86. 
2. Chi, M.T.H., Siler, S.A., Jeong, H., Yamauchi, T., & Hausmann, R. (2001). Learning from human tutoring. Cognitive Science, 25, 471-533. 
3. Mayes, J.T. (2009) Overview of the First Year Enhancement Theme 2006-08: The aims, achievements and challenges. Downloadable from http://www.enhancementthemes.ac.uk/documents/firstyear/FirstYearOverview.pdf 
4. Mayes, J.T. & Fowler, C.J.H. (1999) Learning technology and usability: a framework for understanding courseware. Interacting with Computers 11, 485-497, 1999. 
5. Twigg, C.A. (2003) Improving learning and reducing costs: Lessons learner from Round 1 of the Pew Grant Programme. Troy, New York: Centre for Academic Transformation.