Outcome of Learning Technologies Review

In February, MMU’s ICT Strategy group and Executive approved a set of recommendations for upgrading the university’s learning technologies to support a step-change enhancement in the quality of assessment and learning for students and staff.

Proposals were developed by the Learning Technologies Review project group, steered by the DVC Student Experience. The group’s proposals were informed by HEFCE and MMU policy, advice from JISC and Gartner, evaluations of current systems, focus groups, faculty road-shows, email suggestion boxes, user testing of possible solutions and input from Academic Development Committee. The group presented its findings through a set of scenarios, which set threshold aspirations for staff and students. The scenarios were endorsed by MMU’s ICT Strategy Group and used to review learning technology options in terms of:

  • Fitness to vision
  • Fitness to transformation timeline
  • Robustness and scalability
  • Value-for-money
  • Avoiding lock-in

After detailed consideration of a range of options, the ICTSG decided that a vision of joined-up systems in which a core learning system could be extended through an expanding range of supported tools, would be best served by:

  1. Creating a new Virtual Learning Environment using the open source Moodle software with the Equella digital repository, hosted by University of London Computing Centre with an MMU-based Disaster Recovery facility – the timeline for implementation will be:
    • WebCT Vista will remain the institutional VLE for the 2010/11 academic year
    • Moodle will be available for a limited pilot from September 2010
    • All Units for students starting from January 2011 onwards will be on Moodle
  2. Upgrading the Library Reading List system to the new Talis Aspire system for September 2010
  3. Upgrading the pilot Apple Podcasting system to a production version for September 2010
  4. Developing proposals for a minimum technology specification for all teaching rooms
  5. Allocating a budget for a further year of trials of e-Portfolio systems
  6. Creating a focused organisational development initiative on digital literacy for academic staff
  7. Reviewing other learning technologies, such as synchronous conferencing, in January 2011

More detail is provided below on the context and detail of these decisions.

Background

In 2008, Learning and Research Information Services (LRIS)’ planning activities identified the need for a review of MMU’s learning technologies. The notion of a Learning Technologies Review was subsequently approved by the ICT Strategy Group, and a Project Steering Group established chaired by the Deputy Vice Chancellor for the Student Experience, supported by the Director of LRIS and the Head of the Centre for Learning and Teaching (CeLT). The Head of Learning Research Technologies, Mark Stubbs, and Principal Lecturer in Learning Technologies, Neil Ringan, were asked to lead the project. A scoping exercise identified an immediate need for proposals on:

  • the institutional VLE and podcasting;
  • classroom technologies;
  • e-Assessment;
  • e-PDP/e-Portfolio;
  • Information, Advice and Guidance arrangements and
  • analytics to support student retention and success.

As the project progressed, details of our findings and consultation activities were published on this blog.

Review Context

MMU has a long history of exploiting the benefits of technology for learning, teaching and assessment. Enthusiastic colleagues have worked with early releases of the WebCT Virtual Learning Environment (VLE) software since 1998 (Forsyth 2002). Developments took a significant step forward in 2006 with the introduction of the Managed Learning Environment (MLE) project, which brought technology-supported learning into the mainstream through an institutional VLE (WebCT Vista) that was driven by the student records system, and a new focus for staff development activity that involved working with programmes and departments, rather than enthusiastic individuals. In less than three years, use of the VLE climbed to over 1,300 staff and over 30,000 students; our core delivery now depends on the VLE, and a robust service is essential. As software and hardware for our current system approach end-of-life, we felt it prudent to review our learning technologies so that we could give a clear roadmap to our staff and students about what they can expect going forward.

It is important to acknowledge some policy shifts since the MLE project was established. At the national level, HEFCE’s 2005 strategy for e-learning has been revised to focus on enhancing learning, teaching and assessment through technology (HEFCE 2009). This shift reflects growing recognition of the potential for technology to support learning and related processes, growing recognition that education institutions are not the only ones making such technologies available to learners (see, for instance, CLEX 2009), and a growing desire to normalise use of technology as a way to enhance learning. The new message to institutions is to pursue the developments in infrastructure and staff development that ensure that the technology becomes an enabler for enhancement, rather than a focus itself.

Locally within MMU, our ambition to become the leading university for world-class professionals is now manifest in a Strategic Framework for Learning, Teaching and Assessment (MMU 2009A) and Threshold Standards for the Student Learning Experience (MMU 2009B). We share HEFCE’s optimism that technology can assist the pursuit of service excellence through these initiatives, and note the increasing emphasis given to explicit measures of the student experience and student success in competition for students and funding (BIS 2009). We also note the extremely challenging economic context in which curriculum innovation and service enhancement must take place. Our investment in learning technology must be value for money; it must reduce the burden and frustrations of learning-related processes; and it must support transformation in the quality of academic life for staff and students.

Decisions about our learning technologies will be taken not only in the context of an ambitious change agenda in a challenging economic climate, but also in a shifting technological and cultural context, which has revolutionised other industries, such as music, travel and publishing. The learning technologies market is witnessing significant turbulence: the leading commercial VLE vendor, Blackboard, has bought major rivals WebCT and Angel, and must now move its diverse customer base to a next generation product (BB Learn 9) to reap the benefits of a consolidated product line. The open source Moodle software has seen spectacular growth in adoption, particularly in the further education and schools sectors, and most recently in the UK amongst former WebCT universities such as City, Coventry and Exeter. However, its next generation release (Moodle 2) is a complete re-write and its release date has been put back with rumours that 1.9x customers will have to wait until Moodle 2.1 for a stable version with feature parity. Aside from the Blackboard versus Moodle dilemma, institutions are also contending with the prospect of education-oriented cloud applications from Google and Microsoft. We must choose a core system for at least three years that will give staff the clear roadmap they seek, but that system must be capable of extension and integration; the pace of change means this is not the time to lock ourselves in to a particular technology. We must consider the ease with which we can connect into and out of the core system, and the ease with which we can move to and from that system.

Review Criteria

The technologies we chose must deliver the right things (fitness to vision) at the right time (fitness to transformation timeline) in the right way (robust and scalable) at the right price (value-for-money) whilst preserving our capacity for change (avoiding lock-in). ICT Strategy Group colleagues accepted our recommendation to review possible options in terms of:

  • Fitness to vision
  • Fitness to transformation timeline
  • Robustness and scalability
  • Value-for-money
  • Avoiding lock-in

Scenarios

Our vision for transformative learning technologies has been informed by HEFCE and MMU policy, guidance from the JISC and the Higher Education Academy, knowledge of activity at other institutions, knowledge of the ways in which we have used systems in the past, and consultation with colleagues and students through focus groups, road-shows and email suggestion boxes. Details of this emerging picture have been posted to this blog and were expressed most recently as a set of scenarios (reproduced here for convenience).

Scenario #1

Sally, a senior lecturer, checks her diary and is reminded of her 11am first year tutorial. She checks the VLE to see which students in this tutorial group have engaged with the preparation activities she set and decides to print a summary. She disconnects her netbook from the large screen and keyboard in her office and walks down to the tutorial room via a printer, where she swipes her staff card and collects the list. In the tutorial room she connects her netbook to the large screen and talks her students through the activity she planned before dividing them into four groups. While the activity is progressing the tutor has a quiet word with each of the students about their engagement with the tutorial preparation and notes that two students are missing. In the tutorial the four student groups prepare mini-presentations to summarise their findings using netbooks that some have brought with them; they use the wireless network to upload their mini-presentations to the VLE and take turns presenting, while their colleagues take notes – some on paper, others use electronic annotation and note-taking software on devices they’ve brought to class. After the class, the tutor logs details of the two students against the tutorial they missed, knowing that the Year Tutor will contact them if their overall engagement has fallen below the threshold expectation for the course.

Scenario #2

Jaspreet, a second year engineering student, signs on to Windows Live using the gaming laptop he keeps in his flat; he sees an alert reminding him of an assignment deadline later in the week and decides that he’s going to take advantage of the unlimited texts package on his phone to receive Windows Live alerts by text in future. He’s made a start on the assignment, but wants to check the brief again so he logs on to MyMMU. He scans the latest news from each of the Units he studies and sees that his tutor is offering online assignment support between 2pm and 3pm – he creates an appointment in his phone to remind him. He sees that a new list of revision podcasts has been set up for another of his Units so clicks on the link, which launches iTunes so that he can subscribe. He docks his iPod Touch, which automatically picks up the first audio briefing about the upcoming exam. Jaspreet listens to the briefing on the bus on his way into university. When he arrives he uses Skype on his iPod Touch to call a friend to meet for coffee using the new microphone-headphones he treated himself to recently – he’s pleased his new purchase works. At 1:45 Jaspreet’s phone reminds him of the assignment support session at 2pm, so he checks the display screens and sees that a nearby IT dropin zone has free machines so he heads there. He logs on to the VLE and joins an instant message chat with the tutor in which several of Jaspreet’s classmates are asking questions about the assignment. Jaspreet watches the questions and answers, which cover most of his concerns, and asks a specific question about the relative weighting of conclusions versus the detail of option appraisal. Jaspreet decides to save the Q+A instant message thread to his SkyDrive so that it will be available later when he works on the assignment that evening back in his flat. Having synched his phone with his personal timetable, Jaspreet’s calendar shows a guest lecture this week at 4pm but at 3pm he receives a text reminder telling him the speaker is from a local engineering company. Jaspreet sees in the VLE that a couple of his classmates are online so he asks them via instant messenger whether they are going to the guest lecture. One says he’s down in the cafe and suggests they chat about the upcoming assignment before going along. The two of them chat and then decide to check out the website of the company of the guest speaker. Jaspreet and his friend gather round his iPod Touch and see that the company is recruiting for summer placements. During the guest lecture Jaspreet takes some notes on his iPod Touch that he hopes will be useful if he decides to apply for the summer job.

Scenario #3

Over the last few days Sue, a final year marketing student, has been collaborating electronically with three colleagues on a Powerpoint presentation for a group assignment. From the laptop in her house she sees from her Live@Edu collaborative workspace that Harvinder made some changes late last night to the last two slides; they look good. By instant messenger last night the group agreed that they would practise the presentation this afternoon and that Sue would book a room between 3pm and 4pm. Sue works in a bar on Thursday nights and is therefore not keen to bring her laptop with her today, so she logs on to MyMMU and searches for a room with a PC and projector available between 3pm and 4pm – she finds one free in the Sandra Burslem building and books it, giving her student number and the code of the Unit the presentation is for. She clicks the “download appointment” option to put a reminder in her Outlook diary as she knows this will synchronise automatically through Live@Edu with the calendar on her iPhone. Sue uses Live@Edu to share the appointment with the other members of her group, updates her Facebook status and heads into university for her 11 o’clock lecture on digital marketing. While waiting for the bus, she checks Twitter on her iPhone and smiles to see that her digital marketing lecturer has asked his students to identify a brand based on facts he’s tweeted about the nature, scale and success of their operations. Sue thinks she knows who he’s talking about, but wants to be sure, so grabs a chair in the social study zone on arrival and uses her iPhone to check.

In the lecture, the mystery brand is revealed and the Digital Marketing lecturer builds up a mind-map of the topic of “Search Engine Optimisation” using the online MindMeister mind mapping tool. Sue is able to use her iPhone to participate directly but is too nervous to do so. The lecturer invites those who are able to contribute topics or questions using Tweets with a hashtag of MMU followed by the Unit code. The evolving mind map is projected on the wall, but the lecturer tabs to contributions coming in via Twitter at regular intervals using Twitterfall. Harvinder doesn’t have a mobile data contract, but asks a question using her cheap and cheerful Samsung S5560, which supports Twitter over WiFi. The lecturer spots the question and asks the class for answers. He allows a small debate to develop and then suggests they settle the matter with a show of hands; he captures the outcome on MindMeister. To finish, the lecturer adds the URL for the online Mind Map to his set of Diigo bookmarks for the Unit, which appear automatically within the Digital Marketing Unit area of the VLE for later reference.

After the lecture, Harvinder and Sue decide that they’ll make some final tweaks to the presentation over lunch. Neither has a laptop with them, so Sue checks MyMMU on her iPhone to see if there are any loan laptops available from the library: there are four left. Sue books one out on her library account, and the two of them head to the social learning zone. Over lunch Sue downloads the Powerpoint from Live@Edu, adds a new graph and a reference from this morning’s lecture, and updates the file in the collaborative workspace.

A vision of joined-up systems

The scenarios illustrate how use of learning technologies does not take place in isolation: interactions with personal and administrative systems and underlying infrastructure are major determinants of the quality of the overall experience. We chose therefore to evaluate options against a vision of joined-up systems in which our core learning system is extended through an expanding range of supported tools.

We seek the convenience of a core VLE integrated with our corporate systems, ideally available to students to use with the devices and services of their choosing, and extended through tools that the institution arranges, recommends or recognises:

  • Arranged: MMU creates accounts on these tools, eg Live@Edu and Turnitin and ensures access to training materials
  • Recommended: MMU develops recommendations and supporting training materials for bringing these tools seamlessly into the core, eg using RSS to bring in content from Twitter, SlideShare or YouTube
  • Recognised: MMU is aware tutors are experimenting with these but there is not yet a critical mass of users to research and prescribe integration and training

Over time, innovative tutors and students are likely to identify new tools that could be taken on as “recommended” – support materials would be created demonstrating how a seamless experience can be established with the core; and, over time, thresholds for digital literacy would be raised to embrace the new skills required by all staff to make best use of new tools. In this way we see our learning technology base growing organically over time whilst still maintaining the convenience of a guaranteed entry point through the core VLE for students seeking consistency.

Model of extending a core VLE with arranged, recommended and recognised tools

We are aware that delivery of this vision will likely require work to other institutional systems, such as the student records system, and we look forward to this work progressing in related change projects.

Threshold Expectations

Our scenarios are underpinned by a set of assumptions about staff and student expectations of technology, and we were pleased the ICT Strategy Group endorsed these as meaningful aspirations:

  • All teaching staff will be provided with laptops (and a monitor, keyboard and mouse)
  • All teaching rooms will have a means of plugging a laptop into a large screen or projector
  • All staff and students will be able to access WiFi from anywhere within MMU
  • All students will receive a personalised timetable and assignment submission deadlines and be notified rapidly of any changes to either
  • All students will be able to book small teaching rooms when rooms are not timetabled
  • All students can expect their engagement to be monitored for active progress review
  • All staff and students can expect institutional calendar appointments and news items to be made available for use on personal devices where legally and technically possible
  • Every Unit will have a presence in the VLE

Implied Integration

Our vision of joined-up systems presumes data exchange between corporate systems. We therefore sought out solutions that would enable this vision.

System integrations and data flows for a joined-up learning environment

Option Appraisal

We considered options in four packages:

  1. New VLE (including integration, IAG, analytics and migration)
  2. Enhanced learning resources (podcasts and reading lists)
  3. Classroom technology threshold
  4. E-Portfolio/E-PDP

New VLE

MMU currently uses software from the leading commercial VLE provider, and Blackboard provided us with a detailed rationale for, and upgrade path to, its new version, BB Learn 9. However, BB Learn 9 combines lessons learned from the Blackboard classic product-line and the WebCT product-line, so to our users BB Learn 9 would effectively be a new product. We therefore decided to take the opportunity to compare this “upgrade” of our current software against the leading open source provider, Moodle. Both solutions can be run either in-house by MMU staff or by a managed hosting partner; this generated four options for consideration:

  • Blackboard In-House (VLE#1)
  • Blackboard Hosted (VLE#2)
  • Moodle In-House (VLE#3)
  • Moodle Hosted (VLE#4)

After careful consideration of the options against the review criteria, the ICT Strategy Group felt Hosted Moodle edged a close-run competition on the grounds of being well placed to deliver our vision whilst also being a cost effective, flexible and extendable place to ride out current uncertainties in the learning technology market.

For all options we are acutely aware that the success of the new system will depend on the training, information, advice and guidance available to academic staff as they move from WebCT Vista to the new system, and the way the new system is embedded in administrative business processes and linked explicitly to curriculum transformation. Detailed planning of VLE implementation in the context of delivering transformed Learning, Teaching and Assessment will be vital, as will early engagement of relevant stakeholders from faculties and central services. We look forward to taking this forward in the coming months.

Enhanced Learning Resources

A key role of the VLE is to signpost learning resources to learners. Preliminary PhD research had shown access to learning resources through the VLE to be correlated strongly with student success. The VLE repository provides a mechanism for storing and signposting files created by academic colleagues, such as PowerPoint slides and Word handouts, but this can be complemented with library materials (both physical and electronic), externally-hosted resources (such as links to YouTube videos) and audio-visual content that academic colleagues produce themselves or commission externally. Our scenarios assume that a student will be able to find all the learning resources for a Unit in one place, so the learning resource systems we provide and recommend must be capable of providing feeds that can be aggregated. Our scenarios also assume that learners will be able to access our learning resources on their personal devices and we therefore wish to exploit established content distribution channels, such as iTunes, to reach them. To deliver the convenience of a one-stop-shop for Unit learning resources and allow audio and video materials to become readily available on personal devices we identified the need to upgrade our library reading list and podcast systems.

Library Reading Lists

MMU currently uses the TalisList software to maintain Unit reading lists. The software integrates with other Talis products used for the library catalogue and book ordering, and is linked in a crude way to the MyMMU portal. TalisList is one of the older products in the Talis portfolio and has been superseded by a new product, Talis Aspire, which exploits Talis platform technologies powering the recently-launched UK Government open data site, data.gov.uk.

The ICTSG decided that Talis Aspire would be better able to deliver the convenience of the seamless integrated experience set out in the scenarios, and its collaborative platform created interesting opportunities for learners and the institution to benefit from cost-effective community recommendations: “Units like this also recommended …”

Podcasts

In response to growing interest in using audio and video materials to support learners, a pilot project has been underway using an Apple Podcast Producer server to format, store and distribute audio and video files. The Apple server has been integrated with MMU’s network directory, the WebCT Vista VLE and the MyMMU portal. However, as a pilot, the server has not been configured to be sufficiently resilient and scalable for use as a production service by all MMU staff. We therefore proposed that investment be made now to establish a production-ready podcasting service for the 2010/11 academic year. We specified that this service must also be capable of supporting iTunesU when the institution determines that there is sufficient content and ownership for a successful launch.

The ICTSG decided that the pilot Apple podcasting service should be upgraded to production status, in readiness for iTunesU, and suitable screen capture software should be installed on all staff machines.

Classroom Technology

Although it proved difficult to elicit staff requirements for classroom technologies directly, interviews for the staff laptop experiment in Humanities and AV/IT planning meetings for the new Business School/Hub building provided strong evidence of a need for teaching rooms to meet a basic threshold whereby staff can plug a laptop in to either a big screen or projector and access internet resources wirelessly. Uncertainty about whether a particular room will have the right (or in the case of some teaching rooms, any) equipment came through very strongly as a barrier to using technology in classrooms. Borrowing projection equipment was cited as too much of a hurdle by staff who often had different classes back-to-back.

We felt a threshold expectation that all teaching rooms would have wifi and large screen projection would be a major step forward in addressing these barriers, particularly when taken together with an aspiration that all teaching staff would have laptops. The notion of a threshold would not preclude specialist teaching rooms going further, such as those simulating school classrooms being equipped with Interactive Whiteboards, or those with frequent need to show the detail of physical objects being equipped with Visualisers, but the intention would be to make connection of a laptop consistently simple in every teaching room.

A full audit of teaching spaces will be necessary to determine the cost implications of meeting this threshold. ICT Strategy Group colleagues invited research to be undertaken on an effective threshold for classroom technology and a proposal to be brought forward for delivering it.

E-Portfolio / E-PDP

Feedback from the PebblePad e-PDP/e-Portfolio projects had revealed that:

  • e-PDP and e-Portfolio have distinct but overlapping requirements
  • if students are to invest in these systems, there must be value at university (for assessment) and beyond (for job and professional body membership applications), and the students must be able to own, access and control e-PDP/e-Portfolio data at university and after graduation
  • ideally, university staff would offer structure to (in the form of suggested tag lists and PDP scaffolding, such as reflective prompts) and feedback on student-owned data that can be tagged to facilitate its retrieval when required
  • e-Portfolio tools need to support the collaborative production of content in a range of media formats by groups of learners
  • e-PDP tools need to support scaffolded reflection and feedback (formative and summative) from invited tutors /peers / mentors / coaches

Whilst the systems colleagues had been using, such as PebblePad and TalentOnView, have undoubted strengths, all the e-PDP/e-Portfolio systems we explored had limitations when faced with the challenge of enabling university-provided structure and feedback on student-owned data. Experience from different subject areas also suggested very strongly that one size does not fit all for e-Portfolio.

After careful evaluation of a number of options, ICT Strategy Group colleagues decided to postpone a decision and invited more trials to further understanding of the PebblePad and Live@Edu options and consider other possibilities such as the integrated professional-skills solution developed to work with Moodle at the University of South Queensland.

Digital Literacy

Tutors’ awareness of and competence with digital technologies is seen as a critical success factor for the effective use of learning technologies (CLEX 2009). We are acutely aware that the investments we make in technology must be matched by investments in the skills of academic colleagues if we are to achieve our intended outcomes.

Learning Technologies, digital literacy and curriculum innovation

Learning Technologies, digital literacy and curriculum innovation

Feedback from focus groups and online surveys told us that students regard as very important the consistency of experience they get across their Units of study. It is therefore vital that we set thresholds for digital literacy for all academic staff and develop systematic and scalable methods for ensuring all colleagues receive appropriate training to bring their skills to an appropriate level. This will represent a major and focussed piece of organisational development drawing on expertise from across the university. Threshold standards will need to be set, a gap analysis undertaken and plans for closing the gap developed in conjunction with relevant stakeholders, ideally to coincide with deployment of the new VLE for the 2011/12 academic year. ICT Strategy Group colleagues asked the Head of CeLT to establish a project group to take this forward.

Further Work

The scoping work we undertook early on in the Learning Technologies Review project identified a number of areas in which progress was necessary to support the Strategic Framework for Learning, Teaching and Assessment. Those areas were prioritised to produce recommendations for Q1 2010 (presented above), and a list of areas to follow up later:

  • collaborative conferencing technology;
  • mobile learning;
  • electronic library;
  • creation and commissioning of media-rich content; and
  • development of an integrated staff portal

A pragmatic approach of tackling pressing concerns and maintaining a watch-list of other technologies was highly recommended by Gartner Analysts. ICT Strategy Group colleagues decided that as development with these technologies was likely to move with varying pace, it would be prudent to schedule a review for a year’s time to assess whether new projects need to be taken forward and whether the list has changed.

References

This entry was posted in News. Bookmark the permalink.