40 Tips for running an Open Online Course or MOOC from those who have experienced them

To coincide with the announcement of a new Learning Design MOOC (now confirmed as starting in January 2013) some of those registering have been invited to tell us about their previous experiences of participating, or helping organise, open online courses. Information about MOOCs can be difficult to come by, so the thirty-five responses we have received so far (which cover 50 separate experiences of 25 different MOOCs) represents a really interesting dataset.

In the survey we asked people about what they had liked and disliked about the MOOCs they had been involved with, what the MOOC did well and not so well, how the issues they had encountered could be resolved and what advice they would give to others planning a MOOC. I’ve taken these responses and attempted to express these as a series of tips – each tip based directly on one or more pieces of advice, suggestions or experiences reported by the survey respondents.

If you would like to add your experiences to those already contributed, please visit the survey. Also, as this survey represents the first stage in a broader schedule of monitoring and evaluation for the MOOC, watch out for further reporting over the next six months.

So here are 40 Tips for running a MOOC (in no particular order)…

1. Be clear about what type of MOOC you are planning  – there is no one way of doing a MOOC for, just like any course, it may draw on constructivism, instructivism,  connectivism etc. (or any combination). Be clear about the pedagogy even if just within your design team.

2. Ensure a balance between the suggested activities (those you have designed) and time allowed for participants to develop/build from them

3. Ensure the course rewards participation – some will be motivated by the promise of awards such as badges or certificates however for others the benefit of an enriching experience, helping others, social interaction over a share interest, or better understanding themselves may be as, if not more, important.

4. Ensure the course mechanics/design focuses on achieving a coalescing or binding together of knowledge  – so as to avoid a spiralling fragmentation of posts, content and participation and generating what one respondent termed ‘too much noise’

5. Consider the expectations and skills of those who will be involved what are the prerequisite skills required? At what point might the degree of difference between participants become an issue?

 6. Provide ‘plenty of structure’ in terms of activities, group working and website organisation/navigation yet also ensure such structure is not a constraint for those who want to go further or to reorganise knowledge in different ways. Consider the negative impact on motivation some feel during scripted courses and, conversely, others feel during very ‘open’ courses

7. Consider future participants as well as current participants will the post, tweets, links etc make sense to someone new visiting the site in 6 months time? Will the course really be ‘open’ across time as well as open geographically at the time it takes place?

8. Expect a high-drop out rate low barriers to entry may also mean low barriers to withdrawal or lower tolerance thresholds). Take the opportunity to reflect on what a drop-out rate of 50% or 90% of registrations would mean for your course, for the participants registering on the course and your own educational mission/purpose in organising it.

9. Create a sense of closure at the end of the coursefor example, as one respondent suggests ‘an assessment, a designed activity or summary podcast’.

10. Build in the potential to scale-up (in terms of numbers), reach further (in terms of accessibility and portability) and be sustained (repeated in future)

11. Keep the pace and depth of topics higher than traditional face to face courses – and in making this change, ensure you recommend or estimate how long tasks should take

12. Determine and communicate a vision for how different forms of knowledge will be handled and privileged within the courseare some forms of knowledge more authoritative? Do anecdotal opinions carry equal weight to empirical published research?

Read more of this post

‘Preliminary findings from a series of staff surveys on perceptions, attitudes and practices of learning design’ paper now available online

Someone recently pointed out that the full text and data from my 2009 ALT-C paper titled ‘Preliminary findings from a series of staff surveys on perceptions, attitudes and practices of learning design’ (co-written with Paul Clark and Andrew Brasher) is difficult to find so I’ve just deposited a PDF version in the Open Research Online repository.

This short paper reported the findings of a series of connected staff surveys which looked at some attitudes and perceptions about issues associated with learning design. This included asking staff if they agree/disagree with the following eight statements:

• ‘The advent of elearning is making the process of creating courses more complex’
• ‘I sometimes feel overwhelmed by the challenge of how to effectively integrate ICT in a course’
• ‘There is a pedagogic need for better integration of learning technologies (e.g. VLE) into courses’
• ‘It is becoming harder to understand how all the parts/components of planned learning and teaching fit together’
• ‘I do not find it difficult imagining how online learning content fits with other course content’
• ‘Technologies available today offer great potential for enhancing the student learning experience’
• ‘Understanding the relationship between pedagogy and learners’ activity is a priority for me’
• ‘Do you believe that more use of visual representations (that show what is to be learnt and how) could help students better understand and plan their study’

These statements were derived from several claims made in the learning design literature about the potential role (and need) for more designed approaches to learning and teaching and ongoing OULDI work at the time (semi-structured interviews, workshops, design observation, and focus groups site). The survey was administered to selected staff in 2009 and 50 members of staff responded.

Whilst this data is now a couple of years old, it is an interesting record of how staff responded to some key questions that are still being asked today: Has the advent of new technologies has created a much more complex situation in the design and planning of teaching and learning? Is this perception widespread across staff? And what is the perceived role for visualisation in the design process?

For example, take the first of the eight statements

Read more of this post

Describing Teaching and Learning Innovation

In this post I want to propose three ‘lenses’ that may be useful for describing and understanding the nature of a teaching and/or learning innovation. Innovation is certainly a prized commodity; deemed important enough to use to structure formal university module development and delivery processes, and to trade across and between practitioners and instititions. We also know that the question of innovation is likely to be an important consideration early on in the process of creating, designing and teaching a module or learning activities (and hopefully later on in the process also!). 

So how do teachers talk about and represent their innovation? I have been involved in a couple of projects that sought to to capture innovation and, like similar attempts elsewhere, found it difficult to define what innovation was. The conclusion is, of course, that innovation is a nebulus term embracing everything from a paradigm shifting idea to borrowing one from a colleague and using in a class. Broadly speaking, one could suggest that innovation is anything that creates new possiblities by combining different knowledge sets that should result in positive change or added value. This expectation for there to be a positive result I think distinguishes innovation from other terms such ‘something different’ or ‘ something new’  and prevents it being abused to the extent that  ‘being innovative’ becomes an aim in itself.   

So if innovation can be many things, do we need to be more precise in how we talk about it? Perhaps, like for other aspects of learning design, it could be helpful to have some lenses with which to view, describe and define it. Fair enough, each lens will present only a partial representation, but together such views may help better understand the character of the innovation and to use and evaluate it effectively.

LENS 1: For who is the innovation innovative? Each individual, group or community who experiences the innovation will likely experience it differently: what is innovative for one module author may not be for their students who did it last year on a previous course; what is innovative for the general way a subject is taught may actually be ‘common practice’ for a ‘leading-edge’ faculty or university; what is innovative in one faculty may not be so across the univeristy.

So this first view represents seven key stakeholders in the educational innovation. When selecting these seven, I looked at the ways practitioners (in interviews or in written documents) spoke about who their innovation was innovative for. I also thought it important that the diagram be organised in terms of the scale/size of the group effected (note the progression from the individual (on the far left) to the entire sector (on the far right)). Arranging the representation in this way should make it is easier to see how extensive or local the innovation may be. 


One way of using this view would be to determine how innovative the innovation is for each stakeholder and assign a score from 0-5. These scores could be written in the blanks provied. Scoring may Read more of this post

Assessing the potential for adaptation and personalisation of teaching and learning

Over the last few months I’ve been working with Simone Davies at the OU’s Learning and Teaching Solutions unit on (a) an online guide to help course authors use the study planner in Moodle to build adaptive elements to their learning content and designed activities (b) a review of current use of the study planner in selected courses (c) developing a specification for a new Moodle Study Planner ‘Decision’ plug-in (d) a review of the nature, benefits and barriers associated with adaptive learning and teaching (adaptive learning systems) based on staff interviews and a literature search. I touch upon (d) and a little of (c) in this post.

In 2008, Kelly and Tangney’s discussion of adaptive learning systems notes that they ‘can bring different benefits such as increased learning performance, greater enjoyment, enhanced motivation and reduced learning time’. In outlining this quartet, they hint at just how useful adaptation could be in an online distance teaching context. However, to date, the realisation of properly adaptive teaching in distance learning and the planning for adaptation in their design remains arguably quite limited.

If there is indeed the potential to make improvements to workload, to better engage and empower students in their own learning, to increase retention and to help students achieve more, then it is imperative that online and blended learning understand when and how adaptation can become an effective teaching tool could be used.

Of course, designing adaptation requires a departure from the linear, single path a learner is taken through a course, be this across a lecture series, a distance learning guide or a series of online learning tasks. Reigleuth makes the point that ‘one size does not fit all’ and indeed, in 1999, the materials for the OU’s own course on Teaching in Higher Education stressed how students learn differently and proposed as good teaching practice that the teacher understand and respect their student needs, approaches to learning, stage of learning development, interests, motivations, backgrounds, prior knowledge, etc. Read more of this post

Introducing learners to open distance learning: a new 1-2 hour study module

The openED online open distance course titled ‘Business and Management Competencies in a Web2.0 world’ commences its third presentation next week. As before, I’ll be involved in the research and evaluation of the course and the use by distance open learners of the website.

For this round, I’m also written a new additional module titled An Introduction to how to learn with open distance courses such as openED.  The aim of this module is to help the learner understand how to study better using open online material and should last 1-2 hours. This module is based on the most common questions asked by openED participants which I have identified from a detailed review of the Round 1 and Round 2 forums and over fifteen hours of text-chat session transcripts. The module uses examples from the openED course to help illustrate ‘real-world’ examples and includes three short reflective exercises (5-10 minutes) aimed at thinking how using online social spaces such as forums can enrich learning experiences. The module includes short sections on:

  • What is open distance learning?
  • When to start?
  • How to study an open module
  • Using forums to meet others, share work and resolve questions
  • Live text-chat sessions
  • Group working
  • Getting feedback and recognition for your work

To see the module in situ check out the openED website. I have also made the original text document (3,600 words) available under a limited creative commons licence for reuse and repurposing. You can download it using this link: Introduction To How to Learn With Open Distance Courses_Cross_2011_v6  

One exciting feature of this module is my sharing with the learners a basic visual representation of the recommended learning route (learning design). I have of course had to abstract the design to suit any of the 10 mondules (each differs in content) so it does not convey the detail or nuance that I would often put in such designs. 

 Image of the openED Task Seuqnece diagram

The intention is that participants work through the Introduction module first. This will better prepare them for the course modules and we will monitor how this contributes to greater or more prolonged engagement. Read more of this post

Altruism and OER /OED: sharing, charity and the process of giving

Read any article about open educational resources or designs and the term ‘sharing’ pops up time and again. Whilst an apparently innocuous, self-explanatory term, for something so closely bound to notions of openness of why we think people want and are open and of how we imagine the mechanism of interaction and co-creation of common resources, it is often evoked with scarily little critical reflection. I propose here that it can be a misnomer and, as such, is operating as much to obstruct our analysis and critical understanding of the mechanisms for enacting openness.

The idea of sharing refers to what could be termed ‘reciprocal altruism’ – a form of altruism where there is anticipated to be exchange of things, often of those judged of broadly similar value. This notion of reciprocal altruism, often with an emphasis on the interaction that takes place not reasons for it, forms a cornerstone of how many people imagine how open education systems will work and should be configured; designers exchange designs, students review each others work, teachers share their resources. Ask why they think such interaction will take place and either the fall-back position of reciprocal altruism is assumed accepts begrudgingly accepts conscious self-interest.

Yet psychological, sociological and economic research tells us that there is more to altruism than reciprocity or sharing. Altruism for example can also take the form of kin selection, or nepotism, with individuals supporting others seen as similar to themselves thereby binding communities closer together. It can also take the form of charity – the act of giving to a stranger; the understanding of this is fundamental to entire sectors!

It seems to me that focusing on the act of ‘giving’ – looking at individual and structural motivations, intents, behaviours and social norms – can enrich our models of OER use. What if we look to interpret contributions to OER repositories or other open learning and design repositories as acts of altruism such as charity or kin-selection? Can we use such perspectives to build a complete picture of what openness is or could be like?

So why do people ‘give’? There are many overlapping theories. One perspective holds that people give to feel good or better about themselves (even to remove a sense of guilt or obligation). This ‘warm glow’ comes from Read more of this post

Pedagogies that travel and those that don’t: Sustainability in open courses and learning design

Last week I presented a paper titled ‘Some Issues Affecting the Sustainability of Open Learning Courses’ written by James Aczel, myself and four other colleagues (2011) working on the EU funded openED project at the EDEN conference in Dublin. The aim of the project is to explore participatory learning in open educational environments by producing and delivering an online open/’free’ course is business management. In so doing, the objective is to not only to produce an OER, but to use open technology, open Web 2.0 methods of communication, open opportunity to facilitate, open peer assessment, and open opportunity to build a resource for reuse. So a key question for the project is to study the effect and issues around ‘opening’ up many aspects of a course together. And one of my roles in the project is to develop and deploy a complex of evaluation methodologies before, during and after each of our three rounds.

The theme of the conference was learning and sustainability and it was interesting to see the present state of thinking around how e-learning was seeking to frame and understand what ‘sustainable learning’ means. The two uses of the concept most prevalent were: 1. The planning to sustain project accomplishments or innovation, and 2. The teaching of themes associated with sustainability within a course. The first is clearly important to the longevity of a project but I find this Read more of this post

Open Innovation and the ‘Next Generation Stage-Gate’ process: Reflections on Learning Design processes 2

Next-Generation Stage-Gating

The Stage-Gate concept was developed by Robert Cooper as an innovation management method for businesses. It is now a widely used mechanism to manage the product development process – in the case of a University this may be the development and production of a course. The object of the model is to support the making of a key decision at each stage – the question: should development continue?– or as Cooper puts it; to ‘Go’ with or ‘Kill’ it. For Cooper, the period between Gates consists of ‘information gathering activities’ followed by an analysis of the result which is then entered in to the Gate for decision.

Over the last five years some work has begun to assess and revise the Stage-Gate model. Indeed, Cooper (2008) himself acknowledges the need for the‘Next Generation Stage-Gate’. Reading through this work, I was encouraged to find comments and observations that aligned with themes discussed in my last blog post. Cooper (2008) offers a range of ways to revise or add to his model. I’ve picked out seven below:

  1. A Flexible Process: we observed (see last post) that learning design activities should be planned in advance of the stage commencing and here Read more of this post

Reflections on Learning Design Process Models 1

Over the last month, Rebecca Galley, Paul Mundin and I have been reflecting on a range of our recent meetings and project related activities (including meeting with colleagues from Birmingham City and Gloucestershire, with our external partners, and OU colleagues). This post shares some of this ‘reflection-in-progress’ and as such uses languages familiar to those exploring new structures for curriculum/learning design processes.  

Here are two different draft representations to help communicate our interpretation of recent work. I think it important that such models seek to weave together the less formal practices and processes (such as design activities) with more formal and necessary demands:

If you click on the images an enlarged version should load.

By way of explanation, these representations were the outcome of our thinking about a range of questions. Some of these were:

  • Does a business case, or any related business approval stage, need any pedagogic information or design? – Possibly not. Could learning design be more damaging than enabling at this stage? – Very possibly (after all the design process has not yet properly started and decisions made here may constrain and limit design options
  • Where in the process is the learning design problem space considered? And how much planning of learning design activities (both formal in the design team, and semi-formal individual work)? The business case may set some constraints but little in respect to important pedagogical aspects associated with the design problem. Recent work in the wider design literature has highlighted the importance of fully understanding the ‘design problem’ before developing a design solution. A number of advantages are cited from promoting a more creative, innovative approach to the problem; to better understanding of the core purpose of the design process and more effective quality assurance and design validation.
  • How important is it to recognise that design stages are not discrete units that ‘end’ once a particular form or document has been submitted. It is not a linear process for each overlaps and may continue to be worked later in the process with more porous links to it’s neighbours (shown by the dotted lines above). Furthermore, it would now appear course development involves a greater range of people throughout the entire process (rather than each just involved in a discrete portion of the process).
  • Is too great an emphasis placed on the submission of a form or specification before work moves on? Instead why not focus on the process of creating these forms and specifications and use the cumulative amount of ‘work’ put in to preparing these documents as indicators of ‘sufficient’ design? Here the notion of doing a ‘sufficient’ amount of design work replaces that of producing a document/forum ‘sufficiently’. Yet how could we record and store documents and design activity ‘evidence’? The most obvious is some form of portfolio/s (e.g. folders on a local or shared area). The diagrams indicate some of these portfolios and each module would need to decide, upfront, what sufficiency may mean for them.
  • Is there enough formal review of the meso- and micro- levels of design? Module design currently takes place at several inter-connected levels, however often formal institutional process fade as design work in earnest begins.
  • Is there sufficient recognition of the micro-design taking place in the production of course materials and services even once courses have been written or operational documents prepared? Maybe not, and a consequence does this mean that micro-design decisions may not be properly supported or articulated?
  • Module/course evaluation (after delivery of the course) is often well formalised, however, can this be enhanced by better definition of the design problem space and planning of activity for the design process at the beginning? There is certainly potential for further quality assurance measures here and this may enable better tracking of decisions and therefore of accountability.

Benchmarking Assessment (conference paper): breaking down barriers and building institutional understanding

Denise Whitelock and myself have recnetly been working on a project to identfy key meaures (or criteria) of assessment processes and practice. The aim to to develop a benchmark instrument for use in better assessing and understanding assessment practice in Higher Education.

As part of this work, I am presenting on our behalf a paper at the Computer Assisted Assessment Conference in Southampton this week. In the abstract we make the point that benchmarking offers a comprehensive way of measuring current practice in an institution per se; whilst also gauging achievement against external competitors. It would appear that although e-learning has been benchmarked with a number of universities in the UK and abroad no one to date has tackled the area of assessment; which is now becoming of more concern with the advent of e-assessment. 

Our paper describes the construction of a set of benchmarking measures/indicators and the outcome of early pilots which combine a survey instrument and semi-structured interview methodologies. The findings suggest that building a comprehensive and robust core of benchmark measures would have great utility and value to institutions; not just in external benchmarking but in internal benchmarks and reviews, setting baselines, exploring the student experience, providing staff with data meaningful to their role and professional development and supporting continuous improvement.

The paper is accompanied by our current working draft of the benchamarking measures. [Since writing this post the measures have been revised further. A new version of the measures is available at: Assessment Benchmarking Criteria v17_A]. We will be very interested to hear your comments or feedback on the paper and the draft.