Open Innovation and the ‘Next Generation Stage-Gate’ process: Reflections on Learning Design processes 2

Next-Generation Stage-Gating

The Stage-Gate concept was developed by Robert Cooper as an innovation management method for businesses. It is now a widely used mechanism to manage the product development process – in the case of a University this may be the development and production of a course. The object of the model is to support the making of a key decision at each stage – the question: should development continue?– or as Cooper puts it; to ‘Go’ with or ‘Kill’ it. For Cooper, the period between Gates consists of ‘information gathering activities’ followed by an analysis of the result which is then entered in to the Gate for decision.

Over the last five years some work has begun to assess and revise the Stage-Gate model. Indeed, Cooper (2008) himself acknowledges the need for the‘Next Generation Stage-Gate’. Reading through this work, I was encouraged to find comments and observations that aligned with themes discussed in my last blog post. Cooper (2008) offers a range of ways to revise or add to his model. I’ve picked out seven below:

  1. A Flexible Process: we observed (see last post) that learning design activities should be planned in advance of the stage commencing and here Read more of this post
Advertisements

Reflections on Learning Design Process Models 1

Over the last month, Rebecca Galley, Paul Mundin and I have been reflecting on a range of our recent meetings and project related activities (including meeting with colleagues from Birmingham City and Gloucestershire, with our external partners, and OU colleagues). This post shares some of this ‘reflection-in-progress’ and as such uses languages familiar to those exploring new structures for curriculum/learning design processes.  

Here are two different draft representations to help communicate our interpretation of recent work. I think it important that such models seek to weave together the less formal practices and processes (such as design activities) with more formal and necessary demands:

If you click on the images an enlarged version should load.

By way of explanation, these representations were the outcome of our thinking about a range of questions. Some of these were:

  • Does a business case, or any related business approval stage, need any pedagogic information or design? – Possibly not. Could learning design be more damaging than enabling at this stage? – Very possibly (after all the design process has not yet properly started and decisions made here may constrain and limit design options
  • Where in the process is the learning design problem space considered? And how much planning of learning design activities (both formal in the design team, and semi-formal individual work)? The business case may set some constraints but little in respect to important pedagogical aspects associated with the design problem. Recent work in the wider design literature has highlighted the importance of fully understanding the ‘design problem’ before developing a design solution. A number of advantages are cited from promoting a more creative, innovative approach to the problem; to better understanding of the core purpose of the design process and more effective quality assurance and design validation.
  • How important is it to recognise that design stages are not discrete units that ‘end’ once a particular form or document has been submitted. It is not a linear process for each overlaps and may continue to be worked later in the process with more porous links to it’s neighbours (shown by the dotted lines above). Furthermore, it would now appear course development involves a greater range of people throughout the entire process (rather than each just involved in a discrete portion of the process).
  • Is too great an emphasis placed on the submission of a form or specification before work moves on? Instead why not focus on the process of creating these forms and specifications and use the cumulative amount of ‘work’ put in to preparing these documents as indicators of ‘sufficient’ design? Here the notion of doing a ‘sufficient’ amount of design work replaces that of producing a document/forum ‘sufficiently’. Yet how could we record and store documents and design activity ‘evidence’? The most obvious is some form of portfolio/s (e.g. folders on a local or shared area). The diagrams indicate some of these portfolios and each module would need to decide, upfront, what sufficiency may mean for them.
  • Is there enough formal review of the meso- and micro- levels of design? Module design currently takes place at several inter-connected levels, however often formal institutional process fade as design work in earnest begins.
  • Is there sufficient recognition of the micro-design taking place in the production of course materials and services even once courses have been written or operational documents prepared? Maybe not, and a consequence does this mean that micro-design decisions may not be properly supported or articulated?
  • Module/course evaluation (after delivery of the course) is often well formalised, however, can this be enhanced by better definition of the design problem space and planning of activity for the design process at the beginning? There is certainly potential for further quality assurance measures here and this may enable better tracking of decisions and therefore of accountability.

Benchmarking Assessment (conference paper): breaking down barriers and building institutional understanding

Denise Whitelock and myself have recnetly been working on a project to identfy key meaures (or criteria) of assessment processes and practice. The aim to to develop a benchmark instrument for use in better assessing and understanding assessment practice in Higher Education.

As part of this work, I am presenting on our behalf a paper at the Computer Assisted Assessment Conference in Southampton this week. In the abstract we make the point that benchmarking offers a comprehensive way of measuring current practice in an institution per se; whilst also gauging achievement against external competitors. It would appear that although e-learning has been benchmarked with a number of universities in the UK and abroad no one to date has tackled the area of assessment; which is now becoming of more concern with the advent of e-assessment. 

Our paper describes the construction of a set of benchmarking measures/indicators and the outcome of early pilots which combine a survey instrument and semi-structured interview methodologies. The findings suggest that building a comprehensive and robust core of benchmark measures would have great utility and value to institutions; not just in external benchmarking but in internal benchmarks and reviews, setting baselines, exploring the student experience, providing staff with data meaningful to their role and professional development and supporting continuous improvement.

The paper is accompanied by our current working draft of the benchamarking measures. [Since writing this post the measures have been revised further. A new version of the measures is available at: Assessment Benchmarking Criteria v17_A]. We will be very interested to hear your comments or feedback on the paper and the draft.

The Lattice model for designing learning: defining the design problem space and guiding the design solutions

Sheila MacNeill recently suggested posting current work or ideas ahead of the JISC Design Bash (taking place later this month in Oxford). Since January I’ve scaled back my time with the OU Learning Design Initiative, however, I am still involved with project managing our JISC funded project. In addition, I have continued to retain my personal interest in how visual conceptualisations and cartographies of learning could benefit the design process, and in how these can enrich, even fundamentally change, the student experience of learning.

In this post I’ll address the first part of this interest – the design process. There are two related concerns; how to set-out and imagine a model of the design problem space (the first step to developing a solution), and how could this be used for a more practical tool to design learning. In this post you will notice that I talk about ‘designing learning’ or ‘designed learning’ rather than ‘learning design’ or ‘Learning Design’ and this is intentional as should become evident. 

The diagram below shows where I am currently in imagining what key dimensions exist in a design problem/solution space and how they link together (click on image to enlarge). I call this the Lattice model because of the inter-relational nature of the design elements. The purpose of laying this out as a network, rather than as a list or linear form, is to make explicit and explore the interconnectedness in designing learning. I would note that this is just a snap shot of a changing model.

The construction of this model has been framed by a number of observations and literatures. I’ll set out a few below but haven’t the space for an exhaustive account:

• Representations of learning designs tend to be concerned more with observable, performed elements of activity but we need to move much further beyond this. The sequence (or swim-lane) visualisation is a good example – a vertical line showing learning tasks with resources, support and sometimes learning outcomes connected to it. This layout was used in an early paper by Oliver and Hetherington in 2001 who showed the ‘three critical elements of learning design environments’ – learning task, learning resources and learning supports – with a basic notational system of rectangles, triangles and circles. Eight years later, these components are still important to learning design – for example Helen Beetham (2009) defines a learning activity as ‘a specific interaction of learner(s) with other(s) using specific tools and resources, orientated towards specific outcomes’ (marked A on the diagram below). Conole’s pedagogy profiler and the OU’s broader project to combine pedagogic and business visualisations of a course are examples of this moving forward with specific representations of aspects of designed and delivered learning. However, it remains uncertain how these descriptions connect together, how they help conceptualise the overall problem/solution space and how far they offer critical understanding. There are still many constraints and drivers to a design are undisclosed. A greater range of dimensions (elements relating to the design) are needed to fully map the design landscape.

• Use of the ‘sociocultural approach’ is an important perspective for Educational psychology in its attempts to theorise the role of culture and society. Although this is certainly not the only theoretical position from which to derive understanding (see later), given its key role any model should aim to accommodate (and yet also push?) this. In doing so we should acknowledge that the designer/teacher is not detached from the design process but implicated at a personal level with it. As the designer is both culturally and historically situated this makes their positionality and ‘intent’ (a term with echoes back to American pragmatism) important. Goodyear talks of the importance of representing intent, Strobel et al (2008) of capturing the design context and my experience at the OU working with Paul Clark and Andrew Brasher in trying to de-construct and visualise existing units revealed how important it is to know the thinking – and evidence supporting that thinking – ’behind’ a design (marked B below). Moving further, there is a need to situate learning as a social act – as Rogoff, for example, holds: learners engage in shared endeavours that are both structured and constituted by cultural and social norms (Rogoff, 1995). However, it difficult to find a language with which to label this dimension/box because traditions in social and cultural theory range widely on how this act could be interpreted and there is now an increasing interlacing between them. For now, I’ve borrowed from Giddens’ structuration theory the notion that there are structural rules and resources and added discourse to this principally as a nod to post-structuralism and hermeneutics (C). This label is therefore vague enough but drives us: to a more nuanced understanding of our students – be this a deeper psychological (Solomon, 2000), social, and cultural (Scheel & Branch, 1993) and the associated opportunity for and means of learning these enable or constrain; and to the intention of the designer and purpose of the activities (and ‘where’ they happen).

• From other design disciplines we learn the importance of first reflecting on and describing the design ‘problem’ space – from which the solution(s) will emerge (i.e. not just racing straight into developing the solution) – see earlier posts. Early IMS Learning Design had little to say about how one actually arrived at the design and whilst patterns outline aspects of the problem, the representation is designed to support someone looking for a solution rather than understanding the problem in the first place.

• The role of assessment in the design needs to be reconsidered – seeing it not as a product but as an activity itself. One option is to understand assessment as a process that ‘acts on’ student output (i.e. an object, action, spoken word, etc.). It would see this output as a resource produced for a specific audience that could be used again later in the learning activity or that could be transformed in to a new artefact/resource (i.e. through the activity of the teacher, student etc.). Irrespective of if, or how, this output is re-used in the learning activity, it will (or should) also constitute the evidence: to demonstrate the learning outcomes/objectives have been achieved (marked D below), to reveal other unanticipated outcomes (after Eisner, Polanyi, etc.) and to support other forms of evaluation (I’ve just jotted down Zakrzewski’s three on the diagram at present).

• There remain many other, often more pragmatic, perspectives to integrate in to the design problem space – thereby reflecting the heterogeneity of educational thought. For example: instructional designs interest in detailing what is to be learnt, learning tasks, student prior learning etc. (marked E); and the belief that a design should be built around key learning or conceptual ‘challenges’ (G). Clearly to appeal to a range of teachers the model should not be restricted to one individual theory of learning. This is in partly why I favour talking about ‘designing learning’ or ‘designed learning’ rather than ‘learning design.’

• Design of a unit of learning is influenced by practical constraints and conditions (H) defined at higher levels e.g. the block or the course (the issue of layers of design and fitting them together has been much discussed and something we’ve looked at in mapping courses), by other ‘evaluation’ demands from the institution or researchers (F), by previous units (for example, prior learning (I)) and by guidelines and training required of staff (J) . The temporal and multi-scale nature of the design problem needs representation (Grey-shaded boxes).

• Visual representation is a powerful means to communicate complex, non-linear, inter-connected relationships. It offers distinct advantages over linear descriptions and can support problem solving performance (for example, Baylor et al., 2005). This is supported by our small-scale studies at the OU (n=45-50) where we have found that a majority of staff said there were aspects of their work that would/do benefits from using visual representation and techniques (81%); they would like to improve their knowledge of visual representation and tools (81%); and that more use of visual representations (that show what is to be learnt and how) could help students better understand and plan their study’ (73%) (Cross et al. 2009)

 

As a practical design tool? 
 
Whilst the model itself can provide a framework for imaging the problem/solution space, of interest to many will be how this model can be translated directly in to a more practical application. The screenshot below shows an early attempt in Excel. Here, each dimension becomes a zone (a box) in which information about the design (be this text, lists or labelled mind-mapped objects) can be inserted. In a typical scenario, the design will evolve and mature as Read more of this post

On the boundary between Learning and Instruction Designing

I came across a posting on the OU’s Cloudworks website the other week asking what the differences were between Learning and Instructional Design. I wonder if the conceptual or practical distances between learning and instructional design are to some degree elastic depending on individual interpretations, definitions, positionality, approach to teaching and breadth of knowledge of the two literatures. Sometimes there will be clear water and sometimes more murky eddies. Whilst it is important to pursue the unravelling of difference, a second direction could be to focus on the boundary between the disciplines; what can each learn from the other and is there overlap in how both grapple with the questions around what design means, how should it be done, how represented, how shared, how evaluated? More importantly, could the introduction of an alternative, and indeed different, perspective spark new questions and save us time in developing solutions.

For example, both have had to contend with methods of visualising conceptual models, the design problem, the design solution and learning or instruction at different levels of granularity. Certainly the way you visually present or re-present a design is important. For both any description is not value free as the describer and, just like the cartographer, this cannot be divorced from, say, their choice of representation form or what, and how it is, represented. Another example is the use of metaphor and analogy associated with aesthetic and performance. Learning design has certainly employed the notion of the play to conceptualise learning and music notion to help imagine how notion could work (James Dalziel speaking at the European LAMS conference hosted at the OU earlier this year). Contrast this with say Parish’s (2007) paper which I’ve just been reading that explores how principles of the aesthetic (art, plays, etc) could support or be integrated in to an instruction design. Indeed, how we conceptualise performance and action touch upon many key understandings such as ‘task’, ‘activity’ and interaction between ‘players’ or other elements in the learning event.  

But in addition to this cross-disciplinary work, both may fit within a broad practice of, for want of a better term, educational design. The question now is what can either learn from the wider design literature? One example is an intriguing study by Boiling I’ve recently come across in to different design cultures on various university design courses. The case study of design in Fine Art describes a very different more communal process of design dissemination and design critique by immediate peers. Group identity and reciprocity appear important, as does negotiations of power and the earning, within the group, of trust and status. What if this model were adopted by a number of learning design teams/ individuals who together form of cohort over the lifetime of their design process?

A second example is from a recent paper (and presentation I attended) by Vasilije Kokotovich. In this he contrasted the practice of novice and expert product designers. The paper found a correlation between the quality of the design problem definition and the degree of creativity and reasoned explanation the final designs were judged to demonstrate. Another key finding of the study was that novice designers tend to head straight to the ‘embodiment’ phase of design – that  is to say, producing the design solution (in the case of learning design, this could be interpreted as the writing, orchestrating and structuring of a course). Expert designers on the other hand took additional steps, such as dwelling on and framing the design problem. The example he gave was that when asked to design a door knob a novice would start thinking ‘what should the door knob look like’ whilst an expert may start by asking ‘what is the problem the door knob is there to solve and what other possible designs exist that could satisfy this purpose better?’ This notion that a design problem will have more than one solution certainly provides justification for learning designers to think round a problem and more creatively and compare choices on offer. More broadly, these findings could help us in supporting learning design more effectively.

Boiling, E. (2003) Design Cultures. Retrieved December 2009 from http://www.indiana.edu/~idt/shortpapers/documents/design_cultures.html

Kokotovich, V. (2008) ‘Problem analysis and thinking tools: an empirical study of non-hierarchical mind mapping’ Design Studies 29, 1. p49-69

Parish, P. (2007) ‘Aesthetic principles for instructional design’ Education Technology Research & Development

Learning Designs as objects

Revisiting a paper I read a few years ago by Miles Ogborn made me wonder if some of the perspectives he adopts in exploring how ‘writing travels’ may help inform our conceptualisation of the making and movement of learning designs. As Miles notes ‘this would mean attending to their different modes of ‘travel’ in terms of the social relations that are constituted around and through these inscriptions as they are made, made mobile, transferred, and make the world.”

Looking at the constitutive role of learning designs as objects could therefore offer new insights. This sees designs as not just representations, but as artefacts in their own right. Just like playing ‘follow-the-lady’ with a street trickster, this is about watching the design object(s); how it moves, when it appears, disappears and re-appears, and the social and cultural context it travels within. This potentially makes more complex the way in which we study how other designers understand and take meaning from designs and about how, of if, one representation really does follow another during the design process.

Ogborn, M. (2002) ‘Writing travels: power, knowledge and ritual on the English East India Company’s early voyages’, Transactions of the Institute of British Geographers, p155-171

Unpacking our ‘distances’ from Learners

Unpacking our ‘distances’ from Learners

Last week, in our Cluster C meeting (convened as part of our involvement in the JISC Curriculum Design Programme), Helen Beetham posed a series of project challenges – one asked how our ‘distance’ from learners may impact our design approach. The most common interpretation in this context would be to mean phyiscal space – the difference, say, between ‘distance’ and ‘face-to-face’ teaching. However, as we begun to discuss in the session, distance can be understood in many other ways too. Not only could the notion of space be further deconstructed, there may be temporal distance (e.g. design may occur days, weeks, even years before the learning event occurs), intentional or necessary distance (e.g. leaving ‘room’ for what Helen termed ‘teacher and learner innovation’, or maintaining a professional or objective distance), distances that constrain (e.g. a perceived intellectual distance between ‘expert’ teacher and learner), conceptual or practical distance (e.g. in understandings or experiences), or distance borne of disagreement or confusion (e.g. in what pedagogy to use).

We may conclude that there may exist multiple distances between designer and learner. This not only raises the question of how the design process acts as a method of negotiating and making sense of these distances (and how to represent them) but it also introduces the question of where exactly the centre to a ‘learner-centred’ approach is, or indeed, if there is just the one. Helen suggested that ‘Good’ design should aim to acknowledge distance and ‘leave unspecified those issues that are better determined at closer range’. This raises a further question; how distant should a designer be and when (and if and how) should they get closer?

Certainly what this does is focus attention not just on the need to understand the spatial configurations of design generated by designing and decision making at different distances from the learner, but in the processes by which designs ‘travel’ through the ‘in-between’ spaces. An example of this may be the effective communication/passing-on of designs to those who are ‘closer’ to the learning (including the learners themselves).