‘Preliminary findings from a series of staff surveys on perceptions, attitudes and practices of learning design’ paper now available online

Someone recently pointed out that the full text and data from my 2009 ALT-C paper titled ‘Preliminary findings from a series of staff surveys on perceptions, attitudes and practices of learning design’ (co-written with Paul Clark and Andrew Brasher) is difficult to find so I’ve just deposited a PDF version in the Open Research Online repository.

This short paper reported the findings of a series of connected staff surveys which looked at some attitudes and perceptions about issues associated with learning design. This included asking staff if they agree/disagree with the following eight statements:

• ‘The advent of elearning is making the process of creating courses more complex’
• ‘I sometimes feel overwhelmed by the challenge of how to effectively integrate ICT in a course’
• ‘There is a pedagogic need for better integration of learning technologies (e.g. VLE) into courses’
• ‘It is becoming harder to understand how all the parts/components of planned learning and teaching fit together’
• ‘I do not find it difficult imagining how online learning content fits with other course content’
• ‘Technologies available today offer great potential for enhancing the student learning experience’
• ‘Understanding the relationship between pedagogy and learners’ activity is a priority for me’
• ‘Do you believe that more use of visual representations (that show what is to be learnt and how) could help students better understand and plan their study’

These statements were derived from several claims made in the learning design literature about the potential role (and need) for more designed approaches to learning and teaching and ongoing OULDI work at the time (semi-structured interviews, workshops, design observation, and focus groups site). The survey was administered to selected staff in 2009 and 50 members of staff responded.

Whilst this data is now a couple of years old, it is an interesting record of how staff responded to some key questions that are still being asked today: Has the advent of new technologies has created a much more complex situation in the design and planning of teaching and learning? Is this perception widespread across staff? And what is the perceived role for visualisation in the design process?

For example, take the first of the eight statements

Open Innovation and the ‘Next Generation Stage-Gate’ process: Reflections on Learning Design processes 2

Next-Generation Stage-Gating

The Stage-Gate concept was developed by Robert Cooper as an innovation management method for businesses. It is now a widely used mechanism to manage the product development process – in the case of a University this may be the development and production of a course. The object of the model is to support the making of a key decision at each stage – the question: should development continue?– or as Cooper puts it; to ‘Go’ with or ‘Kill’ it. For Cooper, the period between Gates consists of ‘information gathering activities’ followed by an analysis of the result which is then entered in to the Gate for decision.

Over the last five years some work has begun to assess and revise the Stage-Gate model. Indeed, Cooper (2008) himself acknowledges the need for the‘Next Generation Stage-Gate’. Reading through this work, I was encouraged to find comments and observations that aligned with themes discussed in my last blog post. Cooper (2008) offers a range of ways to revise or add to his model. I’ve picked out seven below:

1. A Flexible Process: we observed (see last post) that learning design activities should be planned in advance of the stage commencing and here

Exploring Spheres of Sharing: Analysis of Cloudworks Part 3

In March last year I looked at the extent and longevity of contributions made to the Cloudworks website by a sample of 250 users who registered in 2008 and 2009 (approximately 10% of registered users on the site). In this post I’ll ask what pattern of use, especially continued use, has this sample demonstrated over 2010? And is this similar to those who registered in 2010 year?

Cloudworks aims to support the sharing of teaching and learning and my original aim was to get an indication of how many were using the site to really ‘share’ and discuss ideas; that is to say, the proportion who posted on the site (rather than just using it as a resource to read). So how did the 2008-09 cohort do in 2010?

• In 2008-9, my original review found 8% (20 people) had posted two or more contributions over a period greater than 28 days from their first post (what I termed a ‘prolonged’ contribution). In 2010,  just over half of this group (13 our of 20 people) continued to make a contribution to the site. The remaining 7 did not post anything in 2010. However, this decrease is almost balanced by the fact that in 2010, 5 new people (from the original sample) made posts in a period of more than 28 days. This may be an indication of an emergence of a core group (5-7%) of sustained users. Scaling this 10% sample up to the entire Cloudworks usership this suggests there may be some 150 to 250 users making prolonged contributions.
• In 2008-9, some 39% of registered users demonstrated some ‘activity’ on the site (such as posting a comment or link). In 2010, however,  just 14% of the sample had made a contribution to the site.

Taken together, these data seem to point to a core group of contributors posting to the site.

The Lattice model for designing learning: defining the design problem space and guiding the design solutions

Sheila MacNeill recently suggested posting current work or ideas ahead of the JISC Design Bash (taking place later this month in Oxford). Since January I’ve scaled back my time with the OU Learning Design Initiative, however, I am still involved with project managing our JISC funded project. In addition, I have continued to retain my personal interest in how visual conceptualisations and cartographies of learning could benefit the design process, and in how these can enrich, even fundamentally change, the student experience of learning.

In this post I’ll address the first part of this interest – the design process. There are two related concerns; how to set-out and imagine a model of the design problem space (the first step to developing a solution), and how could this be used for a more practical tool to design learning. In this post you will notice that I talk about ‘designing learning’ or ‘designed learning’ rather than ‘learning design’ or ‘Learning Design’ and this is intentional as should become evident. 

The diagram below shows where I am currently in imagining what key dimensions exist in a design problem/solution space and how they link together (click on image to enlarge). I call this the Lattice model because of the inter-relational nature of the design elements. The purpose of laying this out as a network, rather than as a list or linear form, is to make explicit and explore the interconnectedness in designing learning. I would note that this is just a snap shot of a changing model.

The construction of this model has been framed by a number of observations and literatures. I’ll set out a few below but haven’t the space for an exhaustive account:

• Representations of learning designs tend to be concerned more with observable, performed elements of activity but we need to move much further beyond this. The sequence (or swim-lane) visualisation is a good example – a vertical line showing learning tasks with resources, support and sometimes learning outcomes connected to it. This layout was used in an early paper by Oliver and Hetherington in 2001 who showed the ‘three critical elements of learning design environments’ – learning task, learning resources and learning supports – with a basic notational system of rectangles, triangles and circles. Eight years later, these components are still important to learning design – for example Helen Beetham (2009) defines a learning activity as ‘a specific interaction of learner(s) with other(s) using specific tools and resources, orientated towards specific outcomes’ (marked A on the diagram below). Conole’s pedagogy profiler and the OU’s broader project to combine pedagogic and business visualisations of a course are examples of this moving forward with specific representations of aspects of designed and delivered learning. However, it remains uncertain how these descriptions connect together, how they help conceptualise the overall problem/solution space and how far they offer critical understanding. There are still many constraints and drivers to a design are undisclosed. A greater range of dimensions (elements relating to the design) are needed to fully map the design landscape.

• Use of the ‘sociocultural approach’ is an important perspective for Educational psychology in its attempts to theorise the role of culture and society. Although this is certainly not the only theoretical position from which to derive understanding (see later), given its key role any model should aim to accommodate (and yet also push?) this. In doing so we should acknowledge that the designer/teacher is not detached from the design process but implicated at a personal level with it. As the designer is both culturally and historically situated this makes their positionality and ‘intent’ (a term with echoes back to American pragmatism) important. Goodyear talks of the importance of representing intent, Strobel et al (2008) of capturing the design context and my experience at the OU working with Paul Clark and Andrew Brasher in trying to de-construct and visualise existing units revealed how important it is to know the thinking – and evidence supporting that thinking – ’behind’ a design (marked B below). Moving further, there is a need to situate learning as a social act – as Rogoff, for example, holds: learners engage in shared endeavours that are both structured and constituted by cultural and social norms (Rogoff, 1995). However, it difficult to find a language with which to label this dimension/box because traditions in social and cultural theory range widely on how this act could be interpreted and there is now an increasing interlacing between them. For now, I’ve borrowed from Giddens’ structuration theory the notion that there are structural rules and resources and added discourse to this principally as a nod to post-structuralism and hermeneutics (C). This label is therefore vague enough but drives us: to a more nuanced understanding of our students – be this a deeper psychological (Solomon, 2000), social, and cultural (Scheel & Branch, 1993) and the associated opportunity for and means of learning these enable or constrain; and to the intention of the designer and purpose of the activities (and ‘where’ they happen).

• From other design disciplines we learn the importance of first reflecting on and describing the design ‘problem’ space – from which the solution(s) will emerge (i.e. not just racing straight into developing the solution) – see earlier posts. Early IMS Learning Design had little to say about how one actually arrived at the design and whilst patterns outline aspects of the problem, the representation is designed to support someone looking for a solution rather than understanding the problem in the first place.

• The role of assessment in the design needs to be reconsidered – seeing it not as a product but as an activity itself. One option is to understand assessment as a process that ‘acts on’ student output (i.e. an object, action, spoken word, etc.). It would see this output as a resource produced for a specific audience that could be used again later in the learning activity or that could be transformed in to a new artefact/resource (i.e. through the activity of the teacher, student etc.). Irrespective of if, or how, this output is re-used in the learning activity, it will (or should) also constitute the evidence: to demonstrate the learning outcomes/objectives have been achieved (marked D below), to reveal other unanticipated outcomes (after Eisner, Polanyi, etc.) and to support other forms of evaluation (I’ve just jotted down Zakrzewski’s three on the diagram at present).

• There remain many other, often more pragmatic, perspectives to integrate in to the design problem space – thereby reflecting the heterogeneity of educational thought. For example: instructional designs interest in detailing what is to be learnt, learning tasks, student prior learning etc. (marked E); and the belief that a design should be built around key learning or conceptual ‘challenges’ (G). Clearly to appeal to a range of teachers the model should not be restricted to one individual theory of learning. This is in partly why I favour talking about ‘designing learning’ or ‘designed learning’ rather than ‘learning design.’

• Design of a unit of learning is influenced by practical constraints and conditions (H) defined at higher levels e.g. the block or the course (the issue of layers of design and fitting them together has been much discussed and something we’ve looked at in mapping courses), by other ‘evaluation’ demands from the institution or researchers (F), by previous units (for example, prior learning (I)) and by guidelines and training required of staff (J) . The temporal and multi-scale nature of the design problem needs representation (Grey-shaded boxes).

• Visual representation is a powerful means to communicate complex, non-linear, inter-connected relationships. It offers distinct advantages over linear descriptions and can support problem solving performance (for example, Baylor et al., 2005). This is supported by our small-scale studies at the OU (n=45-50) where we have found that a majority of staff said there were aspects of their work that would/do benefits from using visual representation and techniques (81%); they would like to improve their knowledge of visual representation and tools (81%); and that more use of visual representations (that show what is to be learnt and how) could help students better understand and plan their study’ (73%) (Cross et al. 2009)

 

As a practical design tool? 

 

Whilst the model itself can provide a framework for imaging the problem/solution space, of interest to many will be how this model can be translated directly in to a more practical application. The screenshot below shows an early attempt in Excel. Here, each dimension becomes a zone (a box) in which information about the design (be this text, lists or labelled mind-mapped objects) can be inserted. In a typical scenario, the design will evolve and mature as

On the boundary between Learning and Instruction Designing

I came across a posting on the OU’s Cloudworks website the other week asking what the differences were between Learning and Instructional Design. I wonder if the conceptual or practical distances between learning and instructional design are to some degree elastic depending on individual interpretations, definitions, positionality, approach to teaching and breadth of knowledge of the two literatures. Sometimes there will be clear water and sometimes more murky eddies. Whilst it is important to pursue the unravelling of difference, a second direction could be to focus on the boundary between the disciplines; what can each learn from the other and is there overlap in how both grapple with the questions around what design means, how should it be done, how represented, how shared, how evaluated? More importantly, could the introduction of an alternative, and indeed different, perspective spark new questions and save us time in developing solutions.

For example, both have had to contend with methods of visualising conceptual models, the design problem, the design solution and learning or instruction at different levels of granularity. Certainly the way you visually present or re-present a design is important. For both any description is not value free as the describer and, just like the cartographer, this cannot be divorced from, say, their choice of representation form or what, and how it is, represented. Another example is the use of metaphor and analogy associated with aesthetic and performance. Learning design has certainly employed the notion of the play to conceptualise learning and music notion to help imagine how notion could work (James Dalziel speaking at the European LAMS conference hosted at the OU earlier this year). Contrast this with say Parish’s (2007) paper which I’ve just been reading that explores how principles of the aesthetic (art, plays, etc) could support or be integrated in to an instruction design. Indeed, how we conceptualise performance and action touch upon many key understandings such as ‘task’, ‘activity’ and interaction between ‘players’ or other elements in the learning event.  

But in addition to this cross-disciplinary work, both may fit within a broad practice of, for want of a better term, educational design. The question now is what can either learn from the wider design literature? One example is an intriguing study by Boiling I’ve recently come across in to different design cultures on various university design courses. The case study of design in Fine Art describes a very different more communal process of design dissemination and design critique by immediate peers. Group identity and reciprocity appear important, as does negotiations of power and the earning, within the group, of trust and status. What if this model were adopted by a number of learning design teams/ individuals who together form of cohort over the lifetime of their design process?

A second example is from a recent paper (and presentation I attended) by Vasilije Kokotovich. In this he contrasted the practice of novice and expert product designers. The paper found a correlation between the quality of the design problem definition and the degree of creativity and reasoned explanation the final designs were judged to demonstrate. Another key finding of the study was that novice designers tend to head straight to the ‘embodiment’ phase of design – that  is to say, producing the design solution (in the case of learning design, this could be interpreted as the writing, orchestrating and structuring of a course). Expert designers on the other hand took additional steps, such as dwelling on and framing the design problem. The example he gave was that when asked to design a door knob a novice would start thinking ‘what should the door knob look like’ whilst an expert may start by asking ‘what is the problem the door knob is there to solve and what other possible designs exist that could satisfy this purpose better?’ This notion that a design problem will have more than one solution certainly provides justification for learning designers to think round a problem and more creatively and compare choices on offer. More broadly, these findings could help us in supporting learning design more effectively.

Boiling, E. (2003) Design Cultures. Retrieved December 2009 from http://www.indiana.edu/~idt/shortpapers/documents/design_cultures.html

Kokotovich, V. (2008) ‘Problem analysis and thinking tools: an empirical study of non-hierarchical mind mapping’ Design Studies 29, 1. p49-69

Parish, P. (2007) ‘Aesthetic principles for instructional design’ Education Technology Research & Development