Learners as learning designers at the workplace

Prof. Betty Collis, a noted consultant in technology for strategy, learning and change in corporate learning and higher education at the University of Twente in the Netherlands, gave a great keynote address at last week’s JISC programme meeting. She ended with the following ‘provocative  thought’:

Perhaps it’s time to reconsider the orientation that ‘Design for learning refers to the complex processes by which practitioners devise, structure and realise learning for others’. That does not sound to me like the way that learning is going on in organisations.

This ‘provocative thought’ was borne out by many examples  in Betty’s presentation of the way in which employees at Shell used technology – especially wikis – for knowledge sharing and informal learning. She commented on a trend from formal, structured training towards more informal, networked learning within the corporation: over time, employees preferred learning from information shared by their colleagues in a giant, company-wide wiki, than from formal, instructor-led training courses.

During the period that Betty was Leader of Shell-University of Twente Collaborative Project (2001-2005), a strong culture of knowledge sharing was generated in the organisation, with every employee understanding that they had something to teach others. Learning (and teaching) at the workplace became inseparable from getting things done (i.e. working).

This resonates well with the comment made by Jay Cross, Jane Hart et al in a recent article in eLearn magazine:

The accelerating rate of change in business forces everyone in every organization to make a choice: learn while you work or become obsolete. Nonetheless, we never use the word “learning” with a senior executive…

Companies don’t want learning—they want things done….

That’s why we talk about “working smarter.” More than knowing how to get things done, working smarter involves actually doing them.

It strikes me that the way assessment is carried out can have a powerful impact on the nature of learning, especially for work-based learners. One of Betty’s very practical recommendations for enhancing higher education courses was to gear  assessment tasks towards getting students to produce something that could be used as a learning resource by other students. One could take this idea one step further, by focusing assessment tasks on getting learners to generate something that is useful for their colleagues – this could involve sharing information, proposing a solution to a problem at the workplace, or carrying out an experiment to try to enhance workplace processes or outputs.

Those programmes that provide learners with the skills to apply their learning in innovative ways that add value to their own workplace contexts are likely to be the ones that survive the lean times ahead.

Gabi Witthaus, 20 Oct 2010

Audio feedforward for distance-learning assessment support

Recently my BDRA colleague Ming Nie posted an item about the use of audio files or podcasts for feedforward. This was based on work with distance-learning psychology students in the DUCKLING research project, providing dissertation and module assignment support. She highlighted two key benefits in terms of encouraging students to ‘think ahead’ and also providing them with reassurance about being ‘on the right track’.

For the past six months, I have been an online module tutor on a distance-learning course supporting master’s-level management students. As a relative novice in the podcasting arena, this provided a good opportunity to see how audio files could be used to support students’ work on their assignments, coupled with the VLE Discussion Board for assignment and other questions related to the course materials. Some students have access to local tutor support, but others do not. However, for distance learners the assignment is always a potential source of anxiety. So providing resources to complement both the assignment brief and the facility to post discussion board questions seemed likely to be received positively.

The approach comprised three separate audio files, one on assignment process issues and one on each of two assignment questions. At 10 to 14 minutes in length, these would be classified as ‘long’ using the 10-factor model for podcast development (1) derived from the IMPALA research project. However, given the ‘distance’ aspect involved and likely levels of discussion board traffic, providing fewer if longer audio files was a ‘justified compromise’.

It is still early days, but the following observations can be made:

  • The relevance of audio files for non-native English-speaking/English as a study language students, who are able to ‘rewind’ and listen repeatedly to help develop their understanding of the language and of the assessment requirements.
  • The ability to ‘start-stop’ and make notes while listening and then to refer back and use the notes as a reference source or checklist when developing the assignment.
  • The more personal nature of listening to a spoken commentary, compared with reading course materials or asynchronous discussion board Q&A episodes, thus increasing the diversity of teaching media available to students.
  • Students identifying aspects of academic research and writing that their professional background and previous work experience have not highlighted, thus cultivating a different outlook and learning from the study experience rather than from the course materials as such.
  • The use of audio files as vehicles for student discussion in locally-based face-to-face study groups or via ‘closed’ social networking sites set-up by students at the start of the course.

To date, the investment made in interpreting the assignment brief and reflecting on what might be helpful for students seems warranted. Conceptually, this feels no different from preparing personal notes in advance for a class or workshop teaching session where assignment questions might arise, but instead recording the thoughts for wider distribution and remote access.

Roger Dence / 20th November 2009

(1) Edirisingha P, Salmon G and Nie M (2008) “Developing pedagogical podcasts” in Salmon G and Edirisingha P (eds) (2008) Podcasting for learning in universities, SRHE and Open University Press/McGraw-Hill, Maidenhead, 222pp.

On assessment and alignment

Last week I sat the final exam of the Open University’s Certificate in Management. The version I did (B615) was a 1-year, 60-credit course, divided into 4 modules. The course has been a major component of my CPD and a most enjoyable experience. As is the case with most exams, no matter how much you’ve enjoyed the course, you’re glad when it’s over. The results will be out in December.

An interesting aspect of the course was precisely its assessment. We had online discussions, peer feedback and electronic submissions of assignments throughout (5 assignments in total)… yet the final summative assessment was a 3-hour, individual, sit-down, closed-book, handwritten exam. I hadn’t done an exam like that for over 20 years.

Is a final exam the most appropriate method of assessing students on this course? If so, is this type of exam the most suitable option? It seems that this is a prime example of misalignment between the final assessment and everything else that all students are required to do throughout the programme. Allowing the use of word-processors would have helped – not least those of us whose handwriting has deteriorated over the years by doing exactly what the previous stages of this course asks of students, i.e. using computers for all their coursework.

I look forward to receiving my result. In the meantime, I’ll continue to reflect on fit-for-purpose curriculum design and assessment choices.

Dr A Armellini
27 October 2009

Open-book exam? Don’t forget your e-Reader

e-Readers such as the Amazon Kindle and the Sony PRS-505 are enjoying a surge of interest this summer. It is not too difficult to see the e-Reader as a handy item to pack in your holiday luggage. Imagine being able to bring along the entire works of Dickens, Austen, and Shakespeare with room for dozens if not hundreds more, all in a thin, light, attractive gadget which furthermore increases the font to just the right size for your eyes with a single click. e-Readers flow the pages one after another, presenting them in the same way most people read novels.

But can e-Readers serve up textbooks in a way which is helpful and conducive to study? And what about the traditional handouts and notes – could these be usefully offered on e-Readers? Our DUCKLING project is addressing questions such as these.

Looking at handouts and notes alone, there is money-saving potential. Photocopying, collating, and shipping a box of notes to a distance learner for a single module often costs several hundreds of pounds. Shipping a £199 e-Reader (the current price of the PRS-505) loaded with the pertinent files should realise definite savings.

For on-campus students as well, the weight of paper versus the weight of an e-Reader is a factor. Imagine having all textbooks and notes in a single, light gadget, rather than lugging books and papers in a heavy backpack around campus all day. Being able to add one’s own annotations to notes and textbooks is a fairly-necessary feature which the Sony PRS-505 for example does not have, although Sony’s next model version (not yet available in the UK) does.

I recently sat an open-book final exam for an advanced statistics class. The instructor set the exam rules thus: “You may bring any inanimate object into the exam with you, except a laptop.” My instructor, albeit very forward-thinking, was still not ready to allow full access to the internet during an open-book exam. That particular issue may be a topic for another blog post. But my question was: what books and what notes shall I bring to an exam which I knew would be in-depth and complicated, and on a topic for which I did not feel great confidence? Printing out almost all of the notes and handouts from the VLE produced a stack of paper over a foot high. I could not imagine lugging that, plus 3 textbooks, into an exam where I would be sitting at one of those old wooden exam tables less than 2 feet wide.

My immediate thought would be to bring an appropriately-loaded e-Reader to my “open-e-book” exam. Would my instructor have been any more willing to let me do this, however, than to let me bring in my laptop? I suspect that a good search function alone might make the instructor uneasy enough to ban e-Readers as well, on the basis that the search function would be doing the work I should do for the exam. Instructor unease will only increase as e-Readers’ direct connection to the internet improves.

The real question, however, is whether textbook publishers will develop a healthy model of “e-textbooks.” So far, the tendency for some is to charge much more for an “e-textbook” than for a traditional text, on the basis that “students might just give away the e-book to their friends.” If they maintain that stance and e-Reader purchases grow at a rate anything like that of mp3 players, textbook publishers will find themselves in the same predicament as the music industry. It is in everyone’s interest that a healthy model of e-textbook pricing and availability is developed.

 Terese Bird

Do students use feedback?

Today at a lunch party to welcome two new colleagues (Welcome, Terese and Tania!) and to congratulate a colleague on her marriage (Congrats, Sandra!), a few of us found ourselves making small talk about giving feedback to students. (It was pretty deep stuff for small talk, but that’s the way things go around here!) Anyone who has ever given feedback to students on assignments will know the amount of effort required to do so, and the concomitant sense of vague curiosity you feel, as you wonder to what extent the student will engage with it.

Our Head of Engineering and ex-Pro-Vice-Chancellor, John Fothergill, told us of a strategy he had implemented in a recent course, in which 10% of the marks for every assignment were given for evidence that the student had responded to feedback given for the previous assignment.

Another strategy he had tried was to require each student to review a peer’s assignment according to a given set of assessment criteria. He then marked the students’ reviews of their peers’ work, rather than their actual assignments. This had the effect of ensuring that the students took more care than usual to familiarise themselves with the assessment criteria while writing their assignments.

These are relatively simple strategies to implement, with a potentially powerful impact on the teaching and learning process. Does anyone else have anecdotes or strategies to share about getting students to think carefully and critically about the assessment criteria and the feedback they receive?

Gabi Witthaus, 6 July 2009

Crime Scene Investigation in the Garden of EDEN

CSI: Crime Scene Investigation is an American TV series which follows the investigations of a team of Las Vegas forensic scientists, trying to solve mysterious and unusual murders. EDEN is the European Distance and E-Learning Network whose annual conference took place last week in Gdansk, Poland. The link between the two? I found it in a presentation during the EDEN conference, which felt as an inverse “Eureka” moment. And by an inverse “Eureka” I mean the experience of one of the protagonists in the latest Terminator movie when he discovered he was not human as he believed but a machine, programmed to believe it was human (apologies for the spoiler). A lot of pieces fall into place and make sense but you are none the happier.

The presentation that brought all of these analogies was called “Using Multiple Online Security Measures to Deliver Secure Course Exams to Distance Education Students” by a team from Pennsylvania State University’s World Campus. After two days of discussions and demonstrations at EDEN of the power of new technologies to unlock learners and teachers’ creativity, the promise of new and exciting ways for collaboration and discovery for the future of learning, of openness and freedom, the presentation from Penn State suddenly brought home the fact that there exists a very real and very different possible future for learning – that of the CSI approach I mentioned in the beginning.

The security measures in question were brought in at Penn State University in response to the Higher Education Reauthorization Act, effective August 14, 2008 in the USA, that requires institutions to authenticate the identity of distance education students. An excerpt from the Act goes like this:

“The agency or association [i.e., the accreditor] requires an institution [i.e., a college] that offers distance education or correspondence education to have processes through which the institution establishes that the student who registers in a distance education or correspondence education course or program is the same student who participates in and completes the program and receives the academic credit.”

While this sounds very sensible and necessary, I couldn’t help but feel uncomfortable with the set of measures that Pennsylvania University was advocating for the implementation of this requirement, or as they called it The Proctored Exam. It involves biometric authentification (the students are supplied with a device that collects biometric data, a bit like the equipment Border Police use); Real-Time Data Forensics (apparently the typing pattern of each of us is unique, like fingerprints and the Proctoring Exam software requires the examinee to type randomly generated text until enough data is collected to validate their identity); constant live webcam feed, capturing the room in which the student is taking the exam; a piece of software which record every key stroke during the exam and takes control over the students computer, blocking access to the machine’s web browser and all other programmes apart from the exam software. All of those. The reason I keep referring to movies in this blog post is that it is only in movies that I have seen so much technological control over an activity – be it Crime Scene Investigation or Terminator. It made me think that the abundance of fast accessible open knowledge and innumerable possibilities to do things with this knowledge will not necessarily lead to a future of freedom, trust and creativity. Learners in the future may well be faced with a choice. To attend and graduate from an institution where learning is assessed and consequently certified on the basis of what they can do, what they can create and how they can collaborate in an environment of freely accessible knowledge, where identity is established by unique skills and outcomes of learning (for example assessment tasks may be based on individually tailored continuous monitoring of the development of the unique combination of skills and knowledge of individual students). Or, perhaps employers and state authorities will press for uniformity and certification of learning outcomes based on comparisons among students and rankings, where indeed validation of identity will be done through the methods of CSI.

The future will tell… However, taking into account the defeat of the powerful music industry in trying to impose control over what people do with their music through technological means (which is what Penn State University is hoping to do but for learning rather than music), I would place my bets for a future where technology is enabling, individual and supportive, rather than forensic and policing. After all, the founder of Pirate Bay just got elected for an European Parliament MP

Sandra Romenska

BDRA