Online exams webinar recording now available

The recording of our webinar ‘Online exams: migration or transformation’ run jointly between Jisc and EUNIS is now available.

Listen to the recording on our YouTube site to hear the views of:

  • Stuart Allan, who recently completed an MSc on the subject at the University of Edinburgh
  • David Parmentier, Sogn og Fjordane University College, Norway
  • Annette Peet, SURF , Netherlands
  • Martyn Roads, consultant specialising in assessment in the UK FE and skills sector

You can find out more about the background to the webinar in this blog post

The session was held using Blackboard Collaborate and the recording is made available on YouTube to maximise its accessibility. This means you don’t have access to the chat that was taking place so here I have jotted down a few comments and observations from the chat as well as links to some of the key resources suggested by the presenters.

We were using the term ‘online’ in a broad sense: really this was all about summative testing in digital format not necessarily implying that there needs to be full Internet access at all times.

‘Pedagogic’ reasons for moving to digital exams include enhancing the curriculum and better preparing students for employment.

People are often worried about the cost of migrating to digital without having any real idea of costs/benchmarks for existing paper processes.

Interesting point about how students think when writing vs typing. Mogey and Fluck (2015) found that in computer-based exams students are sometimes more concerned with maximising the number of words they can include in their response than with the construction of academic arguments. Mogey N. and Fluck A. (2015) ‘Factors influencing student preference when comparing handwriting and typing for essay style examinations’, British Journal of Educational Technology, 46 (4), pp. 793–802

In relation to collaboration on item banks – technical incompatibility between systems seems to be an issue. Manchester University’s UKCDR study looked at this – medical education boards share successfully across institutions. If the on-screen exam/ item building system is QTI compliant, that helps with migration of items and exam templates. There was also a comment that the education sector needs to look beyond QTI and more at API based content integration.

UK-based awarding bodies and commercial education companies in professional education outside of Ofqual’s regulation have deployed onboard remote invigilation. Service support and cultural acceptability (especially if the remote invigilation provider is non-UK based) are proving the main barriers to adoption.

Surely if students are taught in the same method of delivery as online assessment then they would be more in tune.’

The digital literacy of staff is something we are also experiencing as a challenge. The students are way ahead of anyone, so not a problem. Assessors seem to be the least prepared for an entirely digital exam.’

English government policy has retrenched to terminal summative assessment. Scottish government keen to use digital evidence in qualifications.

Resources from the presenters

You can find Stuarts blog at:

Here’s link to Myyry’s and Joutsenvirta’s publication as recommended by Stuart:

White Paper on online proctoring (in English)

Innovation in digital assessment: assessment themed issue (April 2016 in English)

Assessment Security Selection Model (in English)

Transforming assessment and feedback with technology guide:

EMA processes and system guide:

Supplier responses to system requirements:

EMA readiness tool:

2012 UK Landscape report:

Resources suggested by participants

Surpass Paper+ assessment system as a means from migrating from paper to digital exams

Case study on Saxion University, Netherlands (who used Surpass) and their implementation of e-assessment to deliver over 50k exams last year

GeoGebra for digitising mathematical equations, drawings etc

Unidoodle for allowing students to submit sketch style answers

Jisc guide to digitising learning and teaching materials sustainably

Resources from the e-assessment in mathematical sciences conference at Newcastle University, September 2016

KioWare used in the financial services education field for browser lockdown

Systems produced by Inspera and Uniwise were mentioned and further information about these can be found in the Jisc overview of systems supporting electronic management of assessment

Case studies on the e-assessment association website


Updated product comparisons

It’s a couple of years now since we had a flurry of activity around people sharing and discussing product comparisons (search on product features/comparisons if you want to look at the archives).

Since then we have published a set of supplier responses to the sector-wide specification (spring 2016) and this was updated in September 2016. As many Jisc and other resources link to the original blog post on this topic we have decided it is logical to keep adding updates at the bottom of that page. To see the original and updated supplier responses go to

As the products are continuing to evolve, your views on how well various combinations meet your needs are still of interest to others so please keep sharing. For those using Moodle and Turnitin the comparison on the UCL wiki is regularly updated


Digital Student Tracker: new and improved!

You said we didI’m genuinely excited about the new question set for the Student Digital Experience Tracker.

For the first pilot we cut things down to the bare essentials. We wanted the service to be simple, reliable, to achieve a high completion rate, and to demonstrate a demand. We made a  number of compromises to arrive at questions that tested well across all the sectors. And while we ran a popular and successful pilot, we knew we had probably cut things down too far. We were missing key questions on learners’ digital confidence and study habits, and on how digital technology influenced their learning. Our pilot sites told us these were issues they wanted to know more about; our consultation events confirmed it. And just as we encourage institutions to respond to their student data, we’ve listened to the feedback and we’ve made some changes.

Collecting feedback from the Jisc Student Experience experts’ group

You can now try out the revised core questions for the Tracker here. This is a generic version (we are preparing slightly different versions for HE, FE, Adult and skills, and Online learners). After browsing the questions we will ask you for some feedback to help us understand your needs and continue to make improvements. If you were familiar with the pilot Tracker you’ll see some new response options and some whole new questions. The Tracker will also be more customisable this time around, allowing you to choose some optional questions and to ask learners about your own current concerns using the same instrument. Although completion takes slightly longer as a result, we are confident we are helping you to answer the questions that matter most to you and your learners.

Collecting feedback on the core questions

Collecting feedback on the core questions

If you think your learners deserve a chance to answer these questions and to have their answers heard, please sign up to take part in phase 2.

More about the phase 2 Tracker service

As the Times Higher reports that technology use is critical to university rankings, and as FE colleges continue to respond to the FELTAG agenda, the Tracker allows you to gather reliable evidence about learners’ digital experience. In Phase 2, the Tracker:

  • is open to all
  • has been piloted and proven in practice
  • has been enhanced and extended
  • can be customised to your needs
  • can be implemented across the whole organisation or at a more specialised levels
  • is simple to use, with a single URL for users and guidance at every step
  • incorporates data benchmarking against your sector average

In order to sign up, you or another nominated institutional representative will have to complete a short planning sheet (the deadline for this has now been extended to 7 October). There is more information about taking part in the pilot on the sign-up page.

More about the updated Tracker questions

In line with our consultation findings, we have:

  • Added more detail to response options, making the data more precise
  • Added some new options to existing questions
  • Developed a new question set about how students use technology in their own study time
  • Developed a new question set about students’ digital capabilities
  • Changed the free text questions slightly, recognising that the free text data was highly valued but that the questions asked were too similar
  • Added customisable questions so that you can survey students about local issues alongside nationally benchmarked measures using the same instrument
  • Taken out a few questions that were misunderstood or did not prove useful in practice
  • Updated our guidance to better support the analysis of findings and to help frame appropriate responses

This has resulted in a longer question set but one that is still easy to complete and simple to customise. Crucially, we have kept most of the original questions so that you can compare results from last year if you are keen to carry out the process again.

A reminder that you can try the revised question set here, and sign up for the next phase Tracker roll-out here.

Virtual fieldwork and equivalent experiences

I thought I’d left this debate behind a while ago, but recently I noticed that “reasonable adjustments” and “equivalent experiences” have appeared in learning technology discussions. I should say that I think learning technologists have the best of intentions, but I also think that there are approaches that will lead to two tier provision, either on accessibility grounds, or on financial. As someone who started his HE career in virtual fieldwork, I am very protective of keeping fieldwork in the field. Discussing these issues with James Clay, we identified that some of the real benefits of VR and AR for fieldwork are actually not about accessibility, but about inaccessibility; we can collect  data from some of the most remote and dangerous places in the world such as pacific trenches, inside volcanoes and other planets. Creating rich interactive virtual field trips for these places, rather than places that are just a bit too hard to get to may both be more stimulating, and also teach us more about the technology we are using.

The text below is from a paper I wrote before I spent 5 years working in technology and disability, building the TechDis service and trying to apply the social model of disability to practice.

The idea that virtual fieldwork as an enhancement tool for “real” fieldwork is becoming embedded in departments and institutions, and more importantly the minds of academics involved with fieldwork. Examples of good practice can be found at a myriad of web-sites across educational cyberspace. Increased interactivity and widening access will no doubt accelerate the rate of development beyond that which is already prophesised.

One new area being developed at some universities is the concept of providing a fieldwork experience for students who are unable to access the real field course for various reasons including disability, financial or, in the case of some mature students, family commitments. This is a noble pursuit and worthy of the academic ideals for which all involved in education should strive. But in an educational climate of parity for all students, a virtual field course should, in principle, contain the same learning experiences as that experienced by students in the field.

In providing a comparable learning experience for disabled students the same experience, in a climate of parity, must be offered to able students. In the United Kingdom, it is common that students pay at least some of the costs toward residential fieldwork and that these residential field courses often make up a single module of a degree programme. Students in the UK are under increased financial pressure, and they operate within a system (modules) that encourages “strategic” learning i.e. learn only what they need to get through a particular module or stage of the course. If virtual fieldwork offers the same outcomes and is comparable with real fieldwork, at least in terms of learning objectives, then it should be expected that students will opt for the virtual fieldwork, given the academic and financial pressures.

For those involved in the teaching of fieldwork, the development of virtual resources for the ideal of providing a field experience for all must be tempered with the knowledge that there is a “real” world out there. And for students to truly become familiar with the concepts of our disciplines (be they geological, geographical or even social) we must place an emphasis on fieldwork in the field. Students must experience that which they study first hand.

Funding for learning and teaching development is a valuable commodity in UK higher education. Therefore, before developing field courses in cyberspace perhaps we should revisit the reasons that attract students to the disciplines. If a course offers fieldwork, rather than finding a way of substituting it electronically for disabled students, we should perhaps think about using the funding to break down the barriers which prevent a disabled student going “into” the field. For example, identifying field sites that are accessible for students with mobility issues, equipping university transport with wheelchair access or providing a “helper” when in the field. This approach enables both able and disabled students to experience fieldwork. Education for all is a right, in our disciplines we should acknowledge that fieldwork is an essential part of that education and ensure that we have ways of making it accessible to all in the “field”.

This extract is from an editorial, written at the request of the late John Butler (University of Houston), published in August 2001 in Computers and Geosciences. John was a pioneer of the use of technology in teaching of geosciences and we discussed the motivations for developing virtual fieldwork;  it worries me that learning technologists are still developing “virtual field courses” and using the equivalent learning experience as a reason. More worrying, is that I wonder at the times that this has been used retrospectively as justification to develop technology approaches because we (learning technologists) wanted to ensure further funding, or time, on a project. The social model of disability shows us that it is society that disables the person, not the disability; when we invest in equivalents that keep students out of the field, we are telling them they can never do fieldwork, that should not be the purpose of higher education.


That 5am Fieldwork feeling (after a few beers the night before) is hard to replicate in a simulation



It’s challenging…

…but planning helps!

Over the years I have spent a lot of time working with teachers helping them to embed digital technologies into their practice. I have also collaborated with colleges and universities and seen the strategies they use to embed digital. In an earlier post I described my journey and the approaches I have used for support and strategy. In this series of articles I am going to look at the process that many teachers use for teaching and learning and describe tools, services, but also importantly the organisational approach that can be used to embed the use of those tools into practice.

One of the challenges of embedding digital tech into teaching and learning is making the assumption that teachers are aware of and are able to utilise the digital tools available to them and understand which tools work best for different situations and scenarios.

Gaining that understanding and confidence isn’t easy and often requires a paradigm shift in approaches to using technology and the digital tools and services available. Just because a member of staff has been given the training in how to use the tool or service, it doesn’t mean they know how best to use that tool or service to enhance teaching and learning, and for what function or process of the learning activity the tool would support or enhance.

When I was teaching at City of Bristol College, one of the main reasons I started using and embraced technology was to aid planning my curriculum and lesson planning. The way it actually started was using technology to save time. By using, initially, a word processing package and then a DTP package, I would write and design assignment briefs, handouts and workbooks. The reason for using technology in this way was so I could reuse them the following year. Making them digital meant I could edit and update them if needed.

I also started using a presentation package (Freelance Graphics) to create presentations. There were no digital projectors back then, so these were printed onto acetates in black and white and shown via an OHP. This for me was much better than hand writing onto acetates, again for updating and changing.

Though I did write basic schemes of work for the curriculum at that time, it started to make sense to me to start creating a more detailed scheme of work.

When I noticed the web in the late 1990s I realised that hyperlinks could mean I could create a digital (though back then we called it electronic) scheme of work with live links to the digital resources I had created. It didn’t take much to then add lesson plans to the scheme of work with live links to the presentations, handouts and other resources.

A final step was to start adding extra resources and links, in order to allow for a learner to go to the web site and differentiate their learning journey.

I didn’t initially use digital technologies to plan, but what those digital technologies allowed me to do more effectively was to both plan better, but also link everything together. The process also allowed me to easily and quickly adjust resources and plans as and when required.

It got to the stage where I would plan a whole year in advance and have everything ready for all my lessons and courses.

When I spoke about this to people (outside my college) the response I usually got was I plan the night before and there is no way I could plan more than a week ahead. Their explanation was that they couldn’t know how a lesson would go in advance and therefore couldn’t plan more than one lesson in ahead. At the time I did struggle with a response, but now reflecting on this, I realised that I had in fact planned flexibility into my plans. Combined with links to all the resources and additional stuff, it wouldn’t matter if we didn’t cover everything in a lesson, or if the lesson was cancelled (snow closure for example). It was also later that I recognised as a teacher that though I had a responsibility for my curriculum, it wasn’t my job to teach the whole of the curriculum, it was responsibility to ensure my students learnt the curriculum. Some of this would be through teaching, but some could be through reading, or other learning activities. Some would be formal and some would be informal. Resources could be digital, but they could also be analogue.

Of course back then we didn’t have a VLE, so I “created” a VLE, well it was a website with some additional tools (such as a discussion forum). As I had used digital tools for planning and content creation, it wasn’t a huge job to transfer everything to the website. I do remember buying Adobe Acrobat so I could create PDFs more easily, especially I was using a bizarre range of software to create stuff.

A VLE today makes the whole process of planning much easier and I have written before about this in my series, 100 ways to use a VLE.

100 ways to use a VLE – #25 Scheme of Work

100 ways to use a VLE – #26 Lesson Plans

The main conclusion I came to was that planning was really critical to the success of my curriculum and my teaching. Also technology made the whole planning process easier and quicker.

So what tools are you using to plan your curriculum and your lessons?

Image Credit: Lesson Plans by hurricanemaine CC BY 2.0

It’s still not easy…


…but confidence helps!

Over the years I have spent a lot of time working with teachers helping them to embed digital technologies into their practice. I have also collaborated with colleges and universities and seen the strategies they use to embed digital. In an earlier post I described my journey and the approaches I have used for support and strategy. In this series of articles I am going to look at the process that many teachers use for teaching and learning and describe tools, services, but also importantly the organisational approach that can be used to embed the use of those tools into practice.

One of the challenges of embedding digital tech into teaching and learning is making the assumption that teachers are confident in their use of technology. Gaining that confidence is not easy and often isn’t helped if they have previously used technology and it didn’t work. There have been many times I have heard teachers say that they don’t like using technology as the last time they used it and it didn’t work. They lack the confidence in the tools to work.

The way I used to approach that was by asking what they did when it snowed and the building was closed, the campus had failed to work. Someone had used a permanent marker on the whiteboard, it was unusable. There was a room change and we had to move the students, from a seminar room to a lecture theatre. In all these physical scenarios, a good teacher has the confidence to adjust and adapt what they are going to do. With a snow closure, the scheme of work needs to be adapted to allow the learners to catch up. Losing the whiteboard doesn’t mean the lesson has failed, maybe a different medium, such as paper, could be used. Again a confident teacher can adapt what they are going to do. They are also very likely to use the whiteboard again, once it has been cleaned. Similar story with the room change, adapt the activity for the learners.


I have found that often with technology, that with teachers lacking confidence, this means going into a session with a limited idea of how that technology can be used. If it doen’t work as expected, then it is seen as failure.

Having the confidence to easily adapt and use tools effectively, usually comes with experience, but I also believe that there is more to it than that. Gaining that confidence isn’t easy and often requires a paradigm shift in approaches to using technology and the digital tools and services available. Just because a member of staff has been given the training in how to use the tool or service, it doesn’t mean they confidently know how best to use that tool or service to enhance teaching and learning, and for what function or process of the learning activity the tool would support or enhance.

Confidence usually comes from experimentation, trial and error and practice. It can be difficult to create a culture where experimentation and innovation is expected, encouraged and applauded. A culture where failure is seen as part of the learning process and is also part of the process of innovation.

So what strategies do you use to build digital confidence?

Presence, Digital, Well-Being, People

I have written recently about two areas of thinking around Digital Environments – How lecturers (and others) exist in relation to other people in learning spaces, how they are present; and also some of the technology approaches we are taking with regards to building new environments, such as analytics and adaptive algorithms. I started drafting something a while ago, but it was only after having a variety of conversations at ALT-C with colleagues both there and on v-connecting  that I decided to finally post my (still-developing) thoughts here.

What is emerging for me is the idea of being present in digital, how it is perceived and what are the unique (sic) features of practice in digital spaces. Having had face to face conversations with colleagues I have started to wonder about the Jisc Digital Capabilities Framework, and how and where this fits in.

Six Digital Capabilities

Six Digital Capabilities

Clearly it fits identity and well-being, but I can see an argument that the capability may eventually morph into Identity, Well-Being and Presence. Especially in the case, for example, to reflect the way lecturers need to be able to communicate their presence on a variety of both social media and institutional systems.

A recent example of this was seen in two different lecturers use of the anonymous social media service Yik-Yak. In one case a lecturer was doing a small learning and teaching project and had gone through an ethics committee and had strict guidance on how to behave and interact with students in the space. In the other, the comment “I just to pretend to be one of the students” was made.

As well as ethical boundaries, how you are behaving with your presence is a key feature of digital spaces. In the case of lecturers it will become important to be able to move between the physical and the digital seamlessly allowing students to feel their presence at distance and independent of space and time.

Elsewhere in the UK, and in the US and Canada, there are projects looking at the use of “siri” type apps to interact with students, and I am not arguing that this is either good or bad. However, as these type of tools start to develop it will become more important that staff understand how to portray the complexity of their personality, their essence, that is often easy to see when interacting in the physical, in digital spaces. It’s something that myself and Chris Thomson will be talking about in the digital leaders course (with thanks to Donna Lanclos for co-writing to the content) and it is also something that Lanclos and White  talked about during their ALT-C keynote.

Digital is People

Digital is People

Digital is about people, it always will be. But underpinning that statement there needs to be nuance. Digital is about people, it is about a set of behaviours; it is about a perception of others and self; it is another way of being present with those around us. We need to try to understand what all of this means and how it impacts on our lived experience, online and off.

App and Resource Store – soft launch

Posted on behalf of Justin Haylock, senior innovation developer.

Just over a month ago I wrote to tell you about the upcoming “Store” that my team and I have had the pleasure of developing.  I outlined the way in which the Store will allow for better discovery, the ability to rate and review resources and engage with the owners of the resources and the resources themselves to repurpose them for your own needs (where licensing allows).

I’m now happy to be able to share the first release of the Store with you and invite you to explore its current content (migrated from Jorum) and test and review its functionality.  This preview release is the first part of a staged development process, for details of our plans for additional functionality (including deposit) please see the previous post. Currently the Store holds a wide selection of high quality content from Jorum.  When you click on a resource you’ll have a brief description and an image of the resource to give you a better sense of what it is before you download it.  To access, rate or review a resource you’ll need to be logged in, which you can do through your institutional login or by registering on the website.

This preview release is the first part of a staged development process, for details of our plans for additional functionality (including deposit) please see the previous post.

Now it’s over to you. We would welcome your feedback on all aspects of the Store in order for us to improve as we go along.  Remember this is only the first release and with your help we’re hoping to go from strength to strength in building the highest quality platform of tools and services for the UK, and indeed the world, to enable the effective discovery and delivery of learning materials and resources.

To give us your feedback use the rate and review features just like you would any app in a Store and our resource curation application will allow you to raise issues and make requests in the same way open source software works.  You can rate and review any of the migrated Jorum content but we really want your views on the Store itself.  To make this easier we’ve actually put the Store inside the Store and opened it for curation! Just click on the link to the app and resource Store and then rate and review it as you would any other resources

We’ll be regularly reviewing any comments left and acting on them to continuously improve the Store and fix any problems so please share any thoughts no matter how big or small.  And if you have any ideas for resources you’d like to see or additional functions that you think would be useful please share those as well.  We look forward to working with you to make this the best service we can provide.

Being human #altc

One of the key messagesI took away form the Donna Lanclos and Dave White keynote at ALT-C this year was that we need to remember that there is no such thing as “the university” as we are “the university”.

When someone says “the university” won’t let us  do something, what they are actually saying is that a person in the university won’t let them do something.

We have to remember that policies, procedures and processes are not set in concrete and can be changed. I do realise that there are some legal aspects that mean some illegal activities are still illegal and that’s why you can’t do it!

The other key message for me was that tech and digital are not solutions, but people are. They may use digital for those solutions, but digital in itself is merely a tool to provide a solution. Without adequate training and support, digital tools are just tools.

I also liked their message that models  can hinder development, the use of hierarchical models that imply that this is a ladder to climb, when in reality you can often jump in at any point, and move between different sections, without necessarily needing to move on a linear journey upwards!

I made a couple of sketch notes from the keynote and as rightly pointed out to me, there isn’t much in them, but I did them more for me, than for other people.



These were done using Paper by 53 on an iPad pro with an Apple pencil.

Be online ready!


New resources launched

On Wednesday, during our ‘Creating online experiences that learners will value session at ALT-C we IMG_4231launched a range of online learning resources that have been produced as part of our Scaling up online learning and Digital Student projects.

Along with our report ‘What makes a successful online learner?‘ we launched our beta online learning readiness tool designed to help course/curriculum design leaders, learning & teaching staff and support staff.

The Online learning readiness tool:



  • Takes you through some key questions which identify your personal or teams readiness for creating, delivering or supporting online learning
  • Offers a reflective process by providing a brief synopsis of your experience levels and will offer further resources to take you forward.
  • Points you to key topic areas where you can review selected resources within our three guides and other resources

We are particularly interested in your feedback on the tool – visit for more information.

Scaling up online learning guides

We also highlighted our three scaling up online learning guides which can help with the processes and decisions involved in scaling up online learning at an organisational level. All three guides offer a set of useful checklists to help assess institutional readiness and include examples of useful tools, techniques and approaches. The guides are:

IMG_2639 (003)1. Scaling up online learning guide – providing a strategic view of different models and the implications of implementing online learning at an institutional level. This guide will be of particular interest to senior managers such as pro vice chancellors and directors of learning and teaching.

2. Curriculum design and support for online learning guide – helps you make decisions around curriculum design and to determine support needs when scaling up online learning.

3. Technology and tools for online learning guide – provides guidance, resources and case studies around the use of technology to support online courses and distance learning programmes.

Recommendations to online teachers

As part of an overview of the ‘What makes a successful online learner?‘ report, Helen Beetham shared a list of recommendations for online teachers during the session – for the full presentation that we gave, see below and for more detail on the work, findings and recommendations of the study, download the report.

  • Teach responsively, with consideration to learners’ different: motivations, interests, learning histories and resources
  • Prepare online learners to study online: norms, practices, expectations, good study habits, functional access
  • Enable learners to use their own devices, services and skills
  • Support access to rich and diverse learning content
  • Provide a digital environment that is accessible, social and personalisable: open (for some learners); secure (for others)
  • Address the barriers to success we have identified for specific groups of learners

As always, we are interested in any feedback you may have on our outputs and are happy to answer any questions you may have, so do get in touch.