The recording of our webinar ‘Online exams: migration or transformation’ run jointly between Jisc and EUNIS http://www.eunis.org/ is now available.
Listen to the recording on our YouTube site https://www.youtube.com/watch?v=MjMNanbMpMQ to hear the views of:
- Stuart Allan, who recently completed an MSc on the subject at the University of Edinburgh
- David Parmentier, Sogn og Fjordane University College, Norway
- Annette Peet, SURF , Netherlands
- Martyn Roads, consultant specialising in assessment in the UK FE and skills sector
You can find out more about the background to the webinar in this blog post https://ema.jiscinvolve.org/wp/2016/08/01/online-exams-migration-or-transformation/
The session was held using Blackboard Collaborate and the recording is made available on YouTube to maximise its accessibility. This means you don’t have access to the chat that was taking place so here I have jotted down a few comments and observations from the chat as well as links to some of the key resources suggested by the presenters.
We were using the term ‘online’ in a broad sense: really this was all about summative testing in digital format not necessarily implying that there needs to be full Internet access at all times.
‘Pedagogic’ reasons for moving to digital exams include enhancing the curriculum and better preparing students for employment.
People are often worried about the cost of migrating to digital without having any real idea of costs/benchmarks for existing paper processes.
Interesting point about how students think when writing vs typing. Mogey and Fluck (2015) found that in computer-based exams students are sometimes more concerned with maximising the number of words they can include in their response than with the construction of academic arguments. Mogey N. and Fluck A. (2015) ‘Factors influencing student preference when comparing handwriting and typing for essay style examinations’, British Journal of Educational Technology, 46 (4), pp. 793–802
In relation to collaboration on item banks – technical incompatibility between systems seems to be an issue. Manchester University’s UKCDR study looked at this – medical education boards share successfully across institutions. If the on-screen exam/ item building system is QTI compliant, that helps with migration of items and exam templates. https://www.imsglobal.org/question/index.html There was also a comment that the education sector needs to look beyond QTI and more at API based content integration.
UK-based awarding bodies and commercial education companies in professional education outside of Ofqual’s regulation have deployed onboard remote invigilation. Service support and cultural acceptability (especially if the remote invigilation provider is non-UK based) are proving the main barriers to adoption.
‘Surely if students are taught in the same method of delivery as online assessment then they would be more in tune.’
‘The digital literacy of staff is something we are also experiencing as a challenge. The students are way ahead of anyone, so not a problem. Assessors seem to be the least prepared for an entirely digital exam.’
English government policy has retrenched to terminal summative assessment. Scottish government keen to use digital evidence in qualifications. http://www.gov.scot/Resource/0050/00505855.pdf
Resources from the presenters
You can find Stuarts blog at: https://stuartallanblog.wordpress.com/dissertation-abstract/
Here’s link to Myyry’s and Joutsenvirta’s publication as recommended by Stuart: http://alh.sagepub.com/content/early/2015/03/24/1469787415574053.full.pdf
White Paper on online proctoring (in English)
Innovation in digital assessment: assessment themed issue (April 2016 in English)
Assessment Security Selection Model (in English)
https://www.surf.nl/en/knowledge-base/2016/assessment-security-selection-model.html
Transforming assessment and feedback with technology guide: http://ji.sc/transforming-assessment-feedback-guide
EMA processes and system guide: http://ji.sc/ema-processes-systems-guide
Supplier responses to system requirements: http://ji.sc/supplier-responses-ema
EMA readiness tool: http://ji.sc/emaready
2012 UK Landscape report: bit.ly/jisc-ema
Resources suggested by participants
Surpass Paper+ assessment system as a means from migrating from paper to digital exams https://www.youtube.com/watch?v=ENAmkPWVN6k&feature=em-subs_digest
Case study on Saxion University, Netherlands (who used Surpass) and their implementation of e-assessment to deliver over 50k exams last year http://www.btl.com/case-studies/
GeoGebra for digitising mathematical equations, drawings etc https://www.geogebra.org/
Unidoodle for allowing students to submit sketch style answers http://www.unidoodle.com/
Jisc guide to digitising learning and teaching materials sustainably https://www.jisc.ac.uk/guides/digitising-your-collections-sustainably
Resources from the e-assessment in mathematical sciences conference at Newcastle University, September 2016 http://eams.ncl.ac.uk/
KioWare used in the financial services education field for browser lockdown http://www.kioware.com/kwb.aspx
Systems produced by Inspera and Uniwise were mentioned and further information about these can be found in the Jisc overview of systems supporting electronic management of assessment https://ema.jiscinvolve.org/wp/2016/02/24/supplier-responses-to-uk-he-ema-system-requirements/
Case studies on the e-assessment association website http://www.e-assessment.com/