On 9th December we held a workshop involving many of the contributors to this blog along with other staff from HEIs to take a service design approach to addressing the EMA challenges you have identified. It was a lively and participative day that resulted in 30 solution ideas. The ideas were discussed and voted on by participants in order to identify those that had most potential to provide real benefit to a significant number of stakeholders. We will be holding a further workshop on 22nd of January to further refine the solution ideas and come up with an action plan for taking them forward.
So far the ideas gaining the most support have grouped around five main themes. What is interesting is that between them they address all of the top 10 prioritised challenges and also that there is clearly no one size fits all approach to some of these problems – many of the 30 ideas represented different ways of tackling the same problem. Here is a summary of the most popular ideas to date:
Solution Group 1. – Common Workflows
Challenge/s addressed: Ability to handle variety of typical UK marking and moderation workflows/ Ability to manage marks and feedback separately.
We had a range of ideas around identifying, validating, specifying and gaining consensus around a common set of marking and moderation workflows. There was considerable interest in research already done by the University of Manchester into the range of workflows that exist across different disciplines and enthusiasm for the idea of validating these further across the sector. If we are able to narrow down the diversity of approaches into a set of common models it could help to both inform systems suppliers to influence how systems develop to support those workflows; and also to inform new systems development.
The ideas ranged from simply documenting these workflows in broad terms through turning them into more detailed specifications, to the idea of actually building ‘plug and play’ modules. We were also reminded of an existing open source tool built by the University of Southampton that was designed to deal with many of these workflow scenarios.
Solution Group 2. Holistic Feedback Hub
Challenge/s addressed: Student engagement with feedback/ Ability to gain longitudinal overview of student achievement.
There was consensus around the need for a more programme level/holistic view of feedback, for both tutors and students, to enable a more longitudinal view of student development as well as potentially facilitating greater engagement with feedback. One proposed solution was to develop a ‘holistic feedback hub’, where students and staff can access a programme level view of student feedback (it was noted that the IoE has already developed a tool in Moodle to do this). Another idea was for students to be empowered and enabled to take more ownership of pulling together a programme level view of their feedback by gathering this in their personal spaces (such as an e-portfolio). Ongoing conversations around feedback can be captured in the e-portfolio as part of ongoing engagement with feedback. Students could be encouraged to share their views of feedback with personal tutors in preparation for discussions during their tutorials.
Solution Group 3. Reliable Submission
Challenge/s addressed: Reliability of submission systems.
The ideas in this space focused around making the technical process of submission as simple as possible and clarifying policies and procedures to avoid stress and confusion when things inevitably do go wrong. It was suggested there is a need to analyse all of the possible points of failure and decouple the physical act of submission from the workflows within other EMA systems so that submissions can be acknowledged and held until other functions are in a position to proceed. Policies, procedures, guidance and examples need to encompass the workarounds to deal with points of failure.
Solution Group 4. Interoperability
Challenge/s addressed: Lack of interoperability between marking systems and student records systems/ Ability of systems to support variety of grading schemes.
The ideas relating to this topic covered both data management and technical interoperability. It was suggested there was a need to identify the minimum data storage requirement for each type of system and to consider whether each institution is carrying out functions in the most appropriate system and storing the data in the most appropriate place. There is a need to exchange good practice and existing solutions for common integrations and it was suggested we could go so far as to build some integrations where there are gaps.
Solution Group 5. Good Practice Toolkit
Challenge/s addressed: Need to develop more effective student assessment literacies/ Risk aversion/ Academic resistance to online marking/ Need for greater creativity.
A number of solution ideas relate to the development of guidance and examples to promote an ‘assessment for learning’ rather than ‘of learning’ approach. The suggestion is for some form of toolkit which should address the question ‘what does good assessment design look like?’ and enhance both staff and student assessment literacies. Suggestions for content relate to programme level assessment design, principle led approaches to assessment design, encouraging more dialogue on, and engagement with, feedback; mapping the programme to provide a holistic view; looking at how to quality assure feedback and visioning far-reaching assessment design possibilities.