Mountain Guides inspire Mountain Medicine Assessment Strategy

University of Central Lancashire Lecturer in Mountain Medicine and module leader Nick Wright examines the implementation of a newly adopted practical assessment strategy.

Establishing collaborative partnerships with industry experts is an excellent way to enhance student experiences through the development of enriching teaching and appropriate assessment techniques.  The PGDip Mountain Medicine has enjoyed a long-standing relationship with mountaineering instructors and British mountain guides, who are an integral part of course delivery and assessment. 

Our students are expected to meet demanding mountaineering skill requirements as well as demonstrate clinical skillsets and achieve academic learning outcomes as part of their studies.  A process of continuous assessment is a well-established technique in the practice of assessing mountain leaders and climbing instructor candidates as part of national outdoor training schemes.  Assessment candidates are under the watchful eye of assessors for multiple days rather than a single point of assessment.  Indeed, this strategy is used by our mountain guide team on the course to assess our student’s mountaineering competency. 

Our previous method of assessing higher level clinical decision making and procedural skills within a mountain setting specifically looked at students at a single point in time; a strategy we felt didn’t suit the assessment environment students we’re working in and the numerous elements we were attempting to assess.  Born from our desire to assess the authentic and holistic competence of our student group, we made the decision to align our summative assessment strategy to the one used by the mountain guides and the outdoor instructor industry.

The design of the new strategy was based on van der Vleuten’s assessment utility formula (CARVE):

Cost-effectiveness – our new method of assessment had no additional cost burden compared to previous assessment strategy.  There was no requirement to increase investment in teaching and learning for this strategy to be successfully implemented.

Acceptability – during the development of this assessment strategy we considered the beliefs, opinions and attitudes of assessors, students, external examiners and the external awarding body (we dual award an internationally recognised professional Diploma in Mountain Medicine as well as the PGDip Mountain Medicine) to ensure they aligned and little compromise had to be made between stakeholders.

Reliability – with no practical way of implementing workplace-based assessments, our new strategy replicates the real world as best as possible in the mountain environment students are expected to work.  Our method is potentially prone to subjectivity, but this is largely mitigated by ensuring assessors are adequality qualified and through a process of daily assessor meetings where each student’s performance discussed as a collective.

Validity – the assessment matrix has been adjusted following pragmatic testing and feedback and its current version is able to measure what it is intended to.  We have recently included the requirement for students to prepare and deliver a mini-topic presentation to their peers during their mountain journey, to help us assess the students contribution to evidence based medicine discussion.

Educational impact – this new assessment strategy results in the promotion of learning not just the translation of competent practice.  Students learn through the process of assessment which again aligns with practices used within the outdoor instructor industry.  Our assessment aligns with the learning outcomes of the module.

In order to assess our students against different levels of Miller’s pyramid we required an assessment ‘tool kit’ rather than a single technique.  Utilising multiple methods of assessment enhances the validity and reliability of results.

Students’ knowledge and performance is now continually assessed throughout the residential week at planned and opportunistic times through case-based discussion, casualty scenarios, mini-presentations, and other observed practice.  The following themes are assessed through 12 assessment criteria:

·       Providing patient care

·       Acting as a team member

·       Providing safe and effective scene management/leadership

·       Considering and appraising evacuation choices

·       Providing safe, evidence based clinical practice

Our new method of assessment has many advantages for the individual being assessed, the profession of the mountain medicine practitioner and for the assessor team.  Students no longer have to fear the one high stake opportunity to pass an assessment.  Teaching staff are able to give appropriate feedback to students during the week so they can focus on certain areas and improve their performance where required.  It is now possible to assess a multitude of skills across different contextualised scenarios as well as being able to assess higher order thinking through discussions around evidence-based practice, ethical and legal considerations.  This method also gives the opportunity for multiple assessors to observe the students during the week and give feedback, thus enriching the assessment experience further.

Closer collaboration between all assessors is key to making this method work.  Many hours are spent writing up notes and discussing each student as part of an assessor team meeting each evening.  Giving appropriate feedback/feed-forward to students is also time consuming but ultimately is a pillar to the success of this assessment strategy.

For more information about the PGDip Mountain Medicine click here. Or join us on Facebook by clicking the banner below.