Validity

Validity
Validity addresses whether the assessment measures what it is supposed to measure (Gormley, 2011; Van der Vleuten, 1996).
For both the OSCE stations that I have designed, I have created a complimentary mark scheme. As discussed previously, I opted to produce a global rating scale for the history-taking station and a checklist for the procedural skill station (appendix 1 and 2).
Weighting of items can improve validity of a checklist, which can affect which trainees pass or fail (Sandilands et al., 2014). On reflection, perhaps I could have assigned a greater weighting to the clinical and patient safety components of the checklist rather than the method that I adopted of awarding an automatic fail grade to candidates who demonstrate
I specifically asked for feedback on how they would feel as an assessor examining candidates undertaking the OSCE stations that I have designed. The feedback was overwhelmingly positive and suggested that examiners felt that my stations cover reasonable content in the timeframe allocated and that my OSCE stations are clear and easy to follow. All of my colleagues agreed that they would feel comfortable assessing candidates within the constraints of my OSCE construct. However, there were some minor recommendations relayed based on their evaluation of my OSCE station design. The main points of improvement were regarding further clarity with regards to the mark scheme, especially the global mark scheme used for station 1. Comments suggested that I should include further guidance to guide examiners when giving an overall score for station 1.
Further to this, as a recent former medical student, I used my experience in conjunction with the feedback from current fourth and final year students at The University of Manchester to further evaluate the acceptability of the OSCE stations that I have constructed from a candidate perspective. Given my own experience of OSCEs and the experiences and expectations of the medical students, it was agreed that the OSCE stations were a reasonable assessment of their knowledge and skills, and therefore
The main obstacles to reliability are a poorly designed marking scheme, impossible time frame and inadequate sample curriculum.
In future, to aid further development of OSCE station construct skills, I will carefully consider the finer details of the mark scheme, including considering writing additional guidance for examiners to be used as an adjunct to the mark scheme. I will also consider offering more specific written guidance for examiners and simulated patients or patients involved in the OSCE station set-up, so that they are able to behave in a consistent manner when dealing with questions that they do not have sufficient information to be able to answer.
As physicians, patients bear the consequences of our decisions. Therefore, it is important that achievement of clinical competence to the expected level is assessed appropriately and maintained. OSCEs are one of the key assessments that can be used to evaluate clinical