Deciding how to assess
Collecting the evidence
Collecting ‘sufficient’ evidence is essential in making a judgement about the attainment of competence. Sampling widely across a number of clinical and contextual situations is important to overcome the problem of case specificity.
In the assessment of ‘work’, in contrast to traditional methods there is no single ‘controlled’ method that can be developed. A variety of sources of information will be needed, which gives rise to the notion of a ‘tool-box’ of approved methods (see below). In considering the individual tools it is worth recognising that, even unstandardised, they can be made sufficiently reliable, provided enough sampling occurs, and the tools are used sensibly and expertly (van der Vleuten and Schuwirth, 2005). However, it is important to remember that the tools form part of the overall assessment programme. Attention should focus on the reliability of the entire programme of assessment, not just the individual tools themselves.
From a feasibility point of view, it is likely that a lot of workplace-based assessment activity will occur locally, i.e. within the workplace setting. This calls for effective educational supervision. An educational supervisor should be responsible for co-ordinating the assessments, but the process of collecting the evidence should be learner led.
The educational supervisor can also be involved in making judgements about the learner’s progress against the competencies provided. The potential role conflict for the educational supervisor responsible for facilitating progress and making judgements on it can be overcome, provided the differing roles are made explicit to the learner. In other words, when it’s assessment time, the learner needs to know, and be adequately prepared for it.
The confidence in the reproducibility of judgements in a workplace assessment can be improved through triangulation within the workplace assessment. This involves using different methods to collect evidence and using the views of multiple raters over a period of time.
Triangulation with other assessments external to the workplace is also important. An overarching assessment strategy is essential, in which workplace assessment is supported by rigorous tests, e.g. those of ‘knowledge’ and ‘skills for clinical method’. It is recommended that there should be an overarching assessment strategy (i.e. a programme for assessment) for the whole training period, and that this is blueprinted to the curriculum. Workplace assessment and other appropriate methods form part of this.
- Think of a professional competency area from your own specialty, or the Foundation curriculum. What do you think would constitute ‘sufficient evidence’?