Skip to content. Skip to navigation

Faculty Development

You are here: Home / Workplace Based Assessment / Principles of assessment

Principles of assessment

The traditional educational paradigm below indicates the importance of integrating teaching, learning and assessment, where assessment should figure as an integral part of educational planning. However, it is all too easy to view assessment as a ‘bolt-on’ component at the end of an educational programme.The educational paradigm

 

 

 

 

 

 

Utility

The utility, or usefulness, of an assessment has been defined as a product of its reliability, validity, cost-effectiveness, acceptability and educational impact (van der Vleuten, 1996). In later years the term ‘feasibility’ has been added. Utility can be expressed as an index:

Utility = educational impact x reliablity x validity x cost effectiveness x acceptability

 

Utility can be applied to an entire assessment system or to an individual assessment method or component of the system. The concept is important in that no single element of utility should be regarded as a panacea. Instead, assessment design should pay attention to all the elements within the utility equation, although it is recognised that there may be a ‘trade off’ between the elements.

Thus, traditional approaches to maximise the reliability or reproducibility of assessments can have a negative educational impact on the learner by reducing the opportunity for meaningful developmental feedback. Whereas workplace-based assessments can offer high educational impact but might not be performed as reliably as other medical assessments, e.g. MCQs.

From methods to programmes 

Traditional approaches to developing medical assessments have been founded on the notion that domains of medical competence (e.g. problem solving, communication skills) are stable and generic. As a result, it is possible to design tests that assess these domains separately and reliably. This has led to the ‘one trait, one instrument’ approach to assessment (Schuwirth and Van der Vleuten, 2004).

However, there is a growing realisation that medical competence is specific to particular clinical situations or clinical contexts, and not generic. This is called case specificity and poses particular problems in assessing doctors. To overcome this, the most important thing is to sample widely across the content of the curriculum.

In recognition of the complexity of assessing professional competence there is a growing realisation that assessment should be considered as a programme of activity requiring the assimilation of quantitative and qualitative information from different sources. Assessing doctors in their actual working environment therefore offers the opportunity to gather sufficient information to build up a ‘rich picture’ of them. 

This also has a powerful impact on work-based learning through the provision of feedback and encouragement of reflection, and takes us right back to the educational paradigm.

Thinking point

  • Try applying the utility index to an assessment method with which you are familiar. What are the assessment’s strengths and weaknesses?

 

 

Print module to PDF

Save a PDF of this module, so you can print it and read it in your own time.

Email your comments

Let us know what you think about this module or give us your feedback.

Further information

More information about this module, further reading and a complete list of glossary terms.

Learning activities

Read about the recommended learning activities for this module.