The blessing of assessing complex skills

The assessment of professional competence, particularly in medicine, has always been slanted to the assessment of knowledge. Knowledge is relatively easy to assess.

We can use all of our standardized testing technology (item writing, psychometrics, standard setting, etc.) to make valid assessments. In the seventies we began to realize that skills are also important to assess. The OSCE was brought to life and truly conquered the world. We again used our standardized testing technology, now to assess performance in standardized and simulated settings. A zillion papers have been written about OSCEs and a true “OSCE-ology” has emerged.

Then competency-based education came to life in the nineties. Competency frameworks have been developed in many different countries of this world. One of the striking features of these frameworks is that they emphasize complex skills such as professionalism, communication, collaboration, etc. It is very hard to assess these complex skills with standardized technology. Imagine making a checklist for professionalism. If we would actually do that, we would quickly trivialize what we try to evaluate. Hand-in-hand with competency-based assessment work-based assessment emerged. Assessing complex skills in the real (unstandardized) world posed us with major challenges. We had to start measuring the unmeasurable! Our traditional standardized technology, however, failed us in tackling this problem. We noticed that very summative approaches did not really work well. It leads to ticking boxes, strategic behavior and trivial behavior. 

To tackle this problem, we asked ourselves: how is learning promoted when assessing these complex skills? We found out that a learning culture needs to be established. In a learning culture the purpose is to focus on meaningful feedback that can be used to improve the learner. We discovered the importance of dialogues and trusted relationships between learners and assessors. And we discovered the importance of words. To assess complex skills scores hardly have any meaning. But words do. Narrative information turns out to be much more valuable than scores. These discoveries bring us to a whole new era. We are going to move from an “assessment of learning” discourse to an “assessment for learning discourse”. Central is the learning of an individual and how this learning is best promoted. We move from “assessment drives learning” to “learning driving assessment”. We move from a quantitative discourse into a qualitative discourse. We are asking fundamental questions like: why are we trying to quantify skills that hardly lend themselves for quantification? The pendulum swings back again and we are embracing the subjective.

We haven’t figured it all out yet. But these are exciting times. A different discourse is like a fresh breeze of wind. That is why I think the assessment of complex skills is a blessing for the assessment of professional competence. We have learned a lot in the past number of years and we have quite some discoveries ahead of us!

Cees van der Vleuten