Assessment tests need to reflect the target level of competence in an organisation. Kate Foreman explains
Online learning or assessment systems for measuring or increasing technical knowledge is an effective tool, but its effectiveness can be greatly reduced in the wrong hands.From a risk assessment point of view it is not how much employees know, but how much they don't know. The five questions that an individual gets wrong could result in a complaint or, worse, an accusation of negligence.So what is the best way to establish an individual's knowledge? Well, a good benchmark might be passing a relevant examination. It is generally agreed that possession of an ACII is the industry standard for knowledge. However, when using examination passes as a benchmark, make sure that you know what is in the syllabus and at what professional level the exam is aimed. It is no good expecting a 19-year-old who has just passed the foundation insurance test to be able to deal with complex liability issues.
Training historyConversely, many who can deal with such issues don't have a formal training history and may never have taken exams. This is a common situation within the industry. Many of those people will not want to start taking exams for a variety of reasons, but you will still have to establish their level of knowledge in order to satisfy yourself that they are indeed competent.So how many questions do you need to get right to demonstrate competence? Would it be 100, 20 or just one? Certainly the questions that you select must meet certain criteria: they must be at the right level in terms of complexity, and there must be enough of them to make the assessment meaningful. This means selecting individual subjects for each assessment. So for example, you might decide to assess knowledge of liability. I would suggest that you will need to ask at least 20 questions - and, with a subject of this size and complexity, possibly many more. How many would you expect the individual to get right? Let's look at this logically.If I say I expect you to get 100% correct in order to be assessed as competent, that sounds a bit much doesn't it? However, if you are only setting two questions on the subject, it doesn't seem quite so daunting. It is safe to say that asking one or two questions on a subject does not confirm an individual's competence, even if they do get 100% correct. If we look at a realistic number of questions for each assessment on that subject (and there may well be several) say 25, and we decide that we would like to see a minimum of 20 correct we may be getting closer to a realistic and consistent assessment of an individual's knowledge. I have used the example of liability as it is possible to break the subject down into separate areas. Obviously, that is not always the case, so common sense when deciding on the number of questions that need to be answered is paramount. It is also important to remember that if an individual does not get 100% each time there are still gaps in their knowledge and it is the gaps that are potential problems. You must ensure that you address the questions that were incorrectly answered and ensure that the individual understands the correct answer.It is vital that you understand what is required of employees that are being assessed. That means you must be taking assessments as well, firstly because you need to understand the level of questioning and secondly because you also need to show that your own knowledge is up to the mark.While we encourage you to make assessments fairly 'informal' it is vital that they are robust: they should stand up to scrutiny if challenged.
