Tick-box assessment through multiple choice questions receives a fair share of criticism, but the system is here to stay. Waltham Pitglow explains why

The quality of feedback from readers of this page shows a high degree of understanding of the process of learning and none more so than Alan Norrie of Harvington Underwriting Evesham who won our recent competition.Something we urged early on was that practitioners should learn about the concept and language of learning itself and in particular the process of assessing competence. Linked to that we championed the idea that practitioners should get used to speaking out and expressing opinions, but in a manner that was constructive, well reasoned and if necessary, firm.We have been receiving a growing number of letters from practitioners who are becoming (or have always been) sworn enemies of the multiple choice question (MCQ) as a form of knowledge assessment. Needless to say the writers tend to be mature and, in particular, those that did not experience the MCQ system at school.

System criticismsSome of the criticisms of the system are that: there is no writing and no opinion, it is peppered with negatives and double negatives, there are distracters and a successful outcome can be achieved with pot luck.The position was made crystal clear by a reader who made the point that we do not conduct our business as a form of multiple choice questions. Imagine the situation:Client to broker: "Now Mr Broker, can you tell me what an excess is? Is it:(a) Too much drink on a Saturday night(b) A defined deduction from a claim(c) A defined deduction that is only applied if the claim is under a certain amount(d) Too much sum insured?"All sorts of Pythonesque sketches come to mind.So today, let us remind ourselves exactly why the MCQ system is here to stay and what are its strengths. It is my opinion that the objection to MCQs actually lie in our own laziness, but read on….The critical aspect is that we have moved away from the subjective essay type assessment. It still exists – and quite rightly so – but in terms of basic knowledge, the underlying proposition is that there will be less risk if a practitioner knows a fact. MCQs provide the easiest was of checking as many key facts are known as possible in the shortest possible time (efficiency for all).

Available optionHow do we do that? Can we sit down with someone and ask them to answer 50 questions? Of course we can, although it is much more expensive. Any employer should consider that such an option should be available to staff.One advantage of MCQs is that there are often large numbers of practitioners who need to be assessed on exactly the same knowledge as thousands of others. Man marking is time consuming and expensive. Computers are quicker and cheaper. Marking a 100-question MCQ paper by hand (including handling time) can cost about £50. Systems such as BrokerASSESS can mark and give feedback on as many as you like for a fraction of the cost.Another advantage of the system is that there is very little bias in the system. No worries about examiners having off days or pet answers. The questions and answers are the same for everyone. If an MCQ is written properly, it can be an important learning tool.

So what are the key elements?1) The question and the answer.There are many arguments that flow about this simple start. Does the answer have to be exactly right? Should it be the most right of the options.2) The distracterIt is argued that every MCQ should have an answer which is nearly right or one might lead someone who does not know the answer away from the correct answer. For example:What was the largest island before Greenland was discovered?(a) The UK(b) Crete(c) Madagascar(d) Iceland.The UK, Iceland and Crete are clearly not right to anyone who knows anything about geography and the eye is drawn to Madagascar. But this is a trick question as the answer is Greenland because it was still the biggest even though it was not discovered. Such an answer immediately sparks widespread broker anger. One might say that Australia is bigger than all of these, although a broker academic from Dorking might point out that Australia is a continental mass and not therefore technically an island.3) The feedbackThis is a more modern trend and all credit to the training and competence specialists who have forced this one through. Did you know that less than eight years ago it was felt that someone being tested should not be given the answers and feedback because they might tell someone else the answer to a question and this might devalue the assessment?So the first learning point to anyone who questions the value of MCQ is the standard of the question itself. Questions that you should ask yourself are:

  • If I do not know the answer do I learn something?
  • Are the alternative options of value? Do they reflect answers that people give in real life when they get things wrong?
  • Is the feedback robust? Does it tell me why as well as what?
  • If I get the answer wrong is there enough there that I would be likely to get it right if I were to see the question again?
  • I urge all readers to keep persevering and if, ultimately, this is a method of knowledge assessment that you are really not happy with then say something to the person responsible for your training and competence scheme.The second learning point today is that for most jobs there are only a limited number of knowledge points used in day-to-day work. It does not take many MCQ questions to cover all aspects.

    Source materialThe key to success is facing up to reality when you get something wrong and if there is good feedback and reference to source material, take time out to look up the answers and supporting background information. I do wonder whether it is like the Charles Dickens syndrome. Every time I read Dickens, which is now less and less, I promise myself that armed with a dictionary I will look up all those long words that only Stephen Fry still uses.Of course, I do not bother. Just as I do not read up when I get the answer wrong in an MCQ. I come to the end of a Charles Dickens book no wiser than when I began in much the same way that I move on from an MCQ assessment. I do not bother to learn the right answer.

    Waltham Pitglow is a specialist in training and compliance This page is edited by RW Associates, a specialist in training, compliance and competence. Email: ruy.lopez@brokercompliance.co.uk

    Using this CPD pageFor the vast majority of practitioners and indeed support and supervisory staff in our industry, CPD is about regular learning and study that is planned, recorded, timed and evaluated. If you are a member of a professional body with a CPD requirement then there will be certain rules regarding the quality and nature of study material, and the way in which it is recorded.For staff of GISC members this means recording on your individual training file what the learning was, who provided it and when.It might be structured, such as a course, a learning programme or exam study. But it can be unstructured. This form of study encompasses reading the trade press, technical material or taking part in activities to support your professional body. Some CPD requirements are points related (a little antiquated) and others require a time value to be allocated. For example, it might take one hour to read Insurance Times each week. Most of that could be put as a time value but, in reality, perhaps only an half hour was devoted to learning something. The rule is to be honest with yourself and record the time that is relevant. Always take time to make a note of what you felt you gained from the activity. This is useful information for anyone else considering the same activity.In response to the popularity of our CPD programme each week's CPD page can now be downloaded from our website.

    Topics