Testing Standards

When identifying an educational need in a competence assessment it is important to select a course that covers the required knowledge gap as closely as possible. Petrina Oxshott explains the process

Suggesting that passing an exam is no use in assessing competence, unless one knows the questions and answers that the candidate failed to get right, was just me playing devil's advocate .

Of course, regular readers will know that we do promote the concept of qualification, but let us look at my statement in more detail.

Let us consider once again the compliance officer who has to confirm to the regulator that practitioners are competent. That means competent to do the job in hand.

Ignoring the generic and wide-scale nature of an exam as a basis for a qualification, the key to assessing competence (or the knowledge and understanding components) is a gap analysis. This identifies what is not known or understood from a standard set by the authorised firm, which could be internal or completely (this is unlikely) or partly drawn from external sources that have the skill and expertise to define what might be needed.

Let us suppose that the person responsible for compliance, working with the training manager and perhaps a standards committee, decides that for a particular job, 30% of the knowledge and understanding for the job is contained in the CII CPO1 exam syllabus.

In our opinion, although it is not yet very broker orientated (if the CII does not have plans afoot to address this I would be very surprised), the Certificate of Insurance Practice is a good basic qualification. So where is the problem?

Think about it carefully. In the first place, will the exam cover the syllabus fully and, in particular, will it cover those facts that you deem most important for the job in hand? Secondly, even if it does cover exactly the facts that the project team think are most important, what good is it to know that someone has passed without knowing the exact mark and those questions that the individual did not get right?

Regulatory breach
If it was a critical fact that has to be known for the job in hand, you have no way of addressing the gap and if you allow that person to continue working you may be committing a regulatory breach by allowing an incompetent person to do the job.

Of course qualifications should be encouraged. My personal view is that qualifications, such as the CIP, are invaluable in confirming a reasonable knowledge and understanding of the underpinning facts that are the basis of any sound practitioner working in the modern world.

But it is important that those of you who are responsible for confirming that staff are competent do not get sidetracked by the certificate-waving individual who tells you that they do not need any further assessment to prove competence.

The fact is they do, even if they are referred to as chairman, director, non-executive director or boss.

Which leads to another important point and that is the growing number of exams and certificates that are suddenly arriving on the scene. The latest flashed across my desk the other day and promotes a certificate that can be earned by attending up to 150 exam centres across the UK with a two- hour assessment conducted on a computer.

The leading general insurance education authority (the CII) and the leading representative of the broking community (Biba) are working towards a more modern and refreshing qualification and assessment programme.

So it does worry me that fringe organisations to the industry with no historical connecton to general insurance, suddenly send unsolicited mail-shots extolling the virtues of their wares.

The CII is not and should not be the only insurance educational establishment. We have often supported the Society of Claims Technicians on these pages, but the concept that any organisation can set up a certificate of competence without regulation is a little worrying.

Perhaps there ought to be some form of faculty which can govern standards and give some credibility to qualifications and the content of the assessments and feedback?

If you are going to spend time and money on qualifications then you might consider checking the following as an example of some of the things you might ask:

It may seem to be yet another administration task, but it could save money and time in both the short and long term.

Here is the true story of an organisation some years ago (no longer in existence thank goodness) that offered a form of certification. I had been teaching candidates for some time and was getting a pass rate of over 70% of candidates against the average of only 40%. My contract was not renewed.

Some time later I established that the reason was that the organisation was so concerned at the reduction in revenue from re-sits that it failed to renew the contracts of those tutors who had candidate pass rates above 55%.

Petrina Oxshott is an independent compliance and training specialist

To download a PDF of this article as it appears in the magazine click here .