Searching your way to the top

How does the average consumer look for a motor insurer? Increasingly, by typing “car” and “insurance” into an internet search engine such as Google.

And the site which comes out top of the list is likely to be the first one the consumer visits, meaning Google and its competitors have enormous power over price comparison sites, which live and die by their internet traffic.

Earlier this month, price comparison site GoCompare.com, formerly the site that topped the search list for car insurance, was downgraded to the sixth page of results and consequently suffered an 87% drop in the proportion of search traffic it received.

There have been other examples of price comparison sites suddenly plunging down the rankings. In the brave new world of search advertising, few people understand what determines which company comes top of the results list, or why they might fall.

Can canny online marketing play a role, or are insurers, brokers and online aggregators at the mercy of the search engines? And if so, how much impact does a search result downgrade have on business?

The most popular search engines in the UK – Google, Microsoft Live Search, Yahoo! and Ask – all offer a variety of marketing proposals, the most common being paid for search and natural search.

Paid for search is fairly straightforward – you can pay for an advert against certain search terms, which on Google will appear at the top of the page or on the right hand side of the page. Natural search, however, is where things can get interesting, and the new (and increasingly important) art of search engine optimisation comes in.

So what can companies do to increase their natural search results? Good practice involves designing your site to be search engine friendly – allowing ‘spiders’ to easily assess the information in the site, having relevant links to respected sites, and having lots of content which is frequently updated.

But sometimes all those things are not enough, and companies start looking for other ways to boost their results. And that can take them into a grey area. The search engines themselves police what can and can’t be done, though there is no official regulation.

According to Internet Advertising Bureau (IAB) programme manager Jack Wallington, the most common sins committed by companies wishing to improve their results are spam blogging, using RSS feeds to boost the amount of new content on their site (often without permission), paying for links, and tagging irrelevant keywords, including the names of competitors.

“Search engines are about relevancy and best practice. They will rate sites on how relevant they are, and on all the links leading back to the site,” says Wallington. “If they find the relevancy isn’t as high as it should be, they can assess all the links and place the site in the results where they think it would have come without the paid for or irrelevant links.”

Google clearly warns in its guidelines that paying for excessive irrelevant links can have serious repercussions for natural search results: “This is in violation of Google’s webmaster guidelines and can negatively impact your site’s ranking in search results,” it says.

Yahoo! likewise warns those using “methods to artificially inflate search engine ranking”, and it “reserves the right, at its sole discretion, to take any and all action it deems appropriate to ensure the quality of its index”.

How long such a penalty lasts can vary, says Wallington, but it would normally take one to two months to climb back up the rankings to somewhere near where it had been. All of which can be pretty bad for business.

Punitive action

GoCompare.com is recently thought to have suffered from such punitive action – though Google refuses to confirm to individual companies whether or why it has taken action.

The results were dramatic: according to online monitoring company Hitwise, the number one position saw GoCompare.com take 17.49% of all search traffic for the term in the week ending 26 January 2008. Following the penalisation, in the week ending 9 February 2008, the aggregator received only 2.31% of search traffic for “car insurance” from Google, a fall of 87%.

&#8220It is so competitive out there and there are a lot of smaller players who are prepared to do what they can to get up the ranking. It makes it harder for us to stay at the top.

Sarah Baldwin, Confused.com

However, the situation is not as bleak as those figures might initially suggest. Moneysupermarket.com had a similar run-in with Google in the summer of 2007, where its natural search result was downgraded for about a week.

Moneysupermarket.com head of insurance Richard Mason says the experience had been frustrating as Google would not acknowledge that there was a problem and it simply had to hope its pleas for forgiveness got through to the right people.

“Fortunately because we advertise a lot on TV and we have over 200 portal partners, we get a lot of our business from other partners, so for us it didn’t have that significant an impact,” Mason says.

Detrimental to business

But others say such a downgrade has the potential to be extremely damaging for business. “We get as much as 70% of our business through search engines,” says Comparethemarket.com marketing director Mark Vile. “It would be really detrimental to our business if we were no longer appearing.”

The loss of natural search placing can, of course, be offset by spending more on paid search and other online advertising, but doing so would mean increasing the advertising and marketing spend. Comparethemarket.com currently has a roughly 50/50 split between traffic coming from paid for and natural search, indicating the size of the problem any dramatic drop in natural search placing could cause.

Indeed GoCompare.com’s webtraffic appears to have been fairly stable over the past month, according to statistics from web information company Alexa, suggesting that sites may be more resistant to the actions of search engines than many would have thought. ‘

Confused.com marketing manager Sarah Baldwin says the relationship between search engine and site is more even than many people may think.

“We have done search engine optimisation since day one. They [search engines] don’t have us over a barrel as we have strong organic links for natural search. So it’s a more even relationship,” she says.

Mason says: “They are trying to deliver what customers are looking for. If someone puts car insurance into Google it will try to order the sites as to what it thinks the people are looking for. We have to play by its rules, but that doesn’t mean that we are frustrated by it: we know what we need to do.”

Increasingly resistant

In addition, the big price comparison sites, or aggregator brands, are increasingly resistant to the attempts of search engines to police their own results. As GoCompare.com, Moneysupermarket.com, Comparethemarket.com and others spend more on marketing their own brands, consumers can increasingly find them without the aid of search engines.

Paid for results are also excluded from any action which might be taken over manipulation of natural search results. The algorithms used for paid and natural search are totally different.

While GoCompare.com might be on page six of Google for natural search, they’re right up there on page two for paid search.

So why risk the displeasure of the search engines by trying to improve your rankings? “It’s so competitive out there and there are a lot of smaller players who are prepared to do what they can to get up the ranking. It makes it harder for us to stay at the top,” says Baldwin.

Mason agrees: “If you put car insurance into Google you get over 100 million results. It’s the most competitive part of the web. It is the absolute zenith of competition.”