Take surveys with more than a pinch of salt

17 May 2015 - 02:00 By Jeremy Maggs

Apart from Ed Miliband's awkward mastication of a bacon sandwich, which no doubt will inspire a ribald advertising campaign of some sort, the UK election raised a far more important issue - the veracity of believable market research. The professional association that governs British polling organisations is to launch a full inquiry into the huge inaccuracy of polling during the election campaign, saying it would look at apparent bias in polls across the board that saw large disparities between the industry's final snapshots of opinion and the real result on the night.The marketing research industry in South Africa might also consider soul-searching. Research is widely used in the brand industry, both to pre-test adverts before they are unleashed on the public and to measure the size and composition of an audience a brand wishes to reach before choosing a media platform.story_article_left1Advert testing has been anathema to ad agencies for as long as I can remember, as they claim effective and creative work based on hard-mined insight is perpetually held hostage by a sample of well-meaning consumers who don't necessarily have the real interests of the brand at heart, have little understanding of the strategy, and are often in a one-way mirror room just for the small honorarium and the soggy egg sandwiches.Great breakthrough adverts, say industry professionals, have been jettisoned on the whims of a badly moderated and poorly chosen group and the decision gratefully accepted by a nervous brand manager who is comfortable erring on the side of caution.But it's audience measurement or perception-gathering issues that are of more worry. For some time there has been muttering about unrepresentative local samples, which, according to research company Pondering Panda, often leads to invalid results being delivered to clients.A paper that will be delivered at the upcoming Southern African Marketing Research Association's annual conference will, I'm told, address the flaws in traditional sampling designs, and demonstrate how coverage bias and non-response can skew the results of surveys, leading to incorrect findings.Often, says Pondering Panda, it comes down to who is providing respondents to sample. "If you are only using one sample provider, results are going to be biased, whether respondent selection is supposedly random or not," says CEO Diane Gantz.story_article_right2"What researchers need to do in the new world of digital research is to ensure that they are conducting surveys across multiple respondent bases, chosen to ensure representivity of the target market."She says that although this doesn't eliminate the problem, it does mitigate it by balancing out the biases inherent in individual bases or groups, as no single base or sampling frame is ever free of bias.So while the research industry is grappling with its destiny, brands should always take results with a large handful of salt and be prepared to ask some tough questions of their agencies when the results seem too good to be true.Maggs is a writer and broadcaster and edits the marketing website theredzone.co.za..

There’s never been a more important time to support independent media.

From World War 1 to present-day cosmopolitan South Africa and beyond, the Sunday Times has been a pillar in covering the stories that matter to you.

For just R80 you can become a premium member (digital access) and support a publication that has played an important political and social role in South Africa for over a century of Sundays. You can cancel anytime.

Already subscribed? Sign in below.



Questions or problems? Email helpdesk@timeslive.co.za or call 0860 52 52 00.