Christene McCauley
Measurement and evaluation (M&E) can be a hugely demanding activity in terms of time, resources and expense. While most client firms recognise the importance of evaluation, it is not always easy to achieve robust, cost-effective and useful results that inform business decisions.
At Diageo, we have been wrestling with these issues for ten years. We have found that, if an organisation establishes some guiding principles and ways of working, and focuses on building marketing and consumer planning capability, it can navigate the evaluation minefield.
Given the extensive Diageo brand portfolio and geographic spread, there are many brand activities in the marketplace at any one time. Potentially, this could create enough evaluation to occupy armies of planners and researchers.
To help consumer planners focus their efforts, there are three guiding principles that shape how Diageo applies best practice in evaluation.
Simplicity, not complexity
Taking a pragmatic approach, rather than an academic one
Focusing on getting the inputs right, as opposed to obsessing about evaluation of the outputs.
The first two principles are in no way revolutionary; they are just good practice and are probably adopted by most client companies. Diageo develops best practice for the key marketing disciplines – advertising, sponsorship, experiential, relationship marketing, PR, packaging and digital marketing activity – centrally, with input from key markets and then trains it in via consumer planning teams.
This embeds best practice approaches and guidelines across the globe. The process of centre-led creation of best practice ensures consistency and ease of application around the world, but also frees up the country teams to focus on development and execution instead of wasting time creating a technique someone else has already thought about. If ten people are working on an advertising pre-testing approach, nine of them have wasted company resources and limited the ability to share findings across markets.
MODELS VERSUS GUIDELINES
For some forms of evaluation, such as advertising, there are quite directive Diageo models and tools. But for other disciplines, such as sponsorship and PR – which vary greatly in terms of objectives, content and how they are implemented – guidelines are given, rather than firm recommendations.
The third principle, (input-focused, rather than obsessed about measuring output), is part and parcel of why any measurement and evaluation is conducted. At a very basic level, the purpose of measurement and evaluation is not just about knowing whether and how an activity worked, but being able to apply the learnings and change something in the marketplace by doing more of what is working well, modifying activities that can be improved, and calling a hard stop to those that do not deliver. In other words, proving to improve.
However, often the time and budget required to conduct meaningful evaluation means the learnings aren’t applied until the next brand planning cycle. Putting more investment, time and resources into developing the input to improve the current cycle of activity may be a better way forward than employing armies of people to create dashboards, traffic lights, top-line reports, monthly reports, quarterly reports and deep-dives to demonstrate the effectiveness of outputs from the cycle of activity that has been completed.
If you want simplicity and pragmatism, it is worth considering carefully how and when to use proprietary techniques. The Diageo stance is to avoid buying into proprietary research techniques that tie it to any one supplier unless there is good reason to do so. There are only two exceptions. One is the company’s global consumer segmentation study, which provides a consistent model to manage our brands globally, and the other is quantitative TV advertising pre-testing, which provides a global database of norms to use when assessing TV advertising performance. For all other research, the most appropriate approach for the business is selected, and then ‘Diageo-fied’ for internal teams.
For example, there is a common global brand tracking study used to monitor the equity of Diageo brands and to evaluate the consumer response to large-scale activities, particularly advertising. Consistency of approach enables Smirnoff or Guinness to be compared within, and across, geographies, and also means the same supplier does not necessarily have to be used in all markets to achieve this, but for cost efficiencies, regional suppliers are used for this study.
GETTING IT RIGHT FROM THE START
Advertising is still one of the largest marketing investments Diageo makes, and far more time is spent getting the thinking right with the consumer than is spent tracking and proving effectiveness of it once launched. To equip planners to do this well, training is provided that focuses on the areas in which planners can make a difference. It covers the different stages of defining, developing and delivering advertising activity and barely touches on the final stage of evaluation.
This means time is spent both on understanding the critical areas of the brief, which planners need to own and be accountable for, and how the target consumer can be brought to life when briefing agencies. Qualitative creative development research is covered in detail, specifically understanding what type of research is required, and appropriate stimulus material to use depending on the stage of the ad development process. It is also crucial to know how to use quantitative TV advertising pre-testing constructively to optimise ads, and how to ensure learnings stick right through to the production of the finished film, poster or digital communication. Focusing energies on doing all these things well increases the chances of success of the activity.
DEMONSTRATING PAYBACK
Inevitable questions arise in any discussion of accountability. How great is the pressure from management to demonstrate the financial payback of marketing communications? How is payback realistically achieved, and has it made any difference?
After putting these questions to the planning teams at Diageo, there is variation in the stance taken by management to demonstrate accountability for investment.
“There is moderate pressure,” says one, “and there is general acceptance that the currently available methods of measuring financial payback are expensive and of questionable validity.”
But others appreciate the difficulty of determining the financial payback on communications, and are prepared to settle for much softer measures from the brand tracking study.
“Good intentions are there, but not a lot actually happens,” claims another. “General managers seem to be happy with brand tracking results.”
In some markets (and some of the larger brands) econometrics and marketing mix modelling is conducted, but even then some questions remain about accuracy and reliability. “Econometric modelling has given us as good a steer as we will ever need in the real world,” one planner agrees, “but it is not an exact science, no matter what people say.”
In other markets, people are either not going down the road of proving financial payback at all, or they have given up even trying to do so. “I have been asked many times, but I stopped trying about seven or eight years ago,” one person admitted.
COST-EFFICIENT GROWTH
Reassuringly, when evaluation is conducted – and, more importantly, the learnings are applied – it does lead to decisions that create growth for the brand in a cost-efficient way. One person I interviewed said marketing mix analysis led to incremental spend on TV advertising, resulting in a sales increase on the previous year.
The reason for the huge variation across the business in the amount and degree of rigour with which evaluation is conducted is that there are some real, and sometimes insurmountable, barriers. Some are specific to our industry, some to our markets, but they prohibit the detailed measurement and evaluation that is possible for brands in other sectors.
For example, in some markets, it is impossible to measure sales accurately, and sometimes we just don’t know precisely how much we sell. If you go to parts of Africa or Asia, (Nigeria is the second-largest Guinness market after Great Britain), you will find either that there is, at worst, no sales audit information available or, at best, dubious coverage, particularly for the on trade (pubs, bars, restaurants and other licensed premises).
Some parts of the world are either too remote, too costly or too dangerous to audit with the frequency and rigour that we take for granted in markets where Nielsen and IRI data is commonplace.
Hence, markets take a more pragmatic approach and use softer measures of consumer attitude and behaviour change that they can measure and, over time, at least give directional information on which to aid decision-making. After all, like any research, even the most rigorous and detailed study is only a tool to make better management decisions. It doesn’t make the decision for you – you need to use your brain for that.
Lack of data not only makes ongoing monitoring difficult, but also makes measurement of individual pieces of activity challenging. It prompts us to be creative in how we can evaluate in these situations – for example, collecting bottle tops of our brand and competitors’ for an in-bar promotion and counting them up manually at the end of the night to calculate volume sold and share. It is a primitive approach, but a practical one.
While the techniques and approaches used to evaluate activities are pretty standard, there are two very important things that help Diageo with best practice in evaluation. First is the way marketing and consumer planning capability is built, and second is ensuring any tools used provide simple outputs that give direction.
The marketing capability programme, The Diageo Way of Brand Building, is an internally developed and taught marketing process used across the globe. It comprises a set of tools that are easy to understand and have practical applications for marketers.
It is not a book of theory that sits on the shelf gathering dust, but something that is ingrained in our language and way of working. In fact, new recruits sometimes struggle to do their jobs until they have been trained in the ‘Diageo Way’.
In the words of one of the original creators of the programme, ‘best practice is good practice, rigorously applied and deeply ingrained’. There may well be new and better tools and techniques out there, but having a commonly understood, easy to apply approach across the business far outweighs the incremental benefits of introducing a new technique. To quote Diageo chief marketing officer Andy Fennell: “The debate about whether your onion is better than my triangle is one I quite frankly don’t have time for.”
To build consumer planning capability, there is dedicated senior resource, whose sole responsibility is to raise functional skills in market, brand, innovation and shopper planning and planning behaviours.
KEEPING IT SIMPLE
The uniting philosophy for the ‘Diageo Way’ is to create simple outputs that give direction. The truth is that marketing peers do not need to be bamboozled by complicated research techniques and scientific explanations. They spend most of their time permanently tuned into the world’s most listened-to radio station, WIIFM – ‘What’s in it For Me to help me do my job better?’
Likewise, they don’t always want the latest and hottest techniques; they want consistency and ease of use. That is not, of course, to say we should be completely averse to change.
It is important to keep on top of developments in the industry and ringfence some funding for piloting as appropriate, but it is not appropriate to introduce new research techniques on a whim, or choose those that are only workable in one market. They have to add value to Diageo holistically.
To sum up the Diageo perspective, rigorously evaluating effectiveness and efficiency of marketing investment in markets where it is feasible to do so, is important, but an equally important goal is for marketing and consumer planning approaches that make the complicated simple and make the academic practical for market teams.
And finally, focus on input in addition to – not instead of – output, to increase the chances of success as opposed to proving the levels and degree of failure.