It’s been ten years since the International Association for the Measurement and Evaluation of Communication (AMEC) launched the Barcelona Principles – a set of guiding values to help the industry measure and evaluate the effectiveness of a communications program in a meaningful and relevant manner. At the recent AMEC Summit, the latest iteration – Barcelona 3.0 – was launched. So, how have they evolved?
In the same way that AMEC evolved from an acronym standing for the Association of Media Evaluation Companies to the more holistic Association for the Measurement and Evaluation of Communication, so too have the principles evolved to reflect broader standards of measurement. When the first iteration of the Barcelona Principles was released it was a great leap forward; the industry came together to outline a set of global standards that would help measurement practitioners adhere to best practice. The fact that it was initially interpreted as the (long overdue) death knell of the Advertising Value Equivalent (AVEs) was always slightly disappointing. It was so much more than that. There were principles of planning, the importance of measuring outcomes, the inclusion of social media, as well as highlighting issues relevant to the sector such as transparency.
Based on that, during the online discussions that followed at this year’s summit, I asked whether we still need AVEs referenced in there; the answer was a resounding yes. The extent to which the metric is used differs considerably by region and according to this year’s AMEC Global Member Survey, only 3% of practitioners say AVEs are relevant to an evaluation program. Despite it being in significant decline, it still crops up in RFPs and client requests on a disappointingly regular basis. AVEs of course, like hyperinflated impressions figures, are just proxies for success; blunt tools used in an attempt to show that comms has value. Value of course is not only measured in £/$/€. If you build your evaluation program around the Barcelona Principles 3.0, that question of value will shine through. So, what are they?
1. Setting measurable goals is an absolute prerequisite to communication planning, measurement, and evaluation.
Measurable, absolute prerequisite and planning are the key new words here. Goals that are not measurable are not goals – they’re wish lists. Defining what you want to achieve from a communications perspective, outlining what success looks like and planning how to achieve your goals will cascade into meaningful evaluation metrics. And if you don’t hit your goals, that’s not the end of the world. Evaluation should be as much about course correction and planning as it should be about demonstrating impact. If things are not going to plan, effective measurement will help you get back on track.
2. Measurement and evaluation should identify outputs, outcomes, and potential impact.
Measuring outcomes gives us a seat at the table. Output metrics are still important to track but have you changed hearts and minds? Has your audience changed behavior, bought the product or gone to the destination? If you can show that your program has achieved what it set out to, and ideally in a way that shows meaningful business impact, your internal stakeholders (and budget holders) will listen.
3. Outcomes and impact should be identified for stakeholders, society and the organization.
This has progressed from “the effect on business results can and should be measured”. The repositioning of “stakeholders, society and the organization” is more inclusive and leads to a set of measures with broader appeal. Of course, this is not simply measured through media coverage. Consider how your program can impact behavioral change, impact the sales funnel or lead to improved morale among your workforce. Consider interview and focus groups. Triangulating your analysis with other datasets available in your business will help you better understand the role comms plays within your organisation.
4. Communication measurement and evaluation should include both qualitative and quantitative analysis.
It’s important we don’t lose sight of the importance of quantitative analysis. Reach of coverage, share of voice, journalist engagement and awareness are all good measures, but they need to be overlaid against the right metrics to give maximum insight. Strong share of voice, for example, means nothing unless you understand the context and tone of the debate. Ideally, communication evaluation should not be executed as a standalone initiative. As an always on service you gain the ability to gauge trends in quantitative and qualitative performance and answer the important questions such as “what does good look like?”.
5. AVEs are not the value of communication.
6. Holistic communication measurement and evaluation includes all relevant online and offline channels.
Perhaps of all the principles, this has evolved the most. From an initial position in 2010 stating that “social media can and should be measured”, this has evolved to be far more inclusive. AMEC encourages us to go beyond vanity metrics such as “likes” and “impressions” and to focus on more qualitative metrics. We should track shifts in attitude, purchase intent and behavior change. When you’re undertaking an analysis program consider your impact more widely; include your owned social channels, track engagement on websites and track organic and paid search, paid advertising, the result of surveys and impact on sales can all be tracked.
7. Communication measurement and evaluation are rooted in integrity and transparency to drive learning and insights.
Integrity and transparency must be the foundations on which our industry is built. The key change in Barcelona 3.0 is the addition of “to drive learning and insights”. We operate in a world where although technology is ever improving, the claims of what that technology can achieve are perhaps overstated. Technology is great for processing large amounts of data but it’s the interpretation of that data that gives us insights. Having an approach that blends technology and insights and using those insights to learn and improve is at the heart of what CARMA does. I’ve always believed that while using evaluation for demonstrating impact is what gets the headlines (and the budget!), the true value lies in a cycle of continual learning and improvement.
To find out more about the Barcelona Principles 3.0 and how to put it into practice, click here.
This article was written by Jason Weekes, Commercial Director for Europe and the Americas for CARMA International, the global communications evaluation consultancy. He has spent the last 16 years helping clients prove and improve the effectiveness of their communications programs and has developed services for some of the worlds’ largest companies and political institutions.
About AMEC
AMEC is the International Association for Measurement and Evaluation of Communication, the global trade body and professional institute for agencies and practitioners who provide media evaluation and communication research. AMEC has more than 160 members in 85 countries worldwide.
