Public diplomacy practitioners need to examine and analyse their effectiveness to sharpen, direct and enhance messaging and programmes. However, assessing the impact of public diplomacy has always been a challenge for practitioners. How does one measure the networks and influence that public diplomacy creates when social relationships are inherently complex, taking many forms across years?
We were delighted to be invited by Singapore International Foundation to be part of this important conversation at Public Diplomacy in Asia 2021.
These topics formed part of the conversation at ‘Public Diplomacy in Asia 2021’, organised by Singapore International Foundation, where CARMA’s Co-Managing Partner, Richard Bagnall, shared his tips and insights on Measuring and Evaluating the Effectiveness of Public Diplomacy. He joined a stellar panel of experts – Bec Allen of the Australian Department of Foreign Affairs and Trade (DFAT), Dr Kadir Jun Ayhan of Ewha Womans University, Carrie K. Lee of U.S. Embassy Singapore, Gary Mundy of British Council, and Jonathan McClory of Sanctuary Counsel.
Here are four things to keep in mind when measuring the impact of Public Diplomacy Programmes:
If you can’t measure it, you can’t manage it.
As Richard puts it, “It’s not a nice to have. It’s an integral part of comms”. The crucial need to have measurement and evaluation at the heart of all communications and public diplomacy programmes was one of the key messages all presenters stressed in unison as they shared experiences in their fields of work.
Competition for influence has never been fiercer. Despite the restrictions of the pandemic, the business of public diplomacy continues. Bec Allen, Acting Director, Public Diplomacy Programs at the Australian DFAT, commented that public diplomacy practitioners are under pressure to find new ways of operating and engaging, and being able to prove the impact of their programmes provides a strong evidence when advocating for sustainment or expansion of investment.
The panel called for practitioners to be proactive in anticipating their needs to effectively implement a measurement and evaluation programme – this could mean investing in specific tools and building the necessary skillsets within the team.
“For all that we do, as public diplomacy practitioners, we can only benefit from a more systematic and rigorous analysis and measurement of our impact” Bec said.
So how can public diplomacy be assessed effectively?
As a fundamental, Richard Bagnall, also the Chairman of AMEC, asserted that practitioners need to move away from simply counting outputs (content or coverage generated) and start linking them to the out-takes (audience understanding and opinions) and outcomes (actions or results). With that in mind, the Integrated Evaluation Framework by AMEC, which has guided the UK Government Communication Service, can serve as the best practice approach that links objectives, setting of targets through to implementation and assessment of the impact achieved – a full circle in demonstrating the effectiveness of a public diplomacy programme.
Watch the 10-minute recording of Richard’s presentation:
Dr Kadir, Professor of International Relations, Graduate School of International Studies at Ewha Womans University, reinforced that the success of a programme has to be referenced upon the objectives and expected outcomes it was set out to achieve at the start. Speaking about International Student Mobility Programmes as an example, the expected outcomes of one which trains diplomats and government officials in a host country is to influence policy decisions in the student sending countries. Albeit a longer-term evaluation, measuring the success involves tracing the process of policy transformation and adaptation. Where do ideas of policy changes come from? Have the exchanges helped achieve certain desired policy outcomes in student sending countries?
Gary Mundy revealed that British Council is building a more evaluative culture with the principal aim of driving learning around what is and isn’t working, and why. Drawing heavily in process tracing and outcomes harvesting, the organisation uses this approach to test and identify plausible course of pathways, barriers and levels of engagements to derive at a unique framework in approaching their public diplomacy programmes.
Consider longitudinal study
When dealing with public diplomacy programmes and in fact, most communications programs, results do not happen overnight. It may be years and decades, with patience and dedication, before we see impact, evidently in International Student Mobility Programmes shared by Dr Kadir. Investing in longitudinal studies to evaluate long-term impact is both challenge and necessity.
On this, Carrie K. Lee, Public Affairs Officer at the U.S. Embassy Singapore added that their proactive engagement with alumni of their exchange programmes does not end. They continue to engage them in embassy activities, follow their career development – all these help to shape their future educational exchange programme.
There is no one-fits-all.
As with the consensus of success, there is no one-fits-all when it comes to measurement. Public diplomacy practitioners need to invest in themselves and their critical thinking to contextualise their measurement and evaluation programme.
Trust in COVID-19 Vaccines report
On-demand Webinar: Vax Unpacked: PR and Comms on the Frontline
Full recording of the session is available here.
