Making evaluation easy

November is measurement month #aemcmm. Measurement month is a global campaign by the International Association for Measurement and Evaluation of Communications (AEMC) that highlights the value of measurement in communications.

A few months ago I wrote about the importance of evaluation and measurement in communications and some of the barriers practitioners face in implementing it regularly. This week the focus of measurement month is on the Asia-Pacific region, so I thought I would celebrate #aemcmm by writing a bit more about this topic, looking at the principles and a framework developed by the AEMC. These tools are a great way to start doing measurement and evaluation more regularly. They have set out the high-level approach you should take and then operationalised it with an interactive framework.

I am a huge believer in not reinventing the wheel. My first boss out of uni taught me this lesson, “pilfer, pilfer, pilfer” was one of our mantras in a low budget team. It’s not about plagiarising someone else’s work but not re-treading old ground. If someone has already done great work, you can leverage off – which is why I have shared some free templates on my website.

For this reason, I wasn’t going to develop a whole new evaluation framework or tool for you when there are already great ones available, created by some experts in the field. Not everyone would have heard of the AEMC or the Barcelona Principles, so I wanted to introduce you to them (or remind you if you have seen them before).

In 2015 the AEMC released the Barcelona Declaration of Research Principles 2.0 which provides a set of principles for communications evaluation. These are the second iteration of the principles which were developed by an international group of experts.

 

There are some really interesting concepts here that at first, I struggled with, so I thought I would share my insights.

Outcomes versus outputs

Initially I thought this was basically the same thing, but actually, it’s the difference between counting the things you do and the impact they have. Outputs tell the story of what you produced, the number of visitors to your website, the number of retweets. Output measures do not address the value or impact for your organisation. On the other hand, an outcome is the behaviour changes, level of awareness, the overall performance or achievement that occurred because of the communications tactics.

Measurement versus evaluation

I often think of these as one in the same. They are sequential activities that you undertake separately but should follow one another to wrap up your strategy. Measurement is the collection of quantitative or qualitative data to determine the output of your communications tactic, number of views of a webpage, number of click-throughs on an email. Evaluation is the process of assessing the data you have collected, analysing the impact and whether it demonstrates that you have achieved the desired objectives.

Understanding the effect on organisational performance

Often objectives are focused on communication activities (processes or channels) or client outcomes. Aligning what we do to the corporate strategy and business goals will help us to demonstrate the value that we contribute to an organisation. It can be hard especially if you can see the link between the project and the broader organisational goals, or if the organisational goals aren’t clear to begin with. But like all things in communications sometimes we need to drive from the bottom up, seek out the answer to how this one-piece fits into the bigger picture and demonstrate the strategic links.

The Barcelona principles can be summed up like this, measurement is fundamental to what we do, but we have to look at it holistically and from an organisational perspective. We cannot prove our value to an organisation without measurement and evaluation.

“To prove the value of communication in an age of accountability it is vital to move beyond measuring just the content” AEMC

Professor Jim Macnamara of the University of Technology Sydney (an expert in communications evaluation) has developed an interactive framework to allow you to ‘operationalise’ the Barcelona Principles and demonstrates how to turn Principles into action. The framework can be found on the AEMC website, along with a huge amount of other resources! This is reducing one of the barriers for practitioners making it easier, less guess-work and a repeatable process.

Give it a try! Happy Measurement Month!