Jessie Nicholls manages all communications and marketing activity for Project Oracle: London’s Children and Youth Evidence Hub, which is funded by the ESRC. She also works part-time as the Communications Manager for The Social Innovation Partnership (TSIP).
Can we measure the impact of art?
Many feel that evaluation methods are inappropriate and even directly opposed to the values of art. Most people have felt the effect of a work of art or a play, either intellectually or emotionally. Artistic value is intrinsic and is associated with ideas of aesthetic excellence and individual experience. Efforts to force the qualitative nature of art into the quantitative measurement of other kinds of outcomes, undermine and even threaten this intrinsic value.
As an organisation dedicated to growing and supporting good evidence within the youth sector, it will come as no surprise that we at Project Oracle feel differently about this.
Over the past year, we have repeatedly asked ourselves: what is the best way to measure the impact of an arts-based project like this? And one like this? How can we empower youth organisations to take evidence into their own hands, so that evaluation can be carried out in a sensitive and authentic way? In a way that enables them to show how their intervention gave a young person confidence, or opened them up to a new way of expressing themselves or helped them get through a difficult time at home?
We don’t want youth organisations to feel intimated by these questions; we want them to feel engaged. For it is entirely possible to demonstrate the impact of an arts-based project. Indeed, a failure to do so will put the cultural sector in a weaker position compared to others when budgets are negotiated.
We decided to support a cohort of arts organisations working with children and young people over a period of 12 months, with the aim to: 1) prove that it is possible to measure the value of an arts intervention, without sacrificing integrity; 2) co-develop evaluation methods better suited to this endeavour; and 3) share these twin messages and help boost the evidence fortunes of the wider youth arts sector.
As we come to the end of that year, the lessons to emerge from this experience are shared in our learning report, Impact Pioneers: Lessons in arts evaluation. As individual organisations, cohort members are better equipped to carry out evaluation and to approach funders with a method that works for them. And, as a group, they are in a better position to share with the wider sector a genuine perspective on what works well in arts evaluation.
Here are a few snippets from that report.
Using outcomes frameworks
‘We see benefits to selecting a small number “anchor” outcomes that work across multiple age groups and art forms. By focusing on a select few, it is possible to build a bank of high-quality data that captures the effectiveness of the arts, raising its status as a meaningful route in the pursuit of those desired outcomes … Candidates for these limited spots include improved well-being and engagement with school and statutory services, for good evidence already exists in support of them.’
Creating high-quality tools
‘Is it possible, we asked, to create evaluation tools that provide robust and rigorous data, while also empowering staff and engaging young people? Three months of consultation, conversation and planning led to three key ideas about how to address this challenge; namely, through observation, a focus on behavioural indicators, and the creative use of validated surveys.’
Engaging funders in evaluation
‘A two-way dialogue between funders and youth organisations can help to reveal and establish what good evidence looks like from different perspectives. It can also serve to inform funders about what is realistic and useful on the ground, giving them opportunity to reflect on what they are asking for.’
Sustaining peer-to-peer learning
‘While what we call “infrastructure” is most usually associated with top-down dissemination, we recommend that the same is needed to support a bottom-up, peer-led approach, too. This is perhaps particularly true for the arts, where organisations take a more creative view and pursue softer outcomes.’
The arts all too often lose out when asked to define their impact. Challenging as it is, the evaluation of arts-based programmes is much too important a task not to give it our best shot.