Cultural managers and practitioners increasingly seek to understand and measure the impact of their projects on communities. This is important for best practice operation and ensuring the responsible use of public funds, and is increasingly demanded by funding bodies and decision-makers.
The intention to plan and evaluate responsibly can be challenging, with lack of specialist expertise in evaluation common amongst those with demanding workloads. To date, there have been few specialised tools available to support planning and evaluation of cultural and community projects. When evaluation happens, it often occurs as an after-thought, with experts brought in to measure impacts of activities that have not been considered in the planning stages.
Recognising that simpler solutions are required, the Cultural Development Network (CDN) and Culture Counts have developed two new tools for planning and evaluation of arts projects, ZAPS (by CDN) and Culture Counts. The two together offer a comprehensive solution to the problems of evaluation of cultural initiatives.
ZAPS is an online tool that supports planning of projects; stimulating thinking about why the project is being undertaken, and therefore the change that is sought as a result. It encourages evidence-based work, by providing a bank of information that practitioners can draw on, as well as an iterative process by which project leaders can share findings of their evaluations with peers.
Historically, cultural organisations have counted only outputs, (the number of arts products created, event participants or tickets sold) as metrics of success. CDN has been working with the National Local Government Cultural Forum in Australia over several years to establish an agreed set of measures for inputs, outputs and outcomes of local government’s cultural activity. All outcomes fall within one of five public policy domains (cultural, social, civic, economic and environmental), and all publicly funded activities can generate outcomes across one or more of the domains.
Culture Counts complements this process by offering a technological solution to the collection and analysis of data. Culture Counts measures emotional and intellectual responses from the public, peers and organisations and turns them into big data insights, generating hard evidence to help the arts attract investment and grow audiences. Use of a common language, collection methodology and analysis tool enables data to be interrogated, controlled, aggregated and globally benchmarked.
ZAPS with Culture Counts: Project Workflow
Culture Counts metrics connect to the nationally endorsed CDN measurable outcome schema, enabling measurement across cultural, social, civic, environmental and economic domains. By using an agreed set of outcomes across these five domains, it is possible for all investments or programs to be evaluated using the same criteria, to identify where the greatest change is occurring and where best to direct limited funds to achieve maximum benefit. Outcomes data collected in Culture Counts is fed back into ZAPS to demonstrate the success of the project or identify areas for improvement, forming an evidence base of learnings for the sector.
Both tools have been developed in collaboration with cultural organisations and local government across Australia and overseas, seeking to meet the needs of the sector in practical and user-friendly ways. They are designed to be used by project managers and staff themselves, empowering designers and executors of arts projects to assess and evaluate as they go. The inclusion of outcome-focused planning and integrated evaluation into project design is expected to make significant positive contribution to project impacts. The sector overall can be advanced by tools that enable practitioners to learn from each other.
Stay tuned for a practical case study about the City of Sydney’s involvement in this collaborative initiative.
Image: Alicia Frankovich, Outside Before Beyond, 2017. Installation view Kunstverein für die Rheinlande und Westfalen, Duusseldorf 2017. Photograph: Katja Illner.