Surveys are essential tools for gathering meaningful and reliable data. Consistency plays an important role in the evaluation process, as it provides a way to analyse data on a long-term scale and make changes accordingly.
It is easy to ensure consistency during the evaluation process, especially if you apply standardised questions and metrics throughout. You can reduce bias, improve accuracy, and track year-on-year or event-by-event results if you employ these tactics.
In this blog post, we’ll explore why consistency is so crucial in creating effective surveys and collecting valuable data.
Points to Consider
Consistent data collection starts with a plan. Consider the following questions at the beginning of your project to ensure accurate and effective evaluating.
- How will you ensure reliable data collection?
- Do you have metrics, KPIs or long-term strategic goals?
- What is the scope of the project?
- How will you be analysing your data?
Ensuring Reliable Data Collection
Impartial wording reduces bias
Biased questions can influence participants’ answers, leading to skewed data that does not accurately represent the true opinions or behaviours of the respondents. Therefore, it is crucial to frame questions in a neutral and impartial manner.
Use plain, easy-to-understand language
This also plays into accessibility requirements, as Plain English is the preferable wording to ensure most respondents can understand the questions and respond accurately.
💡 TIP
Making the most of tools such as the Culture Counts Question Bank is another easy way to take the guesswork out of how to formulate these questions.
Establishing Metrics and KPIs
Develop a clear plan on what you and your stakeholders want to measure
The tracking of metrics, KPIs and the big-picture strategic plan is a great way to work out the framework of your evaluation. Having a clear idea of what you and your stakeholders are wishing to measure not only saves time but encourages consistent surveying and the extraction of meaningful data.
For example, you may be wanting to measure brand loyalty over the course of your 5-year strategic plan. Therefore you could include a standardised Net Promoter Score (NPS) question in each survey, and chart the results over time. Another example could be the measurement of which demographics (age, gender, postcode) attended each event over the course of your arts season.
💡 TIP
It is easy to get caught up in the endless possibilities of surveying audiences, but having a core set of questions is highly recommended! The time you take to solidify your goals and work out a template of questions will save you disappointment when it comes to reporting results that are a mixture of information rather than actionable data.
Determining Evaluation Timeline and Scope
Use spreadsheets like our Evaluation Plan to help keep track of key dates
Working out a timeline for your upcoming evaluations is a helpful way to determine the scope of your project. Simply put, accountability equals consistency.
Ensure inter-department communication
Organisations with multiple departments can fall victim to this as there can be a lack of communication between departments. The knock-on effects of this miscommunication can lead to the creation of unique surveys for each department and a disregard of the agreed-upon core questions. Creating a simple spreadsheet that outlines the event, survey and data collection dates are a simple way to mitigate this.
Undertaking Data Analysis
Benchmark against your own results and other datasets
Some of the benefits of consistent data collection include drawing accurate conclusions from data, benchmarking and ‘closing the feedback loop’.
Sample sizes are an integral part of determining the accuracy of data, and the more surveys that use consistent questions, the greater the sample from which insights can be drawn from. The margin of error is greatly decreased when questions are asked consistently, which will benefit you in the long run.
💡 TIP
We find that sample sizes of 100-150 can be sufficient for data analysis for events of considerable size and recommend a minimum of 30 responses. You can also use tools like the ABS Sample Size Calculator.
Furthermore, benchmarking is becoming an increasingly useful resource amongst organisations. Consistency in data collection allows for your organisation to not only benchmark results against yourself, but also larger data sets that may help visualise your impact to stakeholders.
💡 TIP
Culture Counts has the ability to benchmark your results against our big dataset of Arts & Culture, Library or Local Government surveys. Funders love to see a benchmark!
Complete the feedback loop
Closing the feedback loop refers to the process of taking action and providing follow-up communication based on the feedback received. It involves acknowledging, analysing, and acting upon the feedback to address concerns and implementing improvements. By completing this loop, you can demonstrate their commitment to actively listening to your audience and continuously improving your products, services, or processes based on the valuable insights gained from feedback. This iterative process fosters better relationships, builds trust, and enhances overall performance and customer satisfaction. The effectiveness of this process can be assessed through the collection of consistent data over a period of time.
Summary
To summarise, consistency is important as it demonstrates an organisation’s ability to understand the ways in which meaningful and reliable data is collected. Although it can be exciting to want to collect as much data as possible, taking time to strategise a core set of metrics and questions proves invaluable in the long term. You will be able to benchmark your data, compare and contrast over periods of time, and effectively implement any feedback you receive.
