The Department of Local Government, Sport and Cultural Industries (DLGSC) used Culture Counts to survey people attending the Revealed Art Market in April 2019. As part of the evaluation, surveys were captured from 121 members of the public, 5 peer assessors and 7 self assessors.
Each survey contained nine ‘dimension’ questions, asking the respondent about their experience of the event. These artistic quality dimensions have been developed with the arts sector to measure the impact and value of arts and cultural events.
DLGSC engaged Culture Counts to conduct an evaluation of the event. Public, peer and self Surveys were delivered in partnership with Culture Counts. The survey was designed and developed to evaluate the strategic goals and outcomes of the Revealed Art Market.
Each survey contained a range of ‘dimension’ questions, asking members of the public about their experience of the event. These dimensions have been developed and tested in collaboration with industry, practitioners and academics to measure the impact and value of arts and cultural events and activity.
Appropriate dimensions were chosen based on their alignment with DLGSC strategic objectives.
| Domain | Dimension | Dimension Statement |
|---|---|---|
| Cultural | Authenticity | It had a connection to the State/Country we live in |
| Inquisitiveness | It encouraged me to find out more about the art | |
| Meaning | It moved and inspired me | |
| Economic | Collaboration | It connected other artists |
| Social | Connection | It helped me to feel connected to my local community |
| Quality | Distinctiveness | It was different from things I've experienced before |
| Excellence (national) | It is amongst the best of its type in Australia | |
| Local Impact | It's important that it's happening here | |
| Rigour | It was well thought through and put together |
Dimensions are assessed on a Likert scale, in which respondents move a slider to a point that indicates whether they agree or disagree with the dimension statement. An example of a dimension question in the Culture Counts survey tool is presented below.
Survey respondents were asked to provide their age, gender and postcode at the start of the survey. This enables data to be matched to the wider population and responses to be filtered to understand differences in demographics. The charts show the proportion of survey responses captured for each of the age and gender demographics, and the ten postcodes in which the largest number of respondents reside.
More than half of respondents were female (64%) while 35% were male. A smaller percentage of respondents described their gender in another way (1%).
The age group with the largest representation was from people aged between 30-39 years (25%). There was also a large number of respondents aged 40-49 (22%) and 50-59 (21%). The lowest percentage was obtained by respondents under 20 years of age (8%).
The majority of respondents live in the Perth metropolitan area. The most cited suburbs were Fremantle and Bibra Lake (both making up 8% of the total sample).
Respondents were asked whether 2019 was their first time attending the Revealed Art Market. This question can inform expectations and help organisers to better understand the background and prior knowledge levels of their attendees.
The chart shows the percentage of respondents that answered yes or no to the question ‘Is this your first Revealed?’
More than half of respondents were attending Revealed for the first time (59%), demonstrating the event's success in reaching a new market as well as attracting a repeat audience.
Survey respondents moved a slider to indicate whether they agreed or disagreed with the included dimension statements in relation to the event. The following charts contain the response data for 'public' responses, showing the average result for each dimension.
The Culture Counts platform provides various methods to capture survey responses at minimal marginal cost. Achieving larger samples enables organisations to be more confident about the average results and that opinions of the survey respondents are representative of all attendees. The accompanying margin of error chart shows the expected differences for the associated dimension results calculated at a 95% confidence level.
At a 95% confidence interval, the margin of error for dimensions ranged from 2.7% to 4.0%. This means that we can be 95% confident that if we surveyed the entire visitor population the average score for Local Impact would fall within 2.7% of the average generated by the sample.
Margins of error under 5% are considered reasonable representations of the opinion of the population. As the margin of error is below 5%, this sample can be deemed statistically significant and an accurate representation of the audience attending Revealed Art Market 2019.
Culture Counts uses a slider input to measure responses for dimension statements as part of the evaluation methodology. This method also provides the capability to understand response results within a typical 5-point agree-to-disagree format.
The following chart contain the response data for 'public' responses and shows the percentage of people that agreed or disagreed with each of the statements, using a 5-point ordinal Likert scale.
A large majority of people agreed or strongly agreed with each of the nine dimensions. Average scores of 81% or above were received for six of the nine dimensions, with over 97% of people agreeing that it's important it's happening here (Local Impact), that it had a connection to the State/Country we live in (Authenticity), that it was well thought through and put together (Rigour) and that it encouraged them to find out more about the art (Inquisitiveness).
The lower average scores were for Excellence (83%) and Distinctiveness (81%). While still an extremely positive result, it indicates that respondents were less likely to agree that the event was amongst the best of its type in Australia and that it was different from things they've experienced before.
As part of the evaluation methodology, staff and sector peers were asked to conduct a 'Prior' survey, before the event commenced. The purpose of the Prior survey is to record the performance expectation of both the staff who are responsible for delivering the event, as well as of sector peers, who are aware of the event and expect to experience it.
After the event, respondents are also asked to conduct a post-event survey. With this survey, prior expectations can be compared to achieved results. When also compared to public survey responses, this information provides organisers with an effective tool to triangulate results, investigate differences and reflect on theirs achievements and learnings.
The follwoing chart compares the results given by self assessors, peer assessors and public attendees before and after the event.
Self, peer and public scores all followed a similar pattern. Public respondents gave the lowest average scores for eight of the nine dimensions, and the highest score for Local Impact and Authenticity.
Peer assessors gave the highest scores for Local Impact, Collaboration and Authenticity.
Self assessors were most favourable to the Authenticity, meaning and Collaboration dimensions.
Respondents were asked whether they would recommend Revealed Arts Market to a friend or colleague. Respondents could choose a number from 0 to 10 from a menu, with 0 meaning 'not likely at all' and 10 meaning 'extremely likely'.
These results can be used to calculate a Net Promoter Score (NPS). NPS is a standardised metric that seeks to measure loyalty between an organisation and its audience. Respondents with a score of 9 or 10 are considered 'Promoters'. 'Detractors' are those who respond with a score of 0 to 6. Scores of 7 and 8 are considered 'Passives'.
NPS is calculated by subtracting the percentage of respondents who are Detractors from the percentage of customers who are Promoters. This means that an overall Net Promoter Score can range between -100 to +100.
This chart shows the proportion of respondents that would or would not recommend Revealed Arts Market, followed by the calculated NPS below.
76% of respondents chose a score of 9 or 10, meaning they're highly likely to promote the festival to others. 8% recorded a score of 6 or below, indicating they would be unlikely to recommend the festival.
An NPS that is positive (i.e. higher than zero) is felt to be good, and an NPS of +50 is excellent. Revealed Art Market with a score of 70 indicates that audiences have a high level of loyalty towards the event and are much more likely to promote than detract from it when speaking with others.
Dimensions used in the evaluation were also measured in previous years. This consistent use of measurement allows organisers to benchmark their results and track progress in achieving strategic outcomes over time.
The following chart compares the results for 2019 with previous years.
Revealed Art Market dimension scores have trended similarly over the last five years. Since 2015, average scores have improved for Connection, Distinctiveness and Excellence. This indicates the event has been growing in popularity and credibility over the years.
In 2019, scores received were similar to those received following the 2018 event, with very minor variation seen across all nine dimension areas.
Respondents were asked to provide their age and gender as part of the survey. This data enables results to be filtered to understand any differences that exist because of demography.
The following charts highlight a selection of results based on the gender that respondents identified with as well as by age cohort.
Only 1% respondent identified that they were of an alternative gender, meaning that the average score is not statistically significant. As a result, this score has not been included in the gender comparison.
Female gave consistently higher scores for all dimensions, with the largest difference seen for Excellence (+14%).
Respondents Under 20 years of age gave the lowest score for six out of nine dimensions. Other age cohorts gave similar scores across the sample.
Respondents were asked to give one word that best described how they felt about Revealed. Analysis identified the top words provided by the public.
The top 10 words provided by there public were: Connected, Inspiring, Amazing, Overwhelemed, Awesome, Inspired, Fantastic, Excited, Amazed.
Respondents were asked to leave any additional feedback about their experience at Revealed Arts Market. This feedback has been classified into positive, neutral or negative categories, with the percentage of feedback sentiment types presented in the following chart.
Feedback helps organisations understand where the value of visitor experiences lies and how they can be improved and strengthened in the future. A selection of comments have been highlighted underneath.
Massive improvement from last year.
It has really encouraged me to find out more about different parts of Australia. It was so nice to see so many different people here.
Great event. It would be great to see it grow.
we really love coming to the city to see the lights and hope they continue. thank you
Excellent event with an awesome positive vibe. May there be more opportunities for such experiences.
Absolutely loved the children's art area. Really enjoyed learning a new craft...weaving and making....from an indigenous artist from Warakurna. Great for the kids to play and explore and learn.
Enjoyed seeing the younger emerging indigenous artists.
This is possibly one of the best opportunities in the country to meet artists and learn culture. There could be more opportunities for the public to interact.
Put it over two days so it's not so busy.
This report has been prepared by Culture Counts. The authors would like to thank all stakeholders and staff for their participation in this research.
Approved by: Culture Counts.
Date of Approval: 7 June 2019