Great - so you've logged into Culture Counts, but you don't know where to start. If you're having trouble getting set up, the article below will take you through the steps required to create a new evaluation and make your first survey!
When you log in to your Culture Counts account, you will see that we have provided an example evaluation based on a set of standard metrics that have already been tested internationally. As an easy starting point, you can copy then edit individual surveys from this evaluation, or you can create your own evaluation from scratch. We recommend first time users follow the example evaluation, but you can try both and see what suits you best!
In this article:
What is an evaluation?
Evaluations are folders or containers that contain all the information for a particular event, project, or place that you wish to evaluate. If you wish to create several related surveys to evaluate an event, they can all be grouped and organised within a single evaluation. The evaluation will store and present all the data collected through those surveys so that you can quickly see what an event is achieving overall. In time, you will also be able to import event data such as attendance numbers and box office revenues into the evaluation from other sources.
Creating an evaluation by copying the Example Evaluation
About the Example Evaluation
We have created an Example Evaluation to give you some insight into how we design our evaluations.
There are three types of surveys that can be created within an evaluation: a ‘standard’ survey, a ‘prior’ survey and a ‘post’ survey.
A typical Culture Counts evaluation contains three standard surveys:
- a standard survey for members of the public to rate their experience of the event, based on your chosen set of dimensions and any custom questions
- a prior survey, which asks peer and self assessors to describe their expectations of the event. For peer assessors this will be their expected personal experience; for self assessors this should be the expected audience experience of the event. Afterwards you can compare expectations with how the event was actually perceived
- a post survey asks respondents about their actual experience of the event. When linked to a prior survey, it allows you to measure the difference between expectations and reality. This can also help gauge if your objectives were met by comparing your self assessment prior and post survey results
For more information on standard, prior and post survey types, and which survey types to use for different assessors, see ‘Configuring your survey’.
Our Example Evaluation contains examples of a standard public survey, one prior survey and one post survey (self and peer). They are based on a set of tested standardised metrics that were developed in consultation with the cultural sector. We suggest you start by browsing through the surveys in the Example Evaluation to see how they have been set up.
Please note that the Example Evaluation is a shared resource that is available to all Culture Counts users. We therefore ask you not to make or save any changes to the surveys within the Example Evaluation itself, as any changes you make will also show up in the Example Evaluation in the accounts of other users! Please copy the surveys first before you start editing them.
Copying the Example Evaluation
To copy the Example Evaluation, first create a new evaluation for your event. This will be a new container where the copied surveys can be stored. To start a new evaluation, click the New evaluation button at the bottom of the Evaluations panel. You will be prompted to give your evaluation a name (normally the name of the event you are evaluating) before being taken to the evaluation overview screen.
To copy the Example Evaluation surveys into your own new evaluation, open the Example Evaluation so you can see the three example surveys (public, prior and post) described above. Copy the example public survey by clicking Edit to the right of the survey name and selecting Copy from the dropdown list. Replace the copied survey name with a new name that reflects your event and choose your new evaluation to copy the survey into.
We recommend giving each survey a simple name that both people in your organisation and assessors will understand, like the name of the event and the survey type: ‘My Awesome Event – Public Survey’ or ‘My Awesome Event – Prior Survey’.
You will need to repeat the process above to copy each of the three example surveys into your new evaluation. From your new evaluation, you can edit these surveys however you wish. Click the name of the survey that you wish to edit, and you will be taken to the survey Summary page, which shows a brief summary of the survey settings and questions included. From here you can navigate to different pages in the survey builder by clicking the Manage button in the top right corner, or clicking on one of the tabs in the top menu bar (dark grey). The editable pages include
- Configure – to change the survey settings
- Design – to edit, add or remove survey questions
- Invite – to add or remove self and peer assessors
Creating an evaluation from scratch
To start a new evaluation, click the New Evaluation button at the bottom of the evaluations panel. You will be prompted to name your evaluation (normally the name of the event or place you are evaluating), before being taken to the evaluation overview page. From here you can create new surveys, manage existing surveys and download any data that has been collected for the evaluation.
To create a survey for your new evaluation, click Create survey. You will then be taken through the three main stages of survey creation:
- Configure – to change the survey settings
- Design – to edit, add or remove survey questions
- Invite – to add or remove self and peer assessor
Configuring your survey
The Configure page is where you set up the following options for your survey:
This is the survey name that respondents will see when they access the survey.
Start and close dates
The start and close dates will determine when the survey begins and stops taking entries. It is fine to leave the start and close dates blank. If you leave the start date blank, the survey will be available to record responses immediately, and if you leave the close date blank the survey will remain active indefinitely. When inputting the start and close date, we advise that you include an additional one or two days either side of the official start and close dates of your event to ensure that you capture all available survey responses.
Here you can write a short optional introduction to your survey that will appear on the front splash page of your survey. This introductory front page will only appear on the Online and Display survey delivery methods, as interviewers delivering the survey by iPad can verbally provide this information to audiences. This introduction is good place to provide a brief explanation to respondents, such as: “We’d love to hear what you thought! Please share your views on the quality of this event via our short survey. Your important feedback will help us to understand and measure the impact of our events.”
Here you can upload your own logo, which will appear on the front splash page with the survey introduction text.
There are three survey types – Standard, Prior Event and Post Event. In an evaluation where you wish to assess the expectations of peer and self assessors, you will create one of each survey type. In some cases, a prior survey isn’t possible or appropriate, so all respondents (including peer and self assessors) can take the standard survey after the event.
- Standard survey: The standard survey is for your general public audiences or visitors. Audiences are typically asked to complete a standard survey after experiencing an event or visiting a place, to measure the perceived quality and impact of their experience on them.
- Prior survey: Prior surveys allow you to record the expected performance of an event, to later compare with how the event was actually perceived. We typically recommend that prior surveys are only completed by peer and self assessors. It is important for the organisation to complete a prior self assessment to set up their objectives and creative intentions for the event.
- Post survey: Post surveys are also completed by peer and self assessors, to evaluate actual performance against expected performance. Post event data from peer and self assessors can be compared with public data from the standard survey.
When creating prior and post surveys within an evaluation, you must link them together so that Culture Counts knows which data to compare. If you create the prior survey first, you can complete this step when creating your post survey. You will see a ‘Survey Link’ box to the right of your survey type input on the Configure page. Select the correct prior survey from the Survey link dropdown list. This will then sync your prior and post surveys so that all peer and self emails that have been input into the prior survey are copied across to the post survey. It also auto fills prior survey questions into the linked post survey for you. All you’ll need to do to complete your post survey is jump in to the Design page and change the tense of the dimension statements from future to past from the drop down options provided.
This section allows you to choose how the survey will be delivered. Selecting one or more delivery options will automatically configure additional survey settings to correspond with your chosen survey delivery methods. You will be provided with further instructions on how to deliver the survey when you’ve completed the set up process. Culture Counts currently provides the following delivery types:
- Online Survey: to be taken by respondents online via email, social media or on their own smart phones or devices. Online surveys are non-resettable so they can only be taken once by respondents.
- Interviewer: to be facilitated by an interviewer with a tablet computer or iPad. Interview surveys are resettable so that multiple responses can be recorded on the one device.
- Display: to be displayed on a device at a set location, such as a fixed podium or a library computer.
Select all of the methods that you wish to use to deliver your survey. You can use all three methods. A unique survey link will be created for each method, and will be displayed clearly on the Summary page at the end of the survey builder.
You will be presented with advanced options related to the delivery type(s) you have chosen, including a finish URL (which will redirect respondents to a specific website on completion), a timeout (which will reset the survey if the respondent is inactive for a period of time) and access to survey tools (which will enable interviewers to access tools to support the interview process, including monitoring the number of survey responses captured).
Designing your survey
Designing a survey is the process of adding questions and content to your survey. On the Design screen you can add, modify or remove questions.
Adding survey content
In order to add content, simply click the type of question or content and it will appear in the survey section.
We recommend always using four to six of our standard dimensions, to enable benchmarking and ensure your survey remains speedy for audiences to complete. These should be chosen based on the objectives of your evaluation, but feel free to use whichever mix of questions suit your needs. If you want to learn more about best practice for making surveys in Culture Counts, visit our resources section for more information.
The types of content available in Culture Counts are outlined below:
A standardised statement that respondents can agree or disagree with, rated along a slider. Culture Counts dimensions are based on extensive research in each sector in order to identify the key elements that make up the quality of an event, project or place. These elements have been translated into single statements or ‘dimensions’ to be measured using Culture Counts surveys.
One of the main advantages of using these preset dimensions to evaluate the quality of your event is the ability to standardise measurements across the sector. This allows you to measure your performance over time, or against other organisations nationally and internationally.
There is a special selector for adding dimensions to the survey. When you click to add a dimension, you will be presented with a list of dimension categories relevant to your organisation or survey type. Some of these include Quality, Reach and Place. To add a dimension question, simply click the category you wish to use, select the one you want to add, then click "Use selected dimension". After the dimension has been added to the survey, click the dropdown next to ‘Question text’ and select variants of that dimension statement that best suit your survey. This is where you can change the tense of the statement.
A simple slider input, where responses are recorded along a slider that ranges between three values. By default, these are "Strongly disagree", "Neutral", and "Strongly agree", but can be customised when the question is added.
Provides a date input where respondents can nominate a date from a calendar.
Gives the respondent two options to choose from, which default to "Yes" and "No".
A basic number input, where numbers can be input by either typing, or changed by up and down arrows. Minimum and maximum values can be specified.
Provides a dropdown list, where respondents can answer by choosing one of the options.
Provides a selection of answers, of which respondents can answer by selecting one or more of the options.
A short text input designed for smaller input types, such as such as one word answers (up to 12 characters).
A simple text input that allows respondents to type an answer or comment.
This displays a message within the survey but does not require respondents to provide an answer. It is typically used to provide respondents with additional instructions or context.
This content option provides a section to ask for respondents' email contact information. It is a simple text input designed to recognise if the text entered is an email address.
Editing or removing content
In order to edit content, click on the question text you wish to edit and type your desired text. Some content types, like dimensions, cannot be edited as they are standardised. To remove a question, click on the "trash" icon in the top right of the question box. The icon with the four squares allows you to change the ordering of questions within your survey by dragging to the correct position.
Please note that standard demographic questions – age, gender and postcode – are included automatically at the end of all Culture Counts surveys. You do not need to add these questions to the survey yourself.
Inviting peer and self assessors
One of the core functions of Culture Counts is the ability to invite and manage peer or self assessors to your surveys. This can be done easily through our invites page. Simply type in the email of the peer or self assessor you wish to invite in the email field, and press enter to add them.
Completing and distributing your survey
Once you have been through the Configure, Design and Invite stages, you have created your survey and will be taken to the survey Summary page. Here you will be shown an overview of the survey, with the number of questions you have included, a list of the self and peer assessors you have added and a summary of the survey results so far. You can return to this page at any time to see how your survey is progressing, and which of your nominated respondents have attended or completed their surveys.
Distributing the survey to the public
The Summary page will contain the links (URLs) for your survey. There will be a separate URL for each of the different delivery types that you selected at configuration stage. It is important to use the correct one for each method. To distribute a public survey, simply copy these links and use them in the relevant locations. For example, for a survey to be administered by interviews, enter the Interview link into the browser of your iPads or tablet computers. For a survey to be distributed via email, copy the Online link and paste it into the email you are sending to audience members/visitors.
Before distributing the survey, we recommend you click on Preview to the right of the survey link so that you can make sure the survey looks just as you intended it to! The preview link will open the survey in a new tab for reviewing, but will not record any data.
Distributing the survey to peers and self respondents
The easiest way to distribute a survey to self and peer respondents is to send email invitations to your nominated assessors via the Culture Counts platform. If you scroll to the bottom of the survey Summary page, you will see that the system automatically generates a unique link to the survey for each of your nominated self and peer assessors. Each self and peer assessor must be sent his or her own unique link, and it’s easy to do this via the platform. On the Summary page, click the Send invitation button next to your first self or peer assessor. A dialogue box will open to allow you to draft your invitation email. The unique survey URL for that particular respondent will automatically be included in the email.
Note that you can communicate with your self and peer assessors outside the Culture Counts system – for example through your own email account – if you prefer. However, to ensure that responses are recorded against the correct user category and the data shows up correctly in the reporting pages, peer and self assessors must take the survey via the unique URL that the system generates for them. At the Invite stage of the survey builder, enter the email addresses of each of the self and peer assessors that you are inviting to take part, and make sure that you send each assessor the unique URL that will be generated next to their name in the Summary page.
If you are asking self and/or peer assessors to complete both a prior and post survey, you need to invite each assessor to complete the prior survey first. Within each prior survey, Culture Counts automatically asks respondents when they are planning to attend the event. This date will show up in the survey Summary page, along with notification that the peer or self assessor has completed their survey. You will know to send self and peer assessors their links to the post-event survey once they have completed the prior survey and attended the event.
Seeing the results
Once you’ve completed the evaluation and collected the public, peer and self responses, you are able to see your results. Through Culture Counts, you can do this in a number of ways:
Real time graphs
CSV raw data
You can find more information about the ways you can see and present results in our support section as well as detail about navigating the reporting dashboard, and what each section of the dashboard does.