Use your keyboard arrow keys to navigate

Contents

Photo: Unsplash - Ciprian Boiciuc

Project Highlights

Photo: Unsplash - Lily Li

Project Overview

Culture Counts, in conjunction with Public Libraries Australia (PLA) and more than 30 participating libraries across Australia, has completed the third year of a nation-wide data collection initiative to better understand and illustrate the vital outcomes generated by libraries. This document provides a snapshot of the Public Library Evaluation Network (PLEN) results from 2021-22, along with year on year comparisons from the first two years of the project.

Aim

Libraries are an extremely important asset to local communities, but membership numbers and lending statistics only address a small part of the value of a library to its community. Measuring the role of libraries and the value that communities place on them gives libraries the ability to engage more deeply with the public, impact the community more strongly, and provide an evidence-based argument for greater funding. The use of standardised measures means that libraries can learn from direct comparisons with their peers, or from wider sector trends and insights. The project gives libraries the opportunity to test outcome metrics and evaluation methods and provide feedback to ensure appropriateness from an Australian library perspective.

Network Methodology

Participating libraries subscribe to the Culture Counts survey platform; accessing standardised survey templates, unlimited evaluations, in-built metrics, client support and training. Each library surveys public library users at least once annually on outcome measures and usage, and provides the project with annual data related to library expenditure, collections, loans and visitation. All results collected through the project contribute to a large body of evidence about the value of the sector as a whole, with participants able to access a dynamic, filterable big data dashboard. Quarterly webinars enable participants to connect, interrogate the big data dashboard, and discuss implications and learnings.

Evaluation Methodology

Libraries involved in the network collect feedback on library use, performance, personal and community outcomes from public users of the library. This feedback is collected through a short standardised online survey (~4 minutes) that the library can email to its member database, link to on internal computers, deliver to library users via an iPad or tablet device, or encourage users to complete on their own smart phones.

Library staff (self-assessors) and external peer assessors (such as council members or library staff from another library) are also able to reflect on the library service using the same standard outcome measures. This can help the library to identify where its views and expectations are aligned or different to those of the community.

A representative of the library also completes an additional survey for the annual period containing library lending, staff and expenditure statistics.

Photo: Freepik

Library Outcomes

A key part of the project involves asking community members around Australia if they agree or disagree with a statement relating to their experience of the library service, across 11 key outcome areas. Outcomes assessment provides evidence of the impact that library services have on people’s lives and the value that this adds to personal, community and economic wellbeing. The outcome areas align with the Outcome Measures for Australian Public Libraries research (ALIA, APLA, NSLA), ILFA generic learning and social outcomes, Project Outcome, and other community indicator research in Australia and overseas.

The use of standardised measures means that libraries can learn from direct comparisons with their peers, or from wider sector trends and insights. The results help the library to learn where they are generating the greatest impact, where they can improve, and whether certain outcome areas are more relevant or impactful to specific demographics or community groups.

The first chart below shows the percentage of respondents across the whole Public Libraries Evaluation Network who agreed with the associated statement regarding their library experience in 2021-22. The second chart includes the percentage of respondents who strongly disagree, disagree, or are neutral about the outcome statement. The third chart shows the percentage of respondents who agreed with the statement in each year of the project, identifying where the network libraries have seen improvement and how library outcomes have changed over time.

Loading...
Loading...

Year on year

Loading...

Insights

The research shows that communities overwhelmingly agree that libraries play an important role and are felt to be safe and welcoming places. Libraries are also succeeding strongly in their more traditional literacy and learning roles, encouraging users to read more and enjoy learning. Over two thirds of respondents also agreed that their library is changing and becoming more relevant to them and their needs, indicating that greater outcomes may follow in newer library focus areas, such as digital inclusion and enterprise.

Year on year comparisons indicate that libraries participating in the network have increased their impact from 2020-21 across two library dimensions and recorded the same levels of agreement for two dimensions. Seven dimensions have seen lower levels of agreement in 2021-22 following a spike in 2020-21, however only four dimensions have decreased overall since the project started in 2019-20.

A small selection of insights derived from filtering the dynamic dashboard by age, identity, status and location include:

  • Respondents of Aboriginal or Torres Strait islander ancestry recorded higher than average scores for Digital Connection, Relevance and Creativity outcome areas

  • Respondents who speak a language other than English recorded equal or higher than average outcomes for all dimensions except Information and Digital Connection. They also scored highly for event and program dimensions related to Learning, Belonging and Community outcome areas

  • Respondents aged under 20 are less likely to find the role of the library important and relevant to their lives, or to use the library to digitally connect with others

  • Respondents aged 30-49 find Time with Children the most important outcome area

  • Respondents aged over 80 are most likely to find that the library meets their informational and reading needs. They also feel that the library encourages them to read more, plays an important role and is a safe and trusted space, but have lower outcomes across other areas

  • There were no significant differences in outcomes experienced by male and female respondents, however respondents identifying as an alternative gender recorded lower than average outcome scores for all dimensions except Literacy

  • Non-members are less likely to feel that the library plays an important role or encourages them to read more, while more likely to use the library to help with the success of their business (Enterprise) or to digitally connect with others

  • Respondents who had not used the library for more than six months recorded slightly lower than average outcomes scores in all areas except Role, Time with Children and Connection

Photo: Unsplash - Yoonji Kim

ALIA/APLA Outcomes Alignment

To ensure that all data collected can be clearly reported against the achievement of important sector objectives, Culture Counts aligned outcome metrics used in the project with Australian Library and Information Association (ALIA) and Australian Public Library Alliance (APLA) outcome areas, and the UK Inspiring Learning for All (ILFA) social and learning outcomes. This enables the library sector to build on significant research already carried out by peak bodies in Australia and internationally.

This page contains a matrix aligning project outcome metrics with ALIA and APLA outcome areas (as described in their published Guidelines, Standards and Outcome Measures for Australian Public Libraries, July 2016).

Underneath the matrix, the dials illustrate the percentage level of agreement recorded by respondents across the project network for each of the six ALIA/APLA key outcome areas. This is followed by year on year comparisons for the three years of the project.

Alignment with ALIA/APLA Outcomes
ALIA/APLA OutcomesDimension NameLibrary Outcome Measure
Community EngagementConnectionThe library has encouraged me to become more engaged in the community
RoleThe library is an important part of the place where I live
RelevanceThe library is changing in ways that increase its relevance for me
Digital InclusionDigital ConnectionThe library helps me to be more digitally connected to others
Economic and Workforce DevelopmentEnterpriseThe library has helped me to improve the success of my business
Literacy and Lifelong LearningTime with ChildrenThe library has helped me to spend more quality time with my children
SkillsThe library has helped me to learn and gain new skills
LiteracyThe library has encouraged me to read more
Personal Development and WellbeingLearningThe library has helped me to enjoy learning
Stronger and Creative CommunitiesSafe and TrustedThe library makes me feel safe and welcome
CreativityThe library has helped me be more creative in my own life and work
Loading...

Year on year

Loading...

Insights

Strong levels of agreement - greater than 70% of respondents across the network - were recorded for four of the ALIA/APLA outcome areas, with libraries felt to contribute most to the development of Stronger and Creative Communities (80%) and Community Engagement (75%).

Year on year data shows that there was a a significant increase across all outcome areas in 2020-21, with outcomes dropping back slightly in 2021-22. Results recorded by libraries for Community Engagement, Personal Development and Wellbeing, and Literacy and Lifelong Learning were still higher in 2021-22 than when the project began, while Digital Inclusion and Stronger and Creative Communities recorded the same results as 2019-20. Economic & Workforce Development was the only outcome area that recorded 2021-22 results below the 2019-20 average.

Photo: Unsplash - Alexander Wende

ILFA Learning Outcomes

This page contains a matrix aligning project outcome metrics with ILFA Learning outcome areas.

Underneath the matrix, the dials illustrate the percentage levels of agreement recorded by respondents across the project network for each of the five ILFA Learning outcome areas. This is followed by year on year comparisons for the three years of the project.

Alignment with ILFA Learning Outcomes
ILFA Learning OutcomesDimension NameLibrary Outcome Measure
Activity, Behaviour and ProgressionConnectionThe library has encouraged me to become more engaged in the community
Digital ConnectionThe library helps me to be more digitally connected to others
Time with ChildrenThe library has helped me to spend more quality time with my children
Attitudes and ValuesRoleThe library is an important part of the place where I live
RelevanceThe library is changing in ways that increase its relevance for me
Enjoyment, Inspiration and CreativitySafe and TrustedThe library makes me feel safe and welcome
CreativityThe library has helped me be more creative in my own life and work
Knowledge and UnderstandingLearningThe library has helped me to enjoy learning
LiteracyThe library has encouraged me to read more
SkillsEnterpriseThe library has helped me to improve the success of my business
SkillsThe library has helped me to learn and gain new skills
Loading...

Year on year

Loading...

Insights

Strong levels of agreement - greater than 80% of respondents across the network - were recorded for three of the ILFA Learning outcome areas, with libraries felt to contribute significantly to Attitudes and Values (82%); Enjoyment, Inspiration and Creativity (81%); and Knowledge and Understanding (80%).

Year on year data shows that results for Knowledge and Understanding have improved each year, while Activity, Behaviour & Progress and Skills improved on the 2019-20 outcomes but fell slightly short of their peak in 2020-21. Attitudes and Values recorded the highest levels of agreement every year, however this dropped by 1% in 2021-22.

Photo: Pexels

ILFA Social Outcomes

This page contains a matrix aligning project outcome metrics with ILFA Social outcome areas.

Underneath the matrix, the dials illustrate the percentage level of agreement recorded by respondents across the project network for each of the three ILFA Social outcome areas. This is followed by year on year comparisons for the three years of the project.

Alignment with ILFA Social Outcomes
ILFA Social OutcomesDimension NameLibrary Outcome Measure
Stronger and Safer CommunitiesConnectionThe library has encouraged me to become more engaged in the community
Safe and TrustedThe library makes me feel safe and welcome
Digital ConnectionThe library helps me to be more digitally connected to others
Time with ChildrenThe library has helped me to spend more quality time with my children
Health and WellbeingEnterpriseThe library has helped me to improve the success of my business
LearningThe library has helped me to enjoy learning
SkillsThe library has helped me to learn and gain new skills
LiteracyThe library has encouraged me to read more
CreativityThe library has helped me be more creative in my own life and work
Strengthening Public LifeRoleThe library is an important part of the place where I live
RelevanceThe library is changing in ways that increase its relevance for me
Loading...

Year on year

Loading...

Insights

Strong levels of agreement - greater than 80% of respondents across the network - were recorded for the ILFA Social outcome area related to Strengthening Public Life (82%).

Year on year data shows that results recorded by libraries in the network fell slightly in 2021-22 for Health & Wellbeing and Stronger & Safer Communities, but increased overall since the project started in 2019-20.

Photo: Unsplash - Tu Tram Pham

Library Awareness

Respondents were asked whether they were aware of various library resources and services, and whether the library helps them to learn, apply learnings and gain confidence across these service areas. These four stages are expected to be sequential, with library users initially gaining awareness, followed by learning something new, application of learnings and increased confidence.

Awareness: The library has helped me be more aware of applicable resources and services it provides

Learning: The library has helped me learn something new

Application: I intend to apply what I’ve learnt at the library

Confidence: The library has helped me feel more confident

The chart below shows the proportion of respondents who indicated that the library helped their awareness, learning and application across seven resource areas, and gives insight into their progression through each stage. This is followed by a set of charts showing the year on year comparisons for awareness, learning and application. Confidence is reported separately, as the areas of interest are worded differently.

Awareness, Learning, and Application

Loading...
The question statements for each category were as follows: Awareness: The library has helped me be more aware of applicable resources and services it provides in the following areas; Learning: The library helped me learn something new in the following areas; and Application: I intent to apply what I've learnt at the library in the following areas.
Loading...
Loading...
Loading...

Confidence

Loading...
Loading...

Insights

As illustrated by the dimension results, libraries appear to be succeeding in raising awareness and facilitating learning in the traditional areas of reading and education, while also playing an important role as a safe civic space for their communities to connect. Increasing awareness of the changing role of libraries in developing business opportunities and job skills may lead to improvements in learning, application and confidence in these areas.

Year on year data shows that awareness, learning and application has fallen slightly in 2021-22, following a strong increase in most areas in 2020-21. This has had a subsequent impact on confidence levels, with fairly large drops in confidence around becoming involved in the community, with education and the process of learning, when using digital resources, and helping children learn. Interestingly, application of reading and literacy skills increased in 2021-22.

A small selection of insights derived from filtering the dynamic dashboard by age include:

  • Respondents aged under 20 recorded much higher awareness, learning and application associated with reading/literacy, and education; and higher confidence with reading, education and the learning process as a result of their library experience

  • Respondents aged 20-29 demonstrated much higher awareness, learning and application associated with civic/community engagement and economic/business development, and greater confidence about becoming involved in the community, about the job search process and establishing a new business

  • Respondents aged 30-49 demonstrated much higher awareness, learning and application associated with early childhood literacy; and greater confidence with helping children learn. Respondents aged 40-49 also had much higher application of education and lifelong learning than the average

  • Respondents aged 50-59 had higher than average awareness, learning and application in all areas except early childhood literacy, and higher than average confidence with their education and the process of learning

  • Respondents aged 60-79 had greater than average awareness of digital learning services and confidence when using digital resources as a result of their library experience

  • Respondents aged over 80 years had higher than average confidence about reading

  • Respondents who identified that they were aware of library services in any area experienced higher outcomes across the board. This was particularly evident for those who were aware of library services related to job skills and economic/business development

Photo: Unsplash - Jouwen-Wang

Library Use

Respondents were asked their frequency of library use and whether they engaged with library facilities, services and events. This helps libraries to understand usage by demographics and identify links between use and outcomes.

Loading...
Loading...
Loading...
Loading...
Loading...
Loading...

Insights

The data shows that survey respondents are most likely to be engaged users of the library, with more than half having accessed the library service within the last week. This figure has remained constant over the three years of the project, while the percentages of respondents who haven't used the library in longer than six months have fallen in 2021-22.

83% used the library facilities and services (just 1% down from 2020-21), while 60% used library services online from home (up from 51% in 2020-21). Just 20% attended an event or program at the library, which was slightly down on the results from the 2019-20 and 2020-21 financial years.

In 2021-22, less than 1% of respondents were not library members. This was up from 6% in 2019-20, while interestingly 20% of respondents in 2020-21 were not library members. This may indicate that online programs, services and activities provided by libraries during Covid lockdowns encouraged use of the library from people who were not traditional users.

A small selection of insights derived from filtering the dynamic dashboard by age, gender and identity include:

  • Respondents aged 20-29 years are more likely to access library services online from home, less likely to attend an event or program at the library

  • Respondents aged 30-39 years are much more likely than average to attend an event or program, and more likely to be female

  • Respondents aged over 80 are more likely to be male than average, and a lot less likely to access library services online

  • Respondents that speak a language other than English are more likely to be aged under 50, and slightly less likely to access services online from home

  • Respondents of Aboriginal or Torres Strait islander ancestry are more likely to be under 40, and less likely to access library services online from home

Photo: Unsplash - Phil Aicken

Demographics

Demographic information is collected to enable participating libraries and the network as a whole to understand the communities that they service, the impact of services for different groups, and gaps and opportunities for broader and deeper engagement. The 5,404 survey respondents across the network in 2021-22 represent a mix of ages, genders and identities, ensuring that the opinions of a broad range of community members are reflected.

Loading...
Loading...
Loading...
Loading...
Loading...
Loading...

Insights

The data shows that survey respondents are most likely to be female (69%, down from 77% in 2020-21 and 74% in 2019-20), and in the 60-79 age group (43% in 2021-22 and 2019-20, dropping to 35% in 2020-21). The largest percentage of respondents identified as speaking a language other than English at home (20% in 2021-22, slightly down from 21% in 2020-21, but up from 14% in 2019-20). The percentage of Aboriginal or Torres Strait Islander respondents increased from 2% to 5% in 2020-21 and then fell to 1% in 2021-22. Those identifying as LGBTIQA+ increased from 3% to 5% in 2020-21, but dropped back to 4% in 2021-22. The percentage of respondents living with disability fell from 7% to 5% in 2021-22.

Photo: Unsplash - Claudio Schwarz

Outcomes by Demographics

The charts below apply the age and gender filters to the dimension results to identify where there are differences in outcomes experienced by respondents based on their age or gender.

Loading...
Categories with fewer than 10 responses are shown as white dots to indicate that the value may not be representative due to the small sample size.
Loading...
Categories with fewer than 10 responses are shown as white dots to indicate that the value may not be representative due to the small sample size.

Insights

The data shows that the differences in dimension results by age group are not significant for seven of the 11 dimensions, with larger differences occuring for Skills (outcomes falling as age increases), Time with Children (lower outcomes for respondents over 60 years), and Digital Connection (lower outcomes for respondents under 20). The most consistent dimension amongst age groups is Safe and Trusted, with average scores ranging from 81-85%.

The data shows that there were not significant differences in the outcomes experienced by male and female respondents, with females recording marginally higher outcomes for six of the 11 dimensions, equal outcomes for two dimensions, and slightly lower outcomes than males for two dimensions (Digital Connection and Creativity). The one significant difference was for the Time with Children outcome, in which females recorded average scores 9% higher than male respondents. Respondents who identified as an alternative gender recorded lower than average outcomes for all dimensions except Literacy, which was the highest average of 79% from the three groups.

Photo: Unsplash - Martin Adams

Open Text Analysis

Qualitative feedback was provided in the form of responses to two open-ended questions contained in the Library Use and Outcomes Survey:

  • "What do you like most about your library?", and

  • "What could the library do to improve?"

While open text questions provide a wealth of information for libraries, it can be time-consuming to review all responses, interpret the general themes and issues, and then use them to make meaningful change that appeals to a significant proportion of respondents.

In order to assist libraries in quickly understanding the key topics of interest for respondents, Culture Counts carried out a thematic analysis of the open text responses (more than 10,000 responses collected for each question across the network) to identify recurring themes in the text responses.

The charts below show the percentage of responses that align with each of the seven key topics identified through the analysis. Note that each response can mention multiple themes, while others may not mention any of the seven key themes identified. This means that the proportions will not necessarily sum to 100%.

Loading...
Loading...
Photo: Unsplash - George Tasios

Talk to us


We would love to partner with you on your evaluation journey. Please complete the enquiry form and we will be in touch!

Data and Insights by

This report has been prepared by Culture Counts.

We respectfully acknowledge the Traditional Owners of Country throughout Australia and recognise the continuing connection to lands, waters and communities. We pay our respect to Aboriginal and Torres Strait Islander cultures, and to Elders past and present.