Held in Sydney on 27 and 28 November 2012, this conference pulled together some fascinating presentations on challenges for libraries, particularly academic libraries, to be relevant and demonstrate value in the twenty first century.
Peter Rathjen, Vice-Chancellor, University of Tasmania opened the conference with a range of provocative thoughts on universities, information, libraries and relevance. He noted the importance of trends in research – particularly the strengthening of cross disciplinary in the last decade. Provocatively he suggested that if we thought about what are the great libraries there are none in Australia.
Significant concepts explored were:
· Collections including wider community access
Libraries were encouraged to evaluate their contribution against these aspects.
The environment is characterised by changing resources. There has been an evolution from books and scholarly journals to a demand for organisational contribution and knowledge skills. He noted that changing behaviour of researchers meant they were no longer using libraries physically as part of their research mission
He saw a renaissance for libraries in their roles of supporting e repositories and contributing to access for the community including access to research outputs through repositories
Mapping the Library he proposed an evaluation as follows:
He suggested that disciplinary differences may be significant.
In moving to a world full of data intensive equipment, data and information will be the key assets for research programs. There will need to be an investment at a new scale. The nation will require national approaches, broad access protocols, and physical access. We all will need to think internationally. Universities will have value in maintaining data that is not available internationally; the world will come to you because of these strengths. This will be a key aspect in attracting funding.
At the University of Tasmania a program to collect data across the island through broadband to achieve value and insights to give new answers has been developed, It requires very significant funding to achieve systems to set up data collection, manage and develop analytic reports.
Key issues for academic libraries include:
· bibliometrics and usage.
· segmentation is the key disciplinary level research vs. teaching
· a voice for the broad research community.
· Knowledge insights
There is a strong need to connect to senior management including Vice Chancellors and Research offices (including on issues such as infrastructure, bibliometrics, ranking strategies, IT Data management).
RAY CHOATE, University Librarian, University of Adelaide
Ray encourages us to consider redefining what a research collection is - not the number of volumes held but clicks and use – encompassing digital and print collections. In redefining collections the important question is - what do we mean by density (and richness) in the 21st century and digital world including our collections, Google ephemera, data.
He noted that we are all becoming coming more and more selective regarding collection building. The University of Adelaide is re-evaluating existing collections e.g. Czech collection of about 1,000 volumes in light of their value to the university and pressures on the Library.
A major question is what do we do about last copy? The Group of 8 project ceased, as online environment now stable and the premise for the project is no longer applicable.
He discussed storage and collection overlap. The South Australian university libraries have a joint approach to management of off-site materials. A recent study found the store had a 25% overlap, based on a study of 1m monographs in joint SA store. Journals are likely to have much more overlap, not surprisingly. We tend to have more common journal purchasing across the academic environment.
The underpinning concept is a move to just in time, but would it be more effective to order articles via Google or on demand rather than pull from store? A challenging question.
He noted that it is important now to think about our special collections, what will differentiate us from other universities. Part of the collection at the University of Adelaide is university archives, theses, formed collections e.g. Alpine collection which includes ephemera, theatre collection. Digitisation of special collections is critical. This raises the question do they remain unique, to which the answer is yes and no. The University of Adelaide has digitised Fisher collection. It now receives several thousand hits a month. They are now working on the Braggs collection, which will be launched next week. The next adventure is to seek to digitise in three dimensions material from the Bragg collection.
A fundamental issue is creating the workforce for the future - to support traditional use, digitisation and new digital services.
Our role is critical with the emergence of lifelong learning and community support as an important use. He forecasted this as a major issue in the next decade. We need to foster the development and support of scholar librarians especially for special collections.
He suggested looking at creating a Centre for Scholarly Research and innovation including publishing, special collections, as well as information literacy
SHARAN HARVEY, Manager Library Services, Brisbane City Council
Sharan gave background on the Library Service which is located in Brisbane. It is a large library service $48.5 m budget plus $2.5 m capital for 2012/13.
The Library’s vision has 8 themes alighted to the vision of the council. A major theme is one council.
The Library is committed to strong performance leadership. Continuous improvement is adopted at the level of the team. Every individual can tell others how spend council money.
There is a focus on the importance of Lifestyle to the future of Brisbane – a differential point with other cities. Economic development plan released by the Mayor provides a strong context. There has been time to articulate brand change for strategic approach of library move from book to community.
She commented that connecting in a digital world space matters and place matters.
Lifestyle service themes include:
· Read powerhouses for children's literacy
· Get answers
Measuring value has been an important issue for the library service. The measure customer satisfaction - highest council satisfaction service 95% with 95% say it is important to them. Citizens are aware that council is responsible for libraries – 96% from the latest survey.
Performance + importance + future behaviour to give current and future value
She proposed a concept of natural range for funding e.g. library compared to parking or potholes.
A recent study measuring the economic value of libraries was funded by SLQ. The report is available of SLQ website. The findings were that libraries are highly beneficial demonstrating $2.80 benefit for each $1.00 spent.
In addition there is value from being a one stop shop for council services.
72% of customer visits associated with shopping. The Library is seeking to support the late night economy, “Libraries up late”. The objective is to move more of the community sharing the city at night. Amongst other programs are developments of I- skills in the city and a program of high profile authors - branded to the city.
A key goal is building "our reputation".
During Brisbane flood contributed -mass citizen volunteering event 15-16 January run by the Library with remarkable effect. The event marshalled over 22,000 citizens.
She summarised that value is a team sport, we need to give it to get it. And sometimes need to take one for the team graciously.
KAREN JOHNSON, Executive Director, Library Services, Charles Sturt University
Karen described evaluation based on Survey/questionnaires. She talked about the huge range of questionnaires run within the university including the student experience questionnaire. The Library recoded the most significant improvement (plus 7).
CSU Library collects daily feedback through Reftracker, ref chat, web form, Deskstats is used to see trends in issues. Primo feedback an important area of client suggestions.
The Insync survey enables comparative data, useful comments for analysis.
CSU uses a Quality framework based on business excellence framework. Plan Üimplement Üreview Üimprove.
The primary areas identified for improvement were Primo and the website.
ANNE HORN, University Librarian & Executive Director, Academic Support, Deakin University
Anne described the TEALS project, which provides a framework for evaluating physical projects. It is a joint research project with school of architecture and building. It aims to provide a framework and data collection tools to evaluate academic library spaces. Overall goal: to determine whether library spaces function as deigned.
The purpose was to be empowering and reflective - people, programs, places and partners. See Building futures report 2004 p. 7. Focus on students.
The underpinning question was return from investment in buildings.
The methodology was: Phase 1 exploratory research. Phase 2 Pilot study. Phase 3 evaluation framework, package of tools.
Online survey of students library experiences, an observation study list, and a list of questions for staff focus group discussion were used...
Criteria of quality - 10 developed, evidence based. Established quality indicators under each criterion.
Impact starts before you reach the door to the Library from an architectural point of view. What is the story you are telling through the library space that reflects its location?
Dimension for evaluation:
1. Positive image and identity, sense of presence
2. Welcoming and inviting entry
3. Functionality and efficiency incl. control of noise
4. Flexibility and adaptability
5. Variety of spaces for different users and uses
6. Being social and people centred
7. A sense of place and inspiration
8. Environmental comfort and sustainability
9. Access, safety and security
10. Integration of technologies - needs to be further focused
Deakin promise - delighting students is more than numbers thought doors. We need tools that will measure our unique selling proposition (at Deakin).
Now less a fan of iconic projects, major issue that there is a five year gap between design and completion in this sort of project.
ROUNDTABLE ON KPIs to meet stakeholder needs
Challenges of creating meaningful KPIs that are not just numbers that are operational and about process rather than strategic goals.
Measure to influence staff understanding of organisational priorities.
use in elevator conversations, regularly reports and let everyone know.
Lots of examples of good and meaningful statistics and breaking them down to show a pattern.
meaningful full comparisons with other libraries, selective benchmarking.
LINDA MULHOLERON, Manager Library Information and Resources, Westmead Hospital
She described tying library standards to NSW Health's vision and health accreditation standards.
She noted that there was a need to remember we are part of an organisation and communicate to al on the value we give to that organisation. We must evaluate to ensure quality (to standards that may be predetermined).
Again the message was focus on customers (and potential customers).
EQuIP (health industry) standards - national evaluation scheme. includes productivity measures e.g. circulation and gate statistics. Standards for turnaround in requesting ILLs - 24 hours.
EQuIP action plan. She took results into account with four key improvements from the last evaluation. Key issues were after hours access to library, development of CIAP self education models, point of care training, ongoing budget evaluation and review for cost effectiveness and evaluation for transition to e- journals.
DAVID HOWARD ECU
Assumption was that everybody gets libraries. Old vision of books, new of an e collection, rather than an active participant in the university. Need meaningful discussions about the value libraries can add. Need to engage with senior university staff about what we can do. Libraries need to repackage our message. Internal processes are focused on traditional work on collections assuming lectures and traditional teaching go on, rather that reaching out in innovative ways to support online learning. Led approach to get into other business plans e.g. research. Open access and data management are areas we can show leadership in that are focused in the future.
SHARAN HARVEY BCC
2 sides - one delivery to clients, second to reflect on relationship within organisation. Actively contribute to the organisation - be the champion for what you believe in e.g. wi Fi, literacy. Then become Lord Mayor’s deliverables which must be delivered. Be at the table and deliver on the gods, don't be afraid to be accountable. Garden - run the blog, be customer focused, be a good organisational gardener. Have quick responses to remarks e.g. we won't need a library in 5 years. Report in context of bigger agendas, e.g. economic development plan.
Breed champions (including ex bosses).
Timing - understand issues of organisation and do it at the right time. Timing is everything, understand the organisations temperature.
Be loud and proud. sell your expertise. Easy for the Library to be overlooked, make sure all senior staff are taking the right messages to the right conversations. Look at what the universities want help with and how we can contribute.
Can't sell to a want only a need. Different skills needed for staff - need assurance that the advice staff are giving are quality advice. Including on copyright.
Margie JANETTI, University Librarian, University of Wollongong
Partnership with IT Brian Cox to develop Library cube
The challenge was to build a library data warehouse on collections, use and student grades
Data sources :
· Student data
· Loans - snapshot exported weekly
· Electronic usage - ezyproxy log
Logs pulled every 10 minutes, mapped to student identification number. Mapped hours spent on resources to student grade - interpret on average if x hours spent on electronic resources this is what the student mark will be
Important message not just for stakeholders/senior management but for students
Correlation for loanable resources, but not as strong as for electronic resources.
Now part of institutional reports - regular report on hours spent and grades trend over time. Can see distribution of usage by faculties.
Navigation menu offers things like space utilisation, student accommodation and not library one of those reports. "Dimensions" menu offers different ways to slice and dice the data e.g. head count, eftsul numbers, gender, international countries, location. Can drag and drop the dimensions.
· Some limits in correlation,
· arbitrary measures, business rules,
· many external factors affect grades e.g. academic influence.
Conclusions - students prefer electronic resources, 19% of students who didn't use electronic resources failed, only half a percent of heavy users fail. importance of using continuous data. Now think about how to develop communication interval tin strategies.
Questions - when should we be concerned that the resources are not being used. How can data be shared across the university for better support for student achievement.
Can tailor library promotion to students better - to particularly groups etc.
The solution provides more effective information than satisfaction. Only have 3 years data - more work to do. Factors - first year trends tend to carry through. more impact on younger students.
JUDY STOKKER, Director, Library Services, Queensland University of Technology
Traditionally we have had well defined role in supporting research. Tenopir argues we need more than one measure of e journal use.
The fourth paradigm by Jim Gray suggest that changes in research moved from empirical to data exploration (through theoretical and computational)
Federal government investment road map 2002 to 2013. Many reviews and reports on building new research infrastructure including area where libraries can assist including open access and data management. Over past decade significant increase in research funding in Australia. Increases opportunities for academic libraries to contribute.
Changes in subject liaison libraries to support research. Changes in info literacy to provide advice on open access and scholarly publishing. New roles being sought from the library that add value. Question of can or will anyone else offer this service. Is it making a difference? These are perspectives from which to assess developments.
Institutional repositories: new role playing a part in the research process enabling building of collaboration and other researcher benefits. A sign of maturity is that there is an International ranking of repositories.
40% universities are looking at including data in their repositories.
Research training - has expanded enormously. Have had 2000 researchers going through the training in the last 18 months.
Tracking research impact highly valued by VC and DVCs. Started to provide advice, support and
Training on measuring impact in social media.
Strategies for growing from research support into research partnering.
· Building a broader perspective
· Building capacity
Messages for now and into the future
- Know what builds value
- Demonstrate what we do
- need to be ahead of the game
- Not just our business anymore
- Leverage existing strengths and market share
- Alignment with organisational goals
New spaces and roles are vital e.g. electronic lab notebooks.
HOW TO DETERMINE WHAT TO MEASURE: PANEL
Margie JANETTI. Heavily influenced by Australian Business framework. take a more holistic view in thinking about excellence. There is a need for data to provide a platform for change. transformations in measurement over the past decade. 4 key indicators such as demand, to ensure right mix of service and resources. Next organisational excellence to encourage continuous improvement. Third indicator learning and growth - anticipate competences needed now and into the future - professional capabilities. Final indicator is impact. To make us think, challenges us to think about what we do differently, having an impact. Loosely based on BSC. Impact and effect on student academic performance. engagement with a range of studies. Collaboration success needs measurement, inter team within the Library and collaboration across the institution. Will continue to use client satisfaction. Accepting a form of narrative - accept this as a proxy measure for now. Iterative model evolving within the university to develop performance indicators.
Jennifer Peasley. Macquarie’s journey. Historically focused on counting without enough analysis. Numbers not reflecting role or quality of services. Moved to client centred measures of success. For example instead of measuring the number of ills/doc supply it was turn around. not always easy to measure. started to do more qualities information gathering, focus groups, snapshot surveys. Multiple assessment frameworks, including teaching standards framework. Looking at embedding assessment of the library into the more common surveys run for students. Looking at assessment against Megan Oakleaf's checklist. Important to ensure everyone understands what we are doing and why.
Karen Johnson. CUS follows Australian business Excellence framework. Analyse large amount of quantitative data from surveys, SAF data etc. Will be speaking to researchers and Research Office on benefit of library's support services. Importance of attracting higher degree students and retaining them. need hard data to develop narratives and success stories. Gather and publish data.
Concern are we over surveying clients especially students.
importance of regular reports to executive, 2 times a year, about achieving support for a university goal area. Aligned messages.
MAL BOOTH, University Librarian, University of Technology Sydney
Measuring and improving library values. Environmental scans for key issues in research and tertiary education. UTS framework Building and Leading Research Performance.
Strengths - networks, partnerships. OA repositories, copyright expertise, enjoy trust and respect,
Weaknesses - invisibility, own perceptions, OA is misunderstood, limited budget, inflexibility, cautious, etc.
Opportunities - OA promotion, personal relationships, bibliometrics, research metadata management, online help, OA publishing innovation, altmetrics, alignment, new collaborations (publishing, crowdsourcing and other forms), social networks for research, etc.
Threats - Loss of reputation, becoming too digital, OA gold model, permissions/control, time available to researchers (can't do a one size fits all- some can only attend classes on weekend etc), invisibility of our services, missing the boat.
How can we tell - data etc. repository rankings. Invitations and requests to participate in research.
Examples from UTS: UTS publications
- OA repository and E press,
- assistance with data, data archive register, data management workshops, data visualisation
- online support for research
- research skills training support
- research week (annual)
Lessons learnt: keep alignment with university, timing is everything, strict vs. open, blogs must have a pulse, personalisation and voice, OA and APCs, stop doing some things, cater for different needs, go deeper.
EVE WOODBERRY, Pro Vice-Chancellor Students and Social Inclusion, University of New England
Selling to senior executive is very different to selling to users.
Competitive environment requires us to develop economic arguments.
Bradley reforms. Elearning and googalisation.
Libraries need to demonstrate how to add value for student retention and completion as wells as researchers/hdr students.
Making the library visible when many clients are only interacting with the online resources as losing visibility.
Contribution to retention.
- Libraries do add value
- Dependent on the context
- value and impact need to relate to your individual environment
MEREDITH MARTINELLI and JENNIFER BREMMER, Macquarie University
KPIs and critical to the success of the organisation, tied to business goals, measurable and small in number.
Risks, can be internally focused, loosely defined, too may, only associated with numbers
Must be linked to academic plan.
University vision. Quality enhancement model. principles based library. Collection organisation principles, tested with academics. Then plan, also tested with academics.
Macquarie is reviewing their quality framework, so Library will put off review of their quality framework till this is completed.
What are our stakeholders and clients telling us
- Significant increase in use of physical and virtual library spaces
- Insync survey reflected the same issues of space and access to computers, although the availability of these had doubled
- Need greater insight into client expectations, especially segmentation
Assessing what is collected (stats) and whether it is used. testing what does this tell us about service quality and client relationships.
Frameworks. Position Library within workflow of research and teaching/learning (teaching standards framework, research support framework, leaching and learning framework). Provide opportunity to assess "value and impact".
Embedding library outcomes and measures in academic plan and embedding library in new masters of research program.
Value impact and the Macquarie directions. Must be flexible, significant senior leadership at Uni and library changes. Strategic support plans critical. Metrics that matter - stats collected must have a real purpose. Partnering - Woking with campus partners to build value proposition.
US Library assessment conference 2012 take home messages
- Increasing challenge of online education
- Accountability for evidence
- Researchers require access to human knowledge
- Library role to provide access to and manage collections
- Focus on digital collaborations and digital infrastructure.
OLWYEN PRYKE, State Library of NSW
Valuing culture quote form Marie Bashir from One hundred.
Australian Council Culture count conference papers.
SLNSW moved from 16 to 32 KPIs.
Access economics study. Similar project in 2008. Components - evaluation of SLSNW and State library network.
Aimed to be accountable, demonstrate and advocate value....
Counted direct and indirect impacts, based on ROI economic impact assessment, contribution analysis, valuation techniques.
The Library makes a significant contribution to the Australian economy - 2011-12 $67million. For every three people working in the library another job elsewhere in the economy was supported.
For every $1 invested in the library $2 is returned to the community.
Will be released shortly.
Emphasises need for continued funding.
Beyond economics - importance of narrative accounts. 3 case studies.
- How history shaped the City of Sydney
- Bringing treasures to an appreciative audience New South Publishing
- history lost and found: Foxtel and WTFN
Roxanne Missingham, ANU
See presentation on Slideshare: http://www.slideshare.net/roxannemissingham/slideshelf