ISO TR 20983:2003 download.Information and documentation Performance indicators for electronic library services.
I ntrod u cti on
This Technical Report is concerned with the evaluation of electronic library services.
The successful provision of these services depends on close collaboration between publishers, network providers, and librarians. Their needs for data about various aspects of performance are different, and the ability to collect data may not rest with the person most interested in that data. For example, the publisher may be best placed to collect data on the usage of a particular service; whereas the librarian may have most interest in using that data in developing and evaluating the service.
The main purpose of this Technical Report is to spread knowledge about the practice of evaluating electronic library services, Although electronic library services have been developed over more than thirty years, the technical environment has changed significantly in the last ten years. The need for performance indicators for electronic services was acknowledged in ISO 11620: 1998 Information and documentation Library performance indicators. It was also acknowledged that at the time of publication of that International Standard, there were no indicators that had been tested and documented, or that were in widespread use. Although there has been significant progress in the last few years on the definition, testing and documentation of indicators, it is still too soon to determine which indicators will be most generally useful, or become widely used. This Technical Report provides a standardized terminology and concise definitions and descriptions of a selection of performance indicators, in the same format as that used in ISO 11620. Publications listed in the Bibliography [2,3,5] provide more detailed information on data collection and analysis.
Only a small number of the indicators presented here are indicators of service quality: work on this area is not yet mature enough to be included. The indicators that are included have been tested in libraries, or are simple adaptations of similar indicators. In each case they have been judged against the criteria established in ISO 11620. That is. each indicator is judged to have informative content, to be reliable, valid, appropriate, practical, and in some circumstances may be used for comparative purposes.
The indicators presented in this Report are a representative selection of those which have been tested. A further selection would have taken longer to prepare, and to publish. Developments in the measurement and evaluation of electronic library services will be monitored by a Working Group, which will propose additional indicators, and modifications or adaptations to the indicators here as they are tested and validated. This Technical Report should be considered as work in progress towards the development of an International
I Scope
This Technical Report is applicable to all types of libraries in all countries. Limitations on the applicability of individual performance indicators are listed in the scope clause of the description of each indicator (see Annex B).
Indicators may be used for comparison over time for the services provided by the same library. Comparisons between libraries and services should only be made with caution, taking into account differences in the constituencies of the libraries, with good understanding of the indicators used, and careful interpretation of the data.
This Technical Report does not include indicators for the evaluation of the impact of libranes either on individuals or on society.
Throughout the text, the names of indicators are printed with initial capitals for significant words, e.g. Workstation Use Rate. This helps to distinguish the names from supporting text.
2 Normative references
The following referenced documents are indispensable for the application of this document. For dated references, only the edition cited applies. For undated references, the latest edition of the referenced document (including any amendments) applies.
ISO 2789:2003, Information and documentation — International libraty statistics
ISO 5127:2001, Information and documentation — Vocabulary
ISO 11620:1998, Information and documentation — Library performance indicators
ISO 11620:1 998/Amd. 1:2003, Information and documentation — Library performance indicators — Amendment 1: Additional performance indicators for libraries
3.10
library collection
all documents provided by a library for its users [Iso 2789]
NOTE 1 Comprises documents held locally and remote resources fOr which permanent or temporary access rights have been acquired.
NOTE 2 Access rights may be acquired by the library itself, by a consortium and/or through external funding.
NOTE 3 Acquisition is to be understood as deliberately selecting a document, securing access rights and including it in the OPAC or other databases of the library. Interlibrary lending and document delivery and excluded.
NOTE 4 Does not include links to Internet resources for which the library has not secured access rights by legal agreements (eg legal deposit right), license or other contractual and/or co-operative agreement. Free Internet resources which have been catalogued by the library in its OPAC or a database should be counted separately.
3.11
population to be served
number of individuals for whom the library is set up to provide its services and materials (ISO 11620]
NOTE For public libraries this will normally be the population of the legal service area (authority): for libraries of an institution of higher education this will normally be the total of academic and professional staff plus students.
3.12
rejected session (turnaway)
unsuccessful request of a database or the OPAC because of requests exceeding simultaneous user limit
(ISO 2789]
NOTE Rejection through entry of wrong passwords is excluded.
3.13
remote session
a successful request of a database or the OPAC established from outside the library building (adapted from
EQUINOXI1I)
4.3.3 IndIcators for traditional library services where analogous Indicators can be found for electronic library services
In this group, the indicators for electronic services are based on the same concepts as those for traditional services, but the basis for measurement is different. For example, Library Visits per Capita would be replaced by an indicator relating virtual visits to the population to be served. The various indicators for lending documents have analogues now that digital lending is possible.
The fact that analogues can be found suggests that, in a future revision of ISO 11620, ii may be necessary to devise composite indicators that integrate traditional and networked services. For example, the indicator Library Visits per Capita could be redefined to include both physical and virtual visits. Two composite indicators are included in this Technical Report: Percentage of Virtual Visits to Total Visits, and Percentage of Information Requests Submitted Electronically. Librarians may wish to create other composite measures and indicators if that proves helpful in their own circumstances. See Table 4.
5.2 Selection of performance indicators
The indicators included in this Technical Report have been developed and tested by libraries in several projects worldwide. They were chosen for this report as most helpful for libraries in general at the present time, and as complying with the criteria for performance indicators named in ISO 11620.
Not all of these indicators might be useful for every type of library or every single library. Libraries will have to decide what indicators are most appropriate for evaluating their special services and products. The decision should be taken in consideration of the library’s mission, goals, and objectives, and the population it is set up to serve. There should be an agreement with the institution and relevant authorities, as well as with users, as to what indicators would be most informative for evaluation of the library’s electronic services.
Collecting data for performance indicators can be time-consuming and expensive. The decision to use certain indicators may therefore also depend on their practicality, or possibilities of automated data collection, or on the urgent need for the evaluation of certain services. This might be the case when an authority requires data for a particular service, or when it is apparent that a service does not operate satisfactorily. It should be ensured that the data of indicators will be used subsequently for operational management and for the planning process.
The indicators named in this Technical Report form a limited set that is applicable to the main electronic library services and that has been already tested and applied by libraries. They might not prove sufficient for all sorts of special services, and libraries will probably develop and test more and specialised indicators for their special needs.
5.3 Limitations
5.3.1 Measuring the impact of electronic library services
The indicators in this Technical Report evaluate the library’s performance by quantitative statements about the supply, use, costs, or market penetration of electronic library services. Libraries are most concerned about the impact of their services, the question whether users benefit by using the services. Such qualitative data should be collected in addition to the quantitative data named in this Technical Report. They can be collected by using methods like user satisfaction surveys, focus groups, or interviews.