Panels

Panel 1	Panel 2
OCKHAM: Coordinating Digital Library Development with Lightweight Reference Models / Martin Halbert, Edward Fox, Lorcan Dempsey, Eric Lease Morgan and Jeremy Frumkin	International Perspectives on Creating a Usability Methodology for Academic Digital Libraries / Wilma Alexander, Teal E Anderson and Ann Blandford

Presentation #1: Lightweight Reference Models and the OCKHAM Framework (20 min)
Martin Halbert, Emory University, USA

Abstract: The OCKHAM meetings were convened by the Digital Library Federation (DLF) to advance the understanding and usage of digital library system architectures in ways that will further collaboration and interoperation between the many development projects currently underway in libraries and related agencies. Initial meetings led to the development of a concept we term lightweight reference models (LRM). This presentation will review the OCKHAM process for rapidly building consensus on LRMs, collaborative multi-institutional projects now underway based on the LRM strategy (including the development of three LRMs and associated collaboratively developed software for E-Reserves, Pathfinder Brokerage, and library portal functions) and anticipated next steps.

Presentation #2: Digital Library Components and OCKHAM (20 min)
Edward Fox, Virginia Polytechnic Institute and State University (Virginia Tech)

Abstract: Component-based digital library development efforts at Virginia Tech and the philosophy that led to such efforts informed the OCKHAM framework. This presentation will review the most recent aspects of the 5S model and the Open Digital Library project work, as well as how they relate to the OCKHAM framework.

Presentation #3: OCKHAM and other Digital Library Architectures (15 min)
Lorcan Dempsey, OCLC, USA

Abstract: This presentation will elaborate and contextualize the OCKHAM framework with reference to other digital library architecture efforts, including DNER, OAIS, and others. The implications of the OCKHAM initiative for cooperative efforts between groups of libraries and library service brokers also will be explored.

Presentation #4: MyLibrary and OCKHAM (10 min)
Eric Lease Morgan, University Libraries of Notre Dame, USA

Abstract: This presentation will describe how a portal application called MyLibrary@NCState will benefit from a number of lightweight reference models such as the ones beginning to be articulated by the OCKHAM framework.

MyLibrary@NCState is a portal application designed for libraries. Given a set of information resources listed in an underlying database, MyLibrary@NCState provides a means for creating a customizable set of Web pages for the library patron allowing them to view information relevant to their particular needs.

In its present implementation, MyLibrary@NCState is an application with many built-in functions for data-entry, display, customization, statistical reporting, etc. While these functions are modular in nature, these modules do not necessarily communicate with applications outside the MyLibrary@NCState environment. The proposed OCKHAM framework articulates a number of functions required by the MyLibrary@NCState application, and effort is being made to incorporate a number of these OCKHAM functionalities into the MyLibrary@NCState system.

The OCKHAM framework describes a number of lightweight reference models common among a majority of digital library systems and applications. MyLibrary@NCState is one such system/application, and this presentation describes how the reference models are being implemented into a customizable interface of sets of library collections and services.

Presentation #5:The E-Reserves LRM (10 min)
Jeremy Frumkin, University of Arizona Library, USA

Abstract: Many academic libraries have implemented (or started to implement) an electronic reserves service. Most of the systems which run electronic reserves services do not provide any standard mechanism for communication with other systems. The University of Arizona and Emory University are developing a lightweight reference model (and associated communication protocol) that allows electronic reserves systems to be built in a way that they can interoperate with other library and academic systems. This lightweight reference model will work in conjunction with other lightweight reference models that are development in conjunction with the OCKHAM movement.

Panel 2

Title:	International Perspectives on Creating a Usability Methodology for Academic Digital Libraries
Panelists:	Wilma Alexander, Edinburgh University Library, UK Teal E Anderson, Sheridan Libraries, Johns Hopkins University, USA Ann Blandford, University College London, UK
E-mail:	Wilma Alexander: wilma.alexander@ed.ac.uk Teal E Anderson: teal@jhu.edu Ann Blandford: a.blandford@ucl.ac.uk
When:	16:30-18:00, Tuesday September 17th
Where	Room Aula Magna
Length:	1 hour and 30 minutes
Session	Plenary Session 10

Description

This session will draw on the varied experience of panellists to consider the future of usability research and its application in the context of digital libraries. Current usability research activity of relevance to digital libraries includes the interactions between user interface design, knowledge management, information mapping and expert systems development. Meanwhile academic libraries of all styles and sizes develop a range of browser-based services for their users. Some have access to specialists who will advise on usability based on the latest research. Others rely on the interested and gifted "amateurs" on their staff.

Panellists offer their responses to the challenges of developing a relevant (and usable) usability methodology for digital libraries. The questions raised by these challenges include the relevance of existing usability metrics and the transferability of findings in e-commerce to the research library context.

Is the challenge of creating usable digital libraries one of designing the best possible interface? Or does it stem from a deeper-rooted challenge of better understanding and modelling of information-seeking behaviour? How do we draw on and synthesise the findings of research in areas such as expert systems modelling to provide hard-pressed academic librarians with relevant tools for their digital library development work? Is "some testing" better than none at all?

The panellists bring different viewpoints and approaches to these challenges, and will use the session to explore the dimensions of the desirable contrasted with the do-able.

Position Paper: Teal Anderson

Digital libraries merit their own set of usability evaluation methods. Digital libraries should not be evaluated using exactly the same processes that are used to evaluate physical libraries, nor should they be evaluated using exactly the same processes that are used to evaluate other web sites, such as e-commerce sites. The types of usability evaluation methods that are appropriate for other web sites (including heuristic evaluation, card-sorting, naturalistic observation, surveys, interviews, focus groups, and task-based, think-aloud protocols) may be appropriate for digital libraries, but we should not assume that identical procedures will allow us to create usable digital libraries.

Task-based protocols seem to require the most attention when they are borrowed from other areas and applied to digital library usability. We cannot assume that measures frequently used in e-commerce usability tests, such as time to complete a task and number of errors, are relevant for digital library usability. That is, even if test participants are able to find a specified item or items on a given topic quickly and without deviating from the navigational course expected by the designers, this does not necessarily mean that the digital library is usable. Conversely, if test participants spend a long time on the tasks and follow an unexpected route, this may not indicate a usability problem. Rather, a user spending more time, finding related items, or finding unexpected yet useful items, may be better indications of digital library usability. Allowing participants to define their own tasks may lead to interesting observations and useful insights, but the results become even more difficult to quantify. Finding quantifiable measures that are as meaningful for digital library usability as time to complete a task is for e-commerce usability will make task-based protocols more valuable tools in digital library usability evaluations.

Our challenge lies in adapting usability methods to digital libraries. Librarians are beginning to take interest in the usability of their digital resources. Let us offer them the best possible usability methods, so that they receive excellent returns on their investment of time and money.

Position Paper: Wilma Alexander

In his now-famous "Call to arms" in 1997 Jakob Nielsen argued that it was essential to use heuristic evaluation methods and establish user testing scenarios which could be applied by "amateurs", if the web were ever to become a usable medium. In spite of this call, usability remains the Cinderella topic, considered grudgingly as an expensive add-on, not just in libraries but throughout the world of web development. Nielsen's argument that the numbers of web pages, and rate of increase in developers creating more pages, requires a pragmatic approach to usability evaluation remains valid, especially in the context of digital library provision.

As tools for the creation of database-driven information web sites become increasingly easy to use, more and more "amateurs" will use them. In order that their efforts in bringing their digital collections to a wider world should not be in vain, we have to find ways to support and encourage these "amateurs" to create usable sites. I would advocate a highly pragmatic approach which combines a strategy for raising staff awareness of basic usability heuristics with user testing methods stripped to a bare minimum.

The issue for me is not one of statistical validity or otherwise, it is that developers, whether systems specialists or information scientists, require education and evidence that not all users view the world as they do. The "of course it's obvious!" syndrome is as widespread in digital LibraryLand as it ever was in the physical world. I believe that true progress in the usable digital library will only come when all of the staff engaged in aspects of digital provision can take a balanced and well-informed approach to web interface development.

Position Paper: Ann Blandford

The usability issues for digital libraries still need to be better understood. We can deal with relatively superficial aspects of design such as details of interface layout and simple aspects of structure, but we do not yet have a good understanding of when and how libraries are used by their intended user populations. Use of digital libraries is typically discretionary -- people can choose to use them or not -- and yet it currently takes the typical, naïve user a substantial length of time (of the order of hours rather than minutes) to familiarise themselves with a new library and to start to reap any benefits from working with it. They need to have clear pay-backs for this investment. The difference in sophistication of search between the general academic user (not trained in the information sciences) and the librarian (who may not have particular domain skills but is an expert in search strategies and information retrieval) is enormous. We need to understand this knowledge gap, and find ways of bridging it through design. In parallel, there are social and organisational aspects of the deployment of libraries -- of what information is available to who, when and where -- that are poorly understood, and that demand more research before we can possibly develop widely applicable methodologies for the design and deployment of libraries.

Established usability methods -- such as the use of personas, scenarios and heuristics -- suitably tailored to digital library applications, have an important role to play in the development of usable digital libraries. There is an urgent need to develop such approaches and to test their use by the library and information technology specialists who are most frequently charged with delivering digital library services. However, that is not the end of the story: further research is also needed on the particularities of the academic digital library domain and of the usability and acceptability features of that domain, so that appropriate usable methodologies can be developed in collaboration between usability specialists and those responsible for the deployment of such technologies.