User Tools

Site Tools


scratchpad

this is a test

06/17 testing

An Introduction to Evergreen

The documentation for the Evergreen project is primarily geared towards a technical audience—that is, one which focuses on development, installation, and support of the Evergreen software and the environment it requires to function well.

The purpose of this article is to provide an introduction to Evergreen for a general, non-technical audience. And from it, provide links to community resources for those wishing both to learn more about Evergreen and to explore the reasons for its success.

What is Evergreen?

Evergreen is software used to run the back-end processes of libraries. This kind of software is referred to as an “Integrated Library System” (ILS) or “Library Management System.” Evergreen is unusual in such software for several reasons. Two reasons are key: First, it was designed to run large, geographically-dispersed, resource-sharing networks and, as such, it has been described as a Consortial Library System. Evergreen is used to manage processes at individual libraries either as part of a consortium or independently of a consortium when required. Second, Evergreen was developed using open source principles. (Open source development is discussed in more detail below.) Open source, while relatively new to libraries, is a method for developing software that is well-known outside the library world and one which has proved to be a successful model and one that has been used in a number of other areas before being adopted in the library world. Most users of ILS software are familiar with the older “proprietary” model of software development.

Consortial software, then, was designed to handle transactions of separate libraries which share some resources. It must be configurable to allow for local variations in practices among the member libraries and Evergreen has been adapted to a variety of local requirements. Consortia that share resources find that their users like having access to more materials and they avail themselves of the increased resources. It is a common experience in these libraries to observe a “consortial effect” where such intra-consortial lending increases dramatically.

Consortial software must also be able to scale to large installations so the underlying database structure must be robust. The ability of Evergreen to handle large installations is discussed further below. Another aspect of scaling is the notion of “scale economies” from Economics and we turn now to that concept within the context of consortial software.

In the Google era, maintaining individual libraries is increasingly impractical as a solution to users' information requirements. Large consortia also gain the benefit of scale economies. The most expensive way to provision an ILS is for libraries to purchase the software independently of any other library. Costs per unit go down as installations increase in size. For example, ten small installations can save money by joining a consortium and having a single installation. The savings can be dramatic as the size of the libraries joining the group increase. However, it should be noted that in resource-sharing consortia, savings from scale economies will often be redeployed to pay for increases in courier services because of the consortial effect—there is greater demand for library materials when resources are shared within the consortium. For less money, your libraries will get much more use and they will have happier users.

A Short History of Evergreen

The Evergreen project began in the US state of Georgia with PINES, the Georgia resource-sharing library network. PINES began as a Y2K project where a number of Georgia public libraries had outdated software that, it was feared, would not be able properly to handle dates after 1999. Georgia libraries have had a long history of cooperation going back at least to the 1940s and with PINES, the Georgia Public Library Service (GPLS), the state library of Georgia, used digital means to improve the resource sharing between libraries. A GPLS whitepaper goes into some detail on the factors involved in the various decisions behind PINES and, later, Evergreen. As mentioned, PINES experienced the “consortial effect” as users of the various libraries discovered a vastly richer set of available library materials. In response, not only did use at each of the libraries increase but also more libraries wanted to join. Eventually, the software used initially for PINES reached a limit and it could not add any more libraries. The software had not been designed for multilibrary consortial environments and it reached the . As the white paper discusses, the decision was made to examine library software market and following that analysis, to develop what became Evergreen in house at GPLS. Given that they were starting from scratch, they designed a system to handle resource-sharing consortia like PINES and the decision was made to use open source prinicples to develop the software. It is also worth noting that the software was designed in a library and with the active cooperation from the library community in Georgia.

More links to sources on the history of Evergreen can be found under the heading of the Evergreen Community in the Evergreen and Open Source Glossary for those who wish to learn more of Evergreen's beginnings. What happened after the first PINES libraries went live with Evergreen over the US Labor Day weekend in September of 2006?

The Growing Evergreen Universe

Two tables here give summary data illustrating the growth of the Evergreen community. Table 1 gives a count of types of libraries over time.

The column heads may not be clear. “EG@0” presents the data on the day Evergreen first went live in 2006. There were 45 library systems and GPLS was the state library. “EG@1” is the cumulative count on the first anniversary of Evergreen's going live in PINES. The cumulative total of public libraries was then 47 as two more library systems joined in that year. “EG@2” gives summary data on Evergreen's second birthday. 15 more public libraries went live on Evergreen and one academic library. And each year since then, there have been more libraries and more of various types joining the community. The last column, “EG@4 9th month” gives cumulative totals through June, 2011.

Table 1: Cumulative Totals for Evergreen Systems by Year, by Type
  EG@0 EG@1 EG@2 EG@3 EG@4 EG@4,
9th month
Public 45 47 62 126 202 420
Academic 0 0 1 10 13 22
K12 0 0 0 2 5 27
State Libraries 1 1 1 2 3 4
Special Libraries 0 0 0 21 28 38
Systems 46 48 64 160 251 511

The second table gives totals for the number of consortia and independent systems running Evergreen. As observed above, Evergreen is the first library system designed from the ground up to be a consortial library system and, as the reader can see, Evergreen has been adopted by consortia in increasing numbers with more on the way. Most of these consortia are state or province wide and most are large and growing.

Table 2: Other Cumulative Totals for Evergreen Systems by Year
  EG@0 EG@1 EG@2 EG@3 EG@4 EG@4,
9th month
Consortia 1 1 4 10 13 18
Independent systems 0 0 3 8 16 20
States/provinces
with Evergreen
Libraries
1 1 7 16 22 28

As of this writing, then, Evergreen is being used in more than 500 library systems with just over 950 branches. All but one of these systems are in the US or Canada.

There are two additional sources of information on the growing Evergreen community. One is a list of the Evergreen libraries found elsewhere on this wiki and the second are two Google maps. One of these maps is in order by the various consortia and the second is in order by the date the libraries migrated to Evergreen. Google Maps has the ability to animate the growth of the community if you like. The directions on the page will tell you how.

One of the themes of this article is to investigate the reasons for Evergreen's success as illustrated by the numbers in the two tables above. There are several discussed so far: the fact that Evergreen was designed to handle the transactions of consortial systems was certainly important as was the fact that the first code was written in a library and with the active participation of the Georgia public library community. The fact that Evergreen is robust and designed for consortia meant that Evergreen can handle large systems at a time when the value of this functionality became clear to librarians. The fact that librarians were involved in the design meant that the software was easy to use. Another important factor is that Evergreen has been developed using open source principles.

What is Open Source Software?

The term “open source” refers to a method for developing software. Although a relatively recent addition to the library world, open source methods have been used for some time and, in fact, once understood by librarians, they will see the similarities with normal library practice. One is the cooperative nature of the “community” of users of Evergreen.

If you are new to open source, two good places to start for more information are elsewhere on the Evergreen wiki: the Open Source Software FAQ is one and the Evergreen and Open Source Glossary is another. The glossary attempts to define the terms most commonly encountered in the open source world but also provides a number of links to more information for those wishing to gain a deeper understanding of how open source development works and to contrast it with the traditional model. A reader new to open source software in libraries would likely find the glossary entry for Open Source itself interesting along with the next two entries: one on the advantages and the other on the disadvantages of open source software.

One clear distinction with open source software is that the software itself—the code—is readily available for free and can be changed and adapted to suit local requirements. In addition, no one company owns the code so that there are often a number of companies supporting the software and they will compete for your business. If you are unhappy with your relationship with one firm, you can find another. A list of companies supporting Evergreen is on this wiki.

The traditional model for library software relied on companies which used “proprietary” methods. The entry in the glossary for proprietary software and the next two entries on the advantages of proprietary software as well as its disadvantages are similar in structure to those cited above for open source advantages and disadvantages.

With this older model for software development, companies owned the software, developed and supported it, and users could not see the code nor could they change it to fit local practices. Normally, one company could support and develop the software which meant that libraries were locked into the one vendor whose software was protected from copying or adapting. With open source software, you can pick another support vendor or use multiple vendors for different functions if that is your preference. In fact, there are a number of libraries which use their own resources to support their installation and do not rely on one of the support companies. One finds a good deal of variation in support options in the Evergreen universe as a result of libraries not being locked into what is, functionally, a monopoly supplier under the traditional model. With open source, no company owns the software. Who owns the software if no one does? In this case, the community of users owns the software. There is a great deal of talk about the “community” among the users and developers of Evergreen—that is, the people in its community.

As important as the freedom open source software gives users, the freedom to develop and work on enhancements to the software is no less important. Open source software development is rapid and practical because the software is cooperatively developed by the community of its users. They see the software in use and they can change things that don't work or suggest changes rather than relying on a company to develop functions at the next upgrade cycle. In open source, one saying heard frequently is: “release early; release often.”. Another is “working code wins.” These two sayings illustrate that the goal is fast, working code. A third: “scratching an itch” refers to a fact of open source development because most development is directed at solving real, practical problems, not theoretical problems.

While open source software is free to download and install, that does not mean there are no costs. In the case with Evergreen, it is complex software requiring skilled administrators and support, but you will not pay for functions in the software already developed elsewhere. Given the complexity of the software and the requirement for technically knowledgeable staff, while the software may be free but it is not always cheap. Cost savings normally occur because factors such as the scale economies discussed earlier and as a result of savings that result in not having to pay for maintenance contracts as with proprietary vendors.

Recently, a new kind of cooperative development has taken hold that follows from the open source software development environment. It has been called the “Evergreen Superconsortium.” In a consortium, a library may give up some autonomy in exchange for a richer environment for that library's users. What is happening in the Evergreen community is cooperation between consortia where they work together to pay for and develop functionality for Evergreen. Development costs will be lower for each participant and the functionality being developed will be for a broader audience. Superconsortial development is increasingly becoming an important part of the community's development practices.

While open source development has proven to be rapid and practical, there were other forces which encouraged libraries to use open source methods.

Why Open Source Library Software?

There are at least two wellsprings of the current open source projects in the library world. One is that the traditional model, which had been largely successful, was increasingly strained. Many of the vendors were being sold and resold and moving from being owned by people with a library background to those with a background in finance and investment. This development was occurring at the time when many of these software products were old and in need of substantial revision to better handle the new information technologies in common use in libraries. If software was originally developed before the Internet or any such newer technology, then the capabilities of using these technologies can be added but often in awkward ways.

The second wellspring is a characteristic of the structure of the library world: there are few well-financed and large libraries and many small and underfunded libraries. This is a consistent fact of the library world. If your company is selling software, are you going to aim to sell to the libraries with more money or those with less? This point is elaborated elsewhere . On the other hand, if you are a library director who wants the capabilities of good software for your library's users and staff but you can't afford expensive but capable software…what are your choices? Both of the two major open source ILSs—Evergreen and Koha—started out as projects with small public libraries. Evergreen specialized in the consortial approach—small libraries joined together to share resources—while Koha was developed for small public libraries operating independently. Each has grown well beyond its beginnings, however.

Scaling to Large Installations

Evergreen is used in a number of small installations and will run on laptops when necessary but it also runs a few quite large installations. One report that has been published each of the last five years illustrates with summary data this point. PINES' busiest day is the day after the US Memorial Day (usually in late May). The latest report can be consulted for details but, briefly, 116,000 items were checked out that day by PINES libraries, and 97,000 were returned for over 200,000 circulation transactions that day. During the busiest hour, 13,000 items were checked out. During the busiest minute, 319 items circulated and the busiest second: 18. Moreover, these numbers were down from last year. The latest figures we have for PINES annual circulation (only check outs but not check ins in this reporting source) report it at 19 million. King County Library System's 51 branches circulated over 20 million and although their totals for that day are lower than those of PINES, they are still impressive.

What is it about Evergreen that makes it this flexible? We will now take a brief sojourn into Evergreen's structure and its development to address this question. This is not a detailed explanation but is a first approximation for the non-technical reader to which this article is addressed. Readers with a deeper technical interest are encouraged to join the various mail lists and IRC channels listed below in the discussion of “Communications Within the Evergreen Community.” These readers will also find the Evergreen documentation which is undergoing continuous revision.

Evergreen is Modern Software

Any large piece of software has a design architecture: how it organizes its elements, what tools it uses, that is, how it does what it does. The design architecture, in turn, is a function of various factors, the two most important being system architecture and computer costs.

Many versions of old software still litter the ILS ecosystem. The software that first ran PINES began development in the era when mainframes ruled the earth. This legacy software has been adapted and changed and added to over the years. Elements had been bolted on and kludged, but, it is a fact that updating old, deployed software has proven to be an almost impossible task almost every time it has been tried. There are a thousand problems—the older the software, the greater the number of problems—and what most vendors of library software have done is start over using the more modern updated practices of the time.

When the original PINES software was developed, computers were immensely expensive and software comparatively primitive. Better practices and more capable small computers existed when Evergreen was envisioned, and the developers used more modern software development processes to run on more capable computers that no longer cost a million dollars each to buy. An examination of alternatives in the market found none that had the capabilities necessary to run PINES then or in the future. What was Georgia to do? Limit PINES? Give up and abandon a policy direction it had followed so consistently, for so long, and that had produced such benefits? Over software?

After much discussion, GPLS embarked on a revolutionary path—an open source, consortial library system to run PINES and to grow with it.

Development Begins

Work began on Evergreen mid-2004. The later a software system is developed, the more modern practices can be used in that development. Evergreen, for instance, was Web-aware and used Unicode for text storage and representation from the beginning. To implement these capabilities, Evergreen did not require the costly and difficult rewrites to the foundational code necessary for ILSs that had their basic code written before Unicode had been developed. Unicode is important because it allows a system to handle non-Roman scripts, including characters in Chinese and Arabic.

Technology - The Underlying Software Architecture*

Evergreen uses a service-oriented architecture (SOA) with representational state transfer (REST) and “n-tier” “multiier” architectural design concepts. Let us explore these concepts a bit.

The development philosophy behind these elements is that software is based on distributed services that communicate and collaborate through a set of standards, that is, it should be “service oriented.” In Evergreen, the open scalable request framework ((OpenSRF), pronounced ““open surf”) is such an SOA framework.

OpenSRF is, in many ways, the key to Evergreen because it does so many things. OpenSRF provides load balancing and is robust and scalable. It also allows the development of software without requiring a detailed knowledge of Evergreen's structure. In effect, the Evergreen layer consists of a number of applications each of which rides on top of OpenSRF. The result is that the developer need only know the APIs and what information the program elements require and what they will reply to write new capabilities into Evergreen. OpenSRF handles the details of implementing a stateful, decentralized service architecture while the Evergreen code supplies the interface and business logic. OpenSRF is now a separate open source project.

Networking follows a design philosophy that is often introduced to students with the OSI Reference Model. The model describes seven layers that pass information among each other to provide network services in a manner similar to what one finds with SOA. “N-tier” application architecture is a related idea where the tiers communicate and collaborate with each other by established standards. We see these operations in our daily lives. If, for example, a new version of Firefox comes out, we can change the old version for the new one without changing the operating system or reformatting our hard drives. We usually do not even have to reboot. Why? Because of the underlying architecture where each program communicates with the other parts of the network software by established rules —application programming interfaces (APIs)—that specify how the various elements involved exchange needed information. So, to change a part or one tier does not require a massive rewrite of the entire code, just what needs to be adjusted. As mentioned, the Evergreen middle layer rides on top of OpenSRF, and the user interfaces (the OPAC and staff client) ride on top of Evergreen.

In short, REST is a set of principles that establishes an architectural style for large, scalable applications.

Evergreen also uses other open source projects—there are no proprietary pieces in Evergreen—to supply needed functionality. One advantage of open source applications is that a developer working on one project can use what is already tested and proven by other projects. PostgreSQL (“Postgres”) is an enterprise-grade relational database system with over 15 years of development. It was chosen for Evergreen because it was virtually the only open source database system that had the capabilities to support a system with the database structure and the transactions load the size of PINES. Another key application is XUL (pronounced “zool”) which stands for XML user interface language. Developed by the Mozilla project, XUL allows fully featured cross-platform Web applications. Perhaps the best known XUL-driven application is the Firefox Web browser.

There are, as you see, a number of pieces to the Evergreen puzzle. These modules work together, and each can be changed when necessary without affecting the others as long as the changes do not affect the communications among them. Using these practices, coupled with the open source framework, provided a flexible method for very rapid development of Evergreen in the early days and continues to this day.

Development Philosophy

As a result of what was learned about the first iteration of PINES, there were several key results expected from Evergreen. The original charge was to build something “scalable, robust and fault-tolerant.” And, of course, open source. What do those three new terms mean?

Scalable means that the software can be deployed in different-sized libraries and that adding capacity is relatively easy. At the point where the decision was being made to move from the proprietary system that ran PINES, the server upgrade for the old system would have cost $1.5 million for a large server. Evergreen development cost the GPLS much less than that.

Learning that lesson early, Evergreen's design permits relatively easy upgrades. It can be upgraded by adding “commodity ” servers—that is, servers from any manufacturer or any do-it-yourself builder, and they can be added as needed.

Because Evergreen is easy to use, as a library's users learn the software, they use it more, and growth can be done by adding relatively cheap, redundant servers, not million dollar boxes running proprietary software. Add a server or two, load OpenSRF, and you have upgraded. Evergreen runs on laptops and runs PINES with 19 million circulation transactions and King County with 20 million.

Robust. Evergreen was designed to be able to keep functioning in dire circumstances. When a backhoe severed the PINES network, the Evergreen staff clients allowed libraries that were disconnected from the central databases to continue checking out books. When the network was restored, the database was updated by an established procedure. There was no need to write down transactions on paper. Consider: PINES has a number of days with 100,000 circulations. If half the network were down for a half a day, that could be 25,000 sheets of paper to be managed. Not with Evergreen.

Fault-tolerant. Servers can fail, and the system will keep on functioning. The use of redundant servers allows one to fail and the others to keep on functioning. Evergreen was designed as much as possible to not go down.

Communications Within the Evergreen Community

The conversations between members of the community takes place in public venues. There are various mailing lists that one can subscribe to from a general list to one for developers who are working on the code and others including discussions of documenting the Evergreen software. Another active means of communicaions take place on Internet Relay Chat (IRC) channels. Evergreen users and developers also meet at conferences, including the Evergreen International Conference which has proven to be a popular meeting place for community members and those investigating Evergreen. The last one was held in April in Decatur, Georgia and the next one will be in Indianapolis, Indiana. The conversations take place in these various places and are continous.

What now?

This introduction to Evergreen gives general information that is intended to give a non-technical overview of the Evergreen project. If you want to learn more, you can follow the various links given here or head to the main Evergreen page. There are a number of decisions that would have to be made. If your library or consortium employs a staff with technically trained staff in IT functions you might encourage them to join any of the communications media listed above and download the software and examine the documentation to help set up Evergreen. If you are investigating setting up a new Evergreen consortium, there is a great deal of experience with this kind of work in the community and you will find its members are willing to help. You might also read Things to Think About for a new Evergreen Library Consortium which has general observations gleaned from the experiences of librarians who have already done this. Pay particular attention to the discussion of governance as this has proved to be a problem with some of the new consortia. If the governance is not organized well, it may well not scale if and when your consortium starts to grow. Meanwhile, you will find that you have more choices for support and migration options than you would if you were dealing with a proprietary product. For instance, you can do it yourself—migrate your data out of your legacy system to Evergreen—as a number of libraries and consortia have done or you can contract with one (or more) of the companies offering to help you with the various aspects aspects of training staff, migrating data, and/or maintaining your Evergreen installation.

Good luck and welcome to the Evergreen community.


* This section and the next are adapted from an article in the ASIS&T Bulletin, December 2008/January 2009, “Evergreen in Context” and used with their kind permission.

scratchpad.txt · Last modified: 2012/06/15 11:10 by lebbeous

© 2008-2017 GPLS and others. Evergreen is open source software, freely licensed under GNU GPLv2 or later.
The Evergreen Project is a member of Software Freedom Conservancy.