Table of Contents

Evergreen Developer Overview

Developing for Evergreen can be a challenge for those new to it. Each piece of Evergreen requires different skills, and many things will require multiple parts of the system be modified.

A basic overview of the system shows three distinct areas that one would develop in:

  1. OpenSRF, forming the backend communication layer for Evergreen. Primarily dealt with via Perl modules, but also including C.
  2. PostgreSQL Database, both in database design and stored procedures. Most stored procedures are written in SQL, PL/pgSQL, or Perl.
  3. Template Toolkit/JavaScript/Dojo/AngularJS, for OPAC and Staff Client development. The staff client is a browser-based client (written mostly in AngularJS) beginning with Evergreen 3.0 (Fall 2017). The former XUL client will be removed in Evergreen 3.2 (Fall 2018).

See also the Evergreen Technical Diagram.

OpenSRF and Services

Evergreen is built upon the Open Scalable Request Framework, or OpenSRF (pronounced "open surf"). OpenSRF provides communication between various "Applications" and the clients that wish to call them. Each application provides a service, and usually the applications are referred to as services because of this.

OpenSRF Basics

OpenSRF currently works by passing JSON messages over XMPP, although in the future that could change. In general the inner workings of OpenSRF are not exposed to clients and services written on top of it, so that the underlying elements can change without having to re-implement the entire system.

A very basic view of how a message passes through the system is as follows:

  1. A request for a service is sent to the router
  2. The router chooses a listener and sends the request on
  3. The listener passes the message onto a drone
  4. The drone acts on the message and sends the response back to the original requester

Low level OpenSRF development may be needed to support new programming languages, message passing improvements, or in the extreme a new underlying communication framework. In that regard, OpenSRF's router and HTTP gateways are written in C, and well maintained API code for clients and servers exists for C, Perl, and Python. In addition, for use of the gateways JavaScript client code is available.

OpenSRF Clients

The Staff Client, OPAC, and some other parts of Evergreen are technically OpenSRF clients. The Staff Client and JSPac talk through the HTTP Translator, and the TPac is a Perl client embedded in the HTTP server.

Perl clients intended for use via HTTP protocols tend to be implemented in the OpenILS::WWW namespace.

In addition, OpenSRF Applications are clients as well, able to make requests to other running applications.

OpenSRF Applications

Most OpenSRF development will be in the OpenSRF Application area. Most OpenSRF and Evergreen services are written in Perl, but a few are written in C for performance reasons.

The list below is an overview of each application by the service name it is assigned. Included is the language the application is implemented in and which (if any) of the Public and XML-RPC interfaces it is exposed via. Public exposed services are available via the HTTP gateways.

PostgreSQL

Evergreen uses PostgreSQL as a storage backend and makes somewhat heavy use of many PostgreSQL features. PostgreSQL databases are split into schemas, allowing for logical groupings of related information. For configuration recommendations, see Optimizing the performance of your Evergreen server.

In addition to PostgreSQL itself one should also be aware of the Fieldmapper IDL, which is one of the ways Evergreen is told how to interact with the database.

Schemas

Many of the schemas are obviously named, and as such do not need much description. Others cover things that are more abstract. Below is a list of all schemas currently used by Evergreen.

Fieldmapper IDL

Fieldmapper maps database fields to Evergreen objects.

The Fieldmapper IDL, or just IDL for short, is the Fieldmapper configuration file (fm_IDL.xml) which tells Evergreen about all of the important aspects of the database. It is comprised of a set of class nodes, each of which contains at minimum a list of fields. In addition, each class may contain links, permissions, and in the case of fully virtual classes a block of SQL that defines the data to be retrieved.

An important note: For the most part, clients and services can only talk to database elements that are defined within the IDL. The notable exception is stored procedures, which are not listed in the IDL.

One of the benefits of using Fieldmapper is that it generates object code for each of the objects defined in the IDL, which allows us to create those objects from within Evergreen and use them to access their respective database tables and views. Also, for classes in the IDL with a controller of "open-ils.cstore", the CStoreEditor creates CRUD methods (to Create, Retrieve, Update, and Delete) and a search method (e.g. $editor->search_biblio_record_entry(), or $editor->update_actor_user()).

OPAC

The OPAC is the primary interface for locating materials in the system and placing holds on those materials for staff and patrons alike. There are, however, two OPAC interfaces in Evergreen right now. The classic JavaScript based AJAX OPAC, somewhat commonly referred to at this point as the JSPac, is the older and more commonly used OPAC. The newer Perl Template Toolkit based OPAC is set to replace it, and is commonly referred to as TPac or TTOpac.

TPAC

TPAC is the recommended catalog interface beginning with Evergreen 2.6(?), and the only included catalog interface as of Evergreen 2.9. One of the goals is to use as little JavaScript as possible. All pages are processed server-side with the Template Toolkit Perl Module. This provides a distinct split between the processing and display layers that is not present in the JSPAC: Perl code builds an environment that is handed to the template engine, which then builds the resulting page.

TPAC development thus comes in two flavors:

  1. Perl "backend" code in the OpenILS::WWW modules for processing and data retrieval
  2. Template Toolkit template code in the Templates directory for formatting and display

JSPAC (no longer included as of Evergreen 2.9)

The JSPAC was built on the JavaScript OpenSRF interfaces and built almost everything outside of the base page layouts client side. Search results and account information were populated via AJAX after the initial page load. This included use of the Dojo Toolkit, though for a number of reasons Evergreen is currently limited to the somewhat outdated Dojo version 1.3.

The JSPAC supported multiple skins, but only one shipped with Evergreen.

Staff Client

The Evergreen browser-based (or "web") staff client is now the main client, and the XUL client is not supported starting with Evergreen version 3.2 (Fall 2018). Because of this, Evergreen 3.1 has an extended support window for sites that are not ready to switch to the web client.

We are currently converting all the old Dojo interfaces into Angular (version 7+). After that, we will re-write the AngularJS interfaces as well. (See angjs_to_ang_migration for the proposed timeline.) Development for the web Staff Client can involve Angular (using TypeScript), AngularJS, JavaScript, (X)HTML, and OPAC development. Perhaps all at the same time, depending on your goals.

History

Circulation features were released as a preview with Evergreen 2.8(?), and the full browser client was released with Evergreen 3.0 in October 2017. XUL interfaces have been converted to AngularJS, while most Dojo interfaces were initially embedded in the web client as-is, with hopes to convert them to AngularJS or new Angular eventually. The old Staff Client was a XULRunner application built on the XULRunner 3 series, which is equivalent to the Firefox 3 series. It was a combination of local and remote XUL pages, JavaScript, and the OPAC itself.

External Code - SIPServer

In addition to the core OpenSRF and Evergreen code there are other projects commonly used with Evergreen that may require work for development projects. The most common is currently SIPServer, which provides a SIP2 interface for Evergreen that can be used with Self Check units, Automated Materials Handlers, PC Reservation systems, and some third party services for authentication.

SIPServer is maintained as a separate project, is written in Perl, and interacts with Evergreen through the OpenILS::SIP Perl package.

Community Interaction

In addition to the information listed in the contributing document at http://evergreen-ils.org/dokuwiki/doku.php?id=contributing there are a number of things to consider.

First and foremost, stay visible and communicate so that others watching your work know you are making progress. If you don't, and are hard to contact, your project may be deemed "abandoned" and someone else may jump in. This is not only potentially frustrating for you if they release their code first, but to them if you suddenly "return" with your work. Either way someone has likely wasted effort that could have gone to other tasks.

As part of that, if you are having issues ask questions, be it on IRC or via the mailing lists. IRC is most useful during the day in the United States/Canada Eastern Time Zone (GMT-5, or -4 during DST), but the mailing lists are usable anytime (though can be slow to get responses).

If you have opened a Launchpad bug keep it up to date. In addition, excepting security issues, you should prefer the public channels over direct communication with individuals. Not only does that ensure that others know you are still there and working on the issue, but you also get the benefit of the community's knowledge (and contribute to it via archives).

As a note on communication via Launchpad, when responding to things on Launchpad via email sometimes Launchpad "eats" messages, grabbing the wrong part of the message for one reason or another, usually signature-related. Thus it is recommended that you double-check your first few email responses to ensure that they made it through to the bug in question.