egdeveloperoverview
Differences
This shows you the differences between two versions of the page.
egdeveloperoverview [2012/05/18 13:03] – created loriayre | egdeveloperoverview [2014/02/28 14:00] (current) – removing; duplicates eg_developer_overview gmcharlton | ||
---|---|---|---|
Line 1: | Line 1: | ||
- | Evergreen Developer Overview | ||
- | ============================ | ||
- | |||
- | Developing for Evergreen can be a challenge for those new to it. Each piece of Evergreen requires different skills, and many things will require multiple parts of the system be modified. | ||
- | |||
- | A basic overview of the system shows three distinct areas that one would develop in: | ||
- | . OpenSRF, forming the backend communication layer for Evergreen. Primarily dealt with via Perl modules, but also including C. | ||
- | . PostgreSQL Database, both in database design and stored procedures. Most stored procedures are written in SQL, PL/pgSQL, or Perl. | ||
- | . XML/ | ||
- | |||
- | OpenSRF and Services | ||
- | -------------------- | ||
- | |||
- | Evergreen is built upon the Open Scalable Request Framework, or OpenSRF (pronounced "open surf" | ||
- | |||
- | OpenSRF Basics | ||
- | ~~~~~~~~~~~~~~ | ||
- | |||
- | OpenSRF currently works by passing JSON messages over XMPP, although in the future that could change. In general the inner workings of OpenSRF are not exposed to clients and services written on top of it, so that the underlying elements can change without having to re-implement the entire system. | ||
- | |||
- | A very basic view of how a message passes through the system is as follows: | ||
- | . A request for a service is sent to the router | ||
- | . The router chooses a listener and sends the request on | ||
- | . The listener passes the message onto a drone | ||
- | . The drone acts on the message and sends the response back to the original requester | ||
- | |||
- | Low level OpenSRF development may be needed to support new programming languages, message passing improvements, | ||
- | |||
- | OpenSRF Clients | ||
- | ~~~~~~~~~~~~~~~ | ||
- | |||
- | The Staff Client, OPAC, and some other parts of Evergreen are technically OpenSRF clients. The Staff Client and JSPac talk through the HTTP Translator, and the TPac is a Perl client embedded in the HTTP server. | ||
- | |||
- | Perl clients intended for use via HTTP protocols tend to be implemented in the OpenILS:: | ||
- | |||
- | In addition, OpenSRF Applications are clients as well, able to make requests to other running applications. | ||
- | |||
- | OpenSRF Applications | ||
- | ~~~~~~~~~~~~~~~~~~~~ | ||
- | |||
- | Most OpenSRF development will be in the OpenSRF Application area. Most OpenSRF and Evergreen services are written in Perl, but a few are written in C for performance reasons. | ||
- | |||
- | The list below is an overview of each application by the service name it is assigned. Included is the language the application is implemented in and which (if any) of the Public and XML-RPC interfaces it is exposed via. Public exposed services are available via the HTTP gateways. | ||
- | |||
- | opensrf.settings (Perl):: | ||
- | Handles loading, parsing, and distributing configuration file information. This can include things like database connection information. | ||
- | opensrf.math (C, Public, XML-RPC):: | ||
- | Primarily for testing use, this service accepts requests and passes them on to the opensrf.dbmath service. Success demonstrates that not only can requests be sent and answers received but that services can communicate with each other. | ||
- | opensrf.dbmath (C, XML-RPC):: | ||
- | Primarily for testing use, this service does the actual processing for the opensrf.math service. | ||
- | opensrf.validator (Perl):: | ||
- | Implements a set of configurable validator chains for various data, such as email addresses. | ||
- | open-ils.acq (Perl, Public):: | ||
- | The Evergreen Acquisitions module. | ||
- | open-ils.actor (Perl, Public, XML-RPC):: | ||
- | Public and Staff Client access to user and org unit information. | ||
- | open-ils.auth (C, Public, XML-RPC):: | ||
- | Authentication service. Implemented in C for performance reasons, handles logins and authentication token validation. | ||
- | open-ils.booking (Perl, Public):: | ||
- | The Evergreen Reservations module. | ||
- | open-ils.cat (Perl, Public, XML-RPC):: | ||
- | The Evergreen Cataloging module, handling creation and editing of bibliographic and asset information. | ||
- | open-ils.circ (Perl, Public, XML-RPC):: | ||
- | The Evergreen Circulation module, handling checkins and checkouts, as well as Hold related information. | ||
- | open-ils.collections (Perl, Public, XML-RPC):: | ||
- | The Evergreen Collections module. | ||
- | open-ils.cstore (C):: | ||
- | CStore is a database search and retrieval service, and the default target for the CStoreEditor Perl class. It does not check or require authentication tokens. It can be used to perform complicated queries via json_query calls. | ||
- | open-ils.fielder (Perl, Public):: | ||
- | Currently unused(?) service for loading FieldMapper information. | ||
- | open-ils.ingest (Perl):: | ||
- | Currently unused pre-indb Ingest module. | ||
- | open-ils.justintime (Perl, XML-RPC):: | ||
- | Action/ | ||
- | open-ils.pcrud (C, Public):: | ||
- | PCrud is a database search and retrieval service that checks and requires authentication tokens to control access to information. Unlike CStore it cannot " | ||
- | open-ils.penalty (Perl):: | ||
- | A wrapper around the OpenILS:: | ||
- | open-ils.permacrud (Perl, Public):: | ||
- | A Perl authentication checking wrapper around CStore calls. | ||
- | open-ils.reporter (Perl, Public):: | ||
- | The Staff creation and query service for the reports system. | ||
- | open-ils.reporter-store (C):: | ||
- | Effectively CStore with a different name, so as to support a different set of IDL tables. | ||
- | open-ils.resolver (Perl, Public):: | ||
- | OpenURL holdings retrieval service. Note: Disabled by default. | ||
- | open-ils.search (Perl, Public, XML-RPC):: | ||
- | Searching related functions, including Catalog and Z39.50. | ||
- | open-ils.serial (Perl):: | ||
- | Serials-related tasks such as receiving items and generating predictions. | ||
- | open-ils.storage (Perl):: | ||
- | Direct database calls for various actions, intended in part to support databases other than PostgreSQL at some unspecified point in the future that will likely never come. Similar to a Perl-based CStore, but with more utility functions included. | ||
- | open-ils.supercat (Perl, Public):: | ||
- | Additional search and browse routines for various catalog elements, such as authorities, | ||
- | open-ils.trigger (Perl):: | ||
- | The Action/ | ||
- | open-ils.vandelay (Perl, Public):: | ||
- | The bib import/ | ||
- | |||
- | PostgreSQL | ||
- | ---------- | ||
- | |||
- | Evergreen uses PostgreSQL as a storage backend and makes somewhat heavy use of many PostgreSQL features. PostgreSQL databases are split into schemas, allowing for logical groupings of related information. | ||
- | |||
- | In addition to PostgreSQL itself one should also be aware of the Fieldmapper IDL, which is one of the ways Evergreen is told how to interact with the database. | ||
- | |||
- | Schemas | ||
- | ~~~~~~~ | ||
- | |||
- | Many of the schemas are obviously named, and as such do not need much description. Others cover things that are more abstract. Below is a list of all schemas currently used by Evergreen. | ||
- | |||
- | public:: | ||
- | This is a default PostgreSQL schema. No tables should be created here, although some generic functions are. New generic functions should be created in the evergreen schema, however. | ||
- | acq:: | ||
- | This is where Acquisitions specific data and functions live. | ||
- | action:: | ||
- | The action schema is used for storing data on things that have and/or are going to happen, such as Circulations, | ||
- | action_trigger:: | ||
- | This is where all Action/ | ||
- | actor:: | ||
- | Actor covers Org Units and Users and information about them. | ||
- | asset:: | ||
- | Asset includes call numbers (volumes) and copy information. In addition, URI information is stored here as they are linked to call numbers. | ||
- | auditor:: | ||
- | The auditor schema contains non-acquisitions auditing information. Beyond functions for creating auditors nothing should be manually created here. Instead call the create_auditor function to create the auditor tables, views, and triggers. | ||
- | authority:: | ||
- | The authority schema covers all authority related information and functions. | ||
- | biblio:: | ||
- | The biblio schema is for bib records and related items. Note that most search and other indexing actually happens in the metabib schema. | ||
- | booking:: | ||
- | The booking schema covers the reservation system' | ||
- | config:: | ||
- | Most (but not all) configuration related tables live in the config schema. The tables are commonly used as lookup tables for values stored elsewhere. | ||
- | container:: | ||
- | The container schema is the storage location for buckets. Note that OPAC Bookbags are a type of biblio record entry bucket. | ||
- | evergreen:: | ||
- | A generic schema mainly for holding generic functions. Ideally tables and views should be created elsewhere. | ||
- | extend_reporter:: | ||
- | For the most part the extend_reporter schema is intended for local table, function, and view additions for reporting purposes. | ||
- | metabib:: | ||
- | The metabib schema holds pre-parsed metadata about bib records, including search information, | ||
- | money:: | ||
- | The money schema contains billing and payment information for billable actions throughout the system. For note, the action.circulation and booking.reservation tables inherit the billable_xact table contained within it. | ||
- | offline:: | ||
- | Evergreen contains a built-in offline mode. The offline schema handles information about processing of the offline circulations. | ||
- | permission:: | ||
- | Permission covers all of the permissions and permission groups in the system. Note that user profile groups are actually permission groups. | ||
- | query:: | ||
- | The query schema houses data for the Query Store (qstore) service. | ||
- | reporter:: | ||
- | Evergreen' | ||
- | search:: | ||
- | The search schema' | ||
- | serial:: | ||
- | Information on serials (like magazines) is stored in the serial schema. Note that serial.unit is inherited from asset.copy. | ||
- | staging:: | ||
- | The staging schema is used to stage potential new users, complete with addresses, cards, and statistical category entries. | ||
- | stats:: | ||
- | The stats schema is created alongside the config schema, but is currently unused. | ||
- | unapi:: | ||
- | Information and functions pertaining to unapi output is stored here. | ||
- | vandelay:: | ||
- | The vandelay bib import/ | ||
- | |||
- | Fieldmapper IDL | ||
- | ~~~~~~~~~~~~~~~ | ||
- | |||
- | The Fieldmapper IDL, or just IDL for short, is what most of Evergreen uses as a basis for what exists in the database. The main thing one needs to be aware of is the fm_IDL.xml file, the contents of which are parsed for the information on various tables, views, and other information in the database. | ||
- | |||
- | The IDL is comprised of a set of class nodes, each of which contains a list of fields at a minimum. In addition to a list of fields they may also contain links, permissions, | ||
- | |||
- | The important thing to know about the IDL is that, for the most part, clients and services can't talk to database elements not defined within it. The notable exception to that is stored procedures, which are not listed in the IDL. | ||
- | |||
- | OPAC | ||
- | ---- | ||
- | |||
- | The OPAC is the primary interface for locating materials in the system and placing holds on those materials for staff and patrons alike. There are, however, two OPAC interfaces in Evergreen right now. The classic JavaScript based AJAX OPAC, somewhat commonly referred to at this point as the JSPac, is the older and more commonly used OPAC. The newer Perl Template Toolkit based OPAC is set to replace it, and is commonly referred to as TPac or TTOpac. | ||
- | |||
- | JSPac | ||
- | ~~~~~ | ||
- | |||
- | To start with, at this point it is recommended that new features not be developed for the JSPac unless they involve security or limiting patron actions. Otherwise development should be targeted at TPac. | ||
- | |||
- | The JSPac is built on the JavaScript OpenSRF interfaces and builds almost everything outside of the base page layouts client side. Search results and account information are populated after the initial page load as the system makes backend calls with the AJAX technologies the browser supports. Development is thus more oriented towards JavaScript programming and XHTML pages than anything else. This includes use of the Dojo Toolkit, though for a number of reasons Evergreen is currently limited to the somewhat outdated Dojo version 1.3. | ||
- | |||
- | Though JSPac supports multiple skins only one is currently shipped with Evergreen. | ||
- | |||
- | TPac | ||
- | ~~~~ | ||
- | |||
- | TPac is the location new features should ideally be implemented, | ||
- | |||
- | TPac development thus comes in two flavors: | ||
- | . Perl " | ||
- | . Template Toolkit template code in the Templates directory for formatting and display | ||
- | |||
- | Staff Client | ||
- | ------------ | ||
- | |||
- | The Evergreen Staff Client is currently a XULRunner application built on the XULRunner 3 series, which is equivalent to the Firefox 3 series. One goal is to eventually move to a later version of XULRunner, another is to create a new staff client built on other technologies. | ||
- | |||
- | The staff client is a combination of local and remote XUL pages, JavaScript, and the OPAC itself. It is compatible with both the JSPac and the TPac, and also loads a number of interfaces that work like the JSPac and/or TPac. Some of these interfaces can be easily used outside of the staff client itself, others expect to be loaded from within it. | ||
- | |||
- | Development for the Staff Client can thus involve XUL, JavaScript, (X)HTML, and OPAC development. Perhaps all at the same time, depending on your goals. | ||
- | |||
- | External Code - SIPServer | ||
- | ------------------------- | ||
- | |||
- | In addition to the core OpenSRF and Evergreen code there are other projects commonly used with Evergreen that may require work for development projects. The most common is currently SIPServer, which provides a SIP2 interface for Evergreen that can be used with Self Check units, Automated Materials Handlers, PC Reservation systems, and some third party services for authentication. | ||
- | |||
- | SIPServer is maintained as a separate project, is written in Perl, and interacts with Evergreen through the OpenILS:: | ||
- | |||
- | Community Interaction | ||
- | --------------------- | ||
- | |||
- | In addition to the information listed in the contributing document at http:// | ||
- | |||
- | First and foremost, stay visible and communicate so that others watching your work know you are making progress. If you don't, and are hard to contact, your project may be deemed " | ||
- | |||
- | As part of that, if you are having issues ask questions, be it on IRC or via the mailing lists. IRC is most useful during the day in the United States/ | ||
- | |||
- | If you have opened a Launchpad bug keep it up to date. In addition, excepting security issues, you should prefer the public channels over direct communication with individuals. Not only does that ensure that others know you are still there and working on the issue, but you also get the benefit of the community' | ||
- | |||
- | As a note on communication via Launchpad, when responding to things on Launchpad via email sometimes Launchpad " | ||
egdeveloperoverview.1337360604.txt.gz · Last modified: 2022/02/10 13:34 (external edit)