Differences

This shows you the differences between two versions of the page.

--- dev:testing:performance_draft_rfp [2013/03/12 08:01] – created klussier
+++ dev:testing:performance_draft_rfp [2022/02/10 13:34] (current) – external edit 127.0.0.1
@@ Line 6: / Line 6: @@
 Evergreen is an open-source integrated library system developed by the Georgia Public Library Service to support 252 public libraries in the Public Information Network for Electronic Services (PINES) consortium. PINES completely transitioned to Evergreen in September 2006. Evergreen is now used by approximately 1,300 libraries around the world.  It offers the following functionality for libraries:
-   * a metadata search engine;
-   * a transaction processing engine;
-   * a web application;
-   * and a fund management system.
-MassLNC is a collaborative project among three Massachusetts library consortia that are using separate instances of Evergreen. Although MassLNC is the issuing entity for this RFP and will be the contracting entity and primary contact, when conducting this evaluation, the consultant(s) is expected to work directly with the open-source community supporting Evergreen.
+  * a metadata search engine (e.g. author, title, subject);
-The budget for the proposed work is _____ to _____.
+  * a transaction processing engine (e.g. checkin, checkout, place request);
+  * a web application;
+  * a fund management system.
+MassLNC is a collaborative project among three Massachusetts library consortia that are using separate instances of Evergreen. Central/Western Massachusetts Automated Resource Sharing, Inc. (C/W MARS), as fiscal agent for MassLNC, is the issuing entity for this RFP and will be the contracting agency. MassLNC will be the primary contact for coordinating the project and for making final decisions. When conducting this evaluation, the consultant(s) is expected to work directly with the open-source community supporting Evergreen.
+The budget for the proposed work is $50,000.
 ====Project Description====
+Evergreen is a highly-scalable system used by many large library consortia as well as individual libraries. The size of the libraries and consortia using Evergreen varies. Appendix A provides data from one particularly large Evergreen system, C/W MARS, based in Worcester, Massachusetts.
+MassLNC and others in the Evergreen community have identified 1) places in the system where performance is suboptimal and 2) questions they hope can be answered through a performance evaluation. The list is available on the Evergreen wiki at:
-Evergreen is a highly-scalable system used by many large library consortia as well as individual libraries. The size of the libraries and consortia using Evergreen varies. Appendix A provides data from one particularly large Evergreen system, the Central/Western Massachusetts Regional Library System (C/W MARS) based in Worcester, Massachusetts.
-MassLNC and others in the community have identified places in the system where performance is suboptimal and questions they hope can be answered through a performance evaluation. The list is available on the Evergreen wiki at:
 [[dev:testing:performance_issues|http://evergreen-ils.org/dokuwiki/doku.php?id=dev:testing:performance_issues]]
-The goal of the performance evaluation is to do a thorough audit of the Evergreen infrastructure to identify any bottlenecks that negatively impact performance in these areas and to work with the community to recommend solutions.  It is not expected that the performance consultant will implement these solutions. Any recommended solutions will be left in the hands of the Evergreen community to implement.
+===Goals===
+The goals for the performance evaluation are as follows:
+  - to work with the community to plan for the software performance evaluation, including, but not limited to, the prioritization of areas that will be evaluated.
+  - to conduct a performance audit of the Evergreen infrastructure to identify bottlenecks that negatively impact performance.
+  - to work with the community to identify solutions and their impact.  It is not expected that the performance consultant will implement solutions to address performance issues. Any recommended solutions will be left in the hands of the Evergreen community to implement.
 A performance evaluation is required in the three following target areas. A more detailed overview of each of these areas can be found at [[:eg_developer_overview|http://evergreen-ils.org/dokuwiki/doku.php?id=eg_developer_overview]].
-===Staff Client===
+==Staff Client==
 The Evergreen Staff Client is currently a XULRunner application built on the XULRunner platform. XULRunner versions are roughly equivalent to Firefox versions for rendering purposes. The staff client is a combination of local and remote XUL pages, JavaScript, and the Online Public Catalog (OPAC). This includes use of the Dojo Toolkit, version 1.3 (released in March 2009). The OPAC is processed server-side with the Template Toolkit Perl Module.
+The Evergreen Staff Client is currently an application built on the XULRunner platform. XULRunner versions are roughly equivalent to Firefox versions for rendering purposes. The staff client is a combination of local and remote XUL pages, JavaScript, and the Online Public Catalog (OPAC). This includes use of the Dojo Toolkit, version 1.3 (released in March 2009). The OPAC is processed server-side with the Template Toolkit Perl Module.
 **Specific Issues/Questions Related to the Staff Client**
-The staff client has exhibited problems with memory leaks, a problem that has gotten worse at some sites with the release of Evergreen 2.3 in September 2012. Work has already begun to address these problems (see [[https://bugs.launchpad.net/evergreen/+bug/1086458]]). There have also been suggestions that an upgrade of the Dojo toolkit or the total elimination of Dojo from the staff client may resolve memory leaks and/or improve staff client performance.
+  * As a first step, the developer community has identified the move of staff client interfaces away from remote XULRunner in improving staff client performance.
+  * The staff client has exhibited problems with memory leaks, a problem that has gotten worse at some sites with the release of Evergreen 2.3 in September 2012. Work has already begun to address these problems (see https://bugs.launchpad.net/evergreen/+bug/1086458). There have also been suggestions that an upgrade of the Dojo toolkit or the total elimination of Dojo from the staff client may resolve memory leaks and/or improve staff client performance. Would an upgrade of the Dojo toolkit or the total elimination of Dojo from the staff client may resolve memory leaks and/or improve staff client performance?
+  * Would a move of staff client interfaces away from XULRunner improve staff client performance?
+  * The staff client currently sends and receives individual XMLHTTPRequests. Are there any performance advantages to moving to an architecture that can support streaming requests?
-The developer community has also identified the move of staff client interfaces away from remote XULRunner as a first step in improving staff client performance.
 ===Messaging===
@@ Line 52: / Line 65: @@
 Evergreen uses PostegreSQL, both in database design and stored procedures. Most stored procedures are written in SQL, PL/pgSQL, or Perl. PostgreSQL databases are split into schemas, allowing for logical groupings of related information.
-In addition to PostgreSQL itself, one should also be aware of the Fieldmapper IDL, which is one of the ways Evergreen is told how to interact with the database. Fieldmapper maps database fields to Evergreen objects.
+In addition to PostgreSQL itself, one should also be aware of the Fieldmapper Interface Definition Language (IDL), which is one of the ways Evergreen is told how to interact with the database. Fieldmapper maps database fields to Evergreen objects.
 The Fieldmapper IDL, or just IDL for short, is the Fieldmapper configuration file (fm_IDL.xml) which tells Evergreen about all of the important aspects of the database. It is comprised of a set of class nodes, each of which contains at minimum a list of fields. In addition, each class may contain links, permissions, and in the case of fully virtual classes a block of SQL that defines the data to be retrieved.
-An important note: For the most part, clients and services can only talk to database elements that are defined within the IDL. The notable exception is stored procedures, which are not listed in the IDL.
+**An important note:** For the most part, clients and services can only talk to database elements that are defined within the IDL. The notable exception is stored procedures, which are not listed in the IDL.
 **Specific Issues/Questions Related to the Database**
@@ Line 62: / Line 76: @@
 In general, are there database queries that can be optimized in the areas where performance has been identified as a problem?
-In past versions of Evergreen, sites were able to utilize settings in search.relevance_adjustment to improve the relevancy of results retrieves through the Online Public Access Catalog (OPAC). As of Evergreen 2.0, many sites deactivated the settings in this table because searches became too slow (see https://bugs.launchpad.net/evergreen/+bug/844374). Is there a way to optimize searching so that tools like these can be re-implemented?
+End users expect OPAC searches to be fast. Are there ways to optimize the searching of library materials through the OPAC?. Related to OPAC searching, in past versions of Evergreen, sites were able to utilize settings stored in the database to improve the relevancy of results retrieved through the OPAC. As of Evergreen 2.0, many sites deactivated the settings in this table because searches became too slow (see https://bugs.launchpad.net/evergreen/+bug/844374). Is there a way to optimize searching so that tools like these can be re-implemented?
 ====Description of Services Required====
@@ Line 85: / Line 99: @@
 ====Appendix A – Data from the C/W MARS Evergreen System====
-/10/2013
 Central/Western Massachusetts Automated Resource Sharing, Inc. (C/W MARS) is an example of a large-scale consortium using Evergreen. The intent of the below data is to give prospective consultants an idea of the database size and level of activity that Evergreen needs to support with minimal impact on performance.
+===Database statistics as of 3/10/2013===
   * Number of Library Systems: 149
   * Number of Individual Library Branches: 170
@@ Line 93: / Line 107: @@
   * Number of Bibliographic Records (a record representing a specific edition of a title): 2,251,590
   * Number of Copy records (individual barcoded items associated with the bibliographic records): 8,597,202
+===Volume of Transactions During Busy Hours===
+   * Tuesday, March 5, 3-4 PM
+         * Checkouts: 6,204
+         * Checkins: 5,930
+         * Hold Requests: 715
+         * **Total Checkouts, Checkins, and Hold Requests: 12,849**
+   * Wednesday, March 6, 3-4 PM
+         * Checkouts: 6,224
+         * Checkins: 5,356
+         * Hold Requests: 641
+         * **Total Checkouts, Checkins and Hold Requests: 12,221**
+   * Thursday, March 7, 3-4 PM
+         * Checkouts: 5,200
+         * Checkins: 3,984
+         * Hold Requests: 631
+         * **Total Checkouts, Checkins and Hold Requests: 9,815**
+====Appendix B - Glossary====
+**Fieldmapper Interface Definition Language (IDL)** - The Fieldmapper IDL, or just IDL for short, is the Fieldmapper configuration file (fm_IDL.xml) which tells Evergreen about all of the important aspects of the database. It is comprised of a set of class nodes, each of which contains at minimum a list of fields. In addition, each class may contain links, permissions, and in the case of fully virtual classes a block of SQL that defines the data to be retrieved.
+**Integrated Library System (ILS)** - An integrated library system is a system for libraries to track items ordered and owned, bills paid, and users who have borrowed materials.
+**Online Public Access Catalog (OPAC)** - The OPAC is the web application the public uses to search the database to find materials owned by the library. In Evergreen, the OPAC is also embedded in the staff client. The OPAC within the staff client may display some information that is not seen when accessing the OPAC via a standard web browser.
+Open Scalable Request Framework (OpenSRF) - OpenSRF provides communication between various “Applications” and the clients that wish to call them. Each application provides a service, and usually the applications are referred to as services because of this. OpenSRF works by passing JSON messages over XMPP
+**Staff Client** - The staff client is the client used by library staff to perform functions like ordering library materials, adding library materials to the database, checking materials in and out to users, sharing materials with patrons at other libraries, finding library materials, managing user information, accepting payments from users, and creating/retrieving statistical reports.