Evergreen DokuWiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== QA Requirements for Evergreen Code Contributions ======

===== Minimum Requirements =====

As of Evergreen version 2.8, new code contributions must be submitted with test code.

==== Evergreen 2.8 ====

One of the main goals for the 2.8 series regarding QA tests is encouraging developers to engage in the process and learn how to build tests.  The only requirement is that any commits which modify code that has a functional test structure in place (e.g. database changes -- see "Types of Tests" below) must be accompanied by at least one test that in some way exercises the change.  It does not have to test every facet of the new code.  We're aiming for one step beyond "Hello, World".  

==== Evergreen 2.9 ====

As of Evergreen 2.9, the test case expectations include:

  * Any time a patch adds or alters a stored procedure, pgTAP tests that exercise its intended functionality should be included.
  * A change to database or Perl code that fixes a bug should be  accompanied by a Perl (t or live_t) or pgTAP regression test – or by a statement from the patch author explaining why a test is infeasible without significant refactoring.  In the latter case, it is expected that an extra signoff be obtained before the patch is merged.
  * Bugfix patch commit messages should explain how to test the bug it fixes. For example: rather than just write a bare "LP#124565: fix Evergreen's cat-petting functionality", provide something more like:

  LP#124565: fix Evergreen's cat-petting functionality
  
  Evergreen does not do an adequate job of petting cats.  To
  reproduce the problem:
  
  [1] Bring up an Evergreen OPAC and place a cat
  in front of it.
  [2] Observe that a hand appears and makes waving
  motions a centimeter over the cat.
  [3] Observe that the cat does not purr.
  [4] Apply the patch.
  [5] This time, verify that the hand actually makes contact
  with the cat.
  [6] Expected result: the cat purrs.



===== Types of Tests =====

==== AngularJS Unit Tests ====

These test the Web Client apps.

  * They are stored in ''Open-ILS/web/js/ui/default/staff/test/''
  * See the [[https://code.angularjs.org/1.6.10/docs/guide/unit-testing#additional-tools-for-testing-angularjs-applications|official instructions]] for details.

Running the tests:
  * See the [[https://evergreen-ils.org/documentation/install/README_3_1.html#install_files_for_web_staff_client|Evergreen README]] for up-to-date details.
  * Steps to run the tests as of Evergreen 3.1:

  cd $EVERGREEN_ROOT/Open-ILS/web/js/ui/default/staff/
  # fetch JS dependencies, described in package.json
  npm install
  # build, run tests, concat+minify
  npm run build-prod   # may not be necessary for running tests
  npm run test

==== Angular Unit Tests ====

See [[dev:angular_dev_best_practices#unit_tests|Evergreen Angular Development Best Practices]]

To run on a running Evergreen server:

<code sh>
cd Open-ILS/src/eg2
npm run test
</code>

=== Common things that break Angular unit tests ===

== Changing DOM selectors ==

If the test is expecting a certain DOM element, but you change the structure within an Angular template, the test might start failing.

Note that you can use text of a button as a selector.  If the button text changes, you'll need to update the selector.

To fix it: update the selector in the test.  Or perhaps the test can be made more general.

== Adding a new service ==

Adding a new service

To fix it: add a mock for the new service in the test.  Or even better, consider if you can simplify or refactor the component so that it does not need many different services.

== Lifecycle/timing issues ==

Interacting with anything asynchronous (especially grid and combobox) requires some care!

To fix it: You may be able to add a setTimeout.  Or use Angular's fakeAsync and tick.
==== Angular e2e (end-to-end) Tests ====

To run with a GUI:

<code sh>
cd Open-ILS/src/eg2
ng e2e
</code>

To run without a GUI:

<code sh>
cd Open-ILS/src/eg2
MOZ_HEADLESS=1 ng e2e
</code>

To run in Chrome instead of Firefox:
<code sh>
cd Open-ILS/src/eg2
npm install --save-dev chromedriver
ng e2e --env chrome # with the GUI
ng e2e --env chrome-headless # without the GUI
</code>

For more tips and documentation, visit the [[https://github.com/evergreen-library-system/Evergreen/blob/main/Open-ILS/src/eg2/CHEAT_SHEET.adoc|Angular client cheat sheet]] and the [[https://nightwatchjs.org/v26/|Nightwatch documentation]].

==== OPAC Javascript Unit Tests ====

To run tests for the javascript in the OPAC:
<code sh>
cd Open-ILS/web/opac/deps
npm i && npm run test
</code>

==== Perl Unit Tests ====

These are used for testing Perl functions which can be executed without a running Evergreen system.  

  * They are built using Perl's http://perldoc.perl.org/Test/More.html
  * Files are kept in the Evergreen repository under Open-ILS/src/perlmods/t/.
  * Tests are run with "make check" in the root of the repository after "configure" and "make" have been run.

For a simple example, see Open-ILS/src/perlmods/t/01-OpenILS-Application.t.  Of note is the following section of code which tests a Perl function in the AppUtils.pm module:

<code perl>
is(                                                                            
    OpenILS::Application::AppUtils::entityize(0, 'èöçÇÈÀ'),                    
    '&#xE8;&#xF6;&#xE7;&#xC7;&#xC8;&#xC0;',                                    
    'entityize: diacritics'                                                    
); 
</code>


==== Perl Live Tests ====

Live tests are used for testing a running Evergreen system.  They can communicate over the network, talk to the database, etc.  To execute live tests, Evergreen must be running and the "concerto" data set must be installed.  

NOTE: The "concerto" dataset can be installed when the database is built using the "--load-all-sample" option.

  * Tests are built using Perl's http://perldoc.perl.org/Test/More.html
  * Files are kept in the Evergreen repository under Open-ILS/src/perlmods/live_t/.

Running the live tests:

<code sh>
# assumes ./configure and make have already been run
cd Open-ILS/src/perlmods
make livecheck
</code>

For a simple example, see Open-ILS/src/perlmods/live_t/00-simple.t.  Note how it's making API calls to the open-ils.storage service.

<code perl>
my $ses = $script->session('open-ils.storage');                                
my $req = $ses->request('open-ils.storage.direct.actor.user.retrieve', 1);     
if (my $resp = $req->recv) {                                                   
    if (my $user = $resp->content) {                                           
        is(                                                                    
            ref $user,                                                         
            'Fieldmapper::actor::user',                                        
            'open-ils.storage.direct.actor.user.retrieve returned aou object'  
        );                                    
        ...
</code>

==== pgTAP Database Tests ====

pgTAP is used to test database behavior.  These tests require access to an Evergreen database, but do not require a running Evergreen server or the concerto data set.

  * Tests are built using http://pgtap.org/
  * Files are kept in the Evergreen repository under Open-ILS/src/sql/Pg/t/

The "pgtap" package, which is not part of a standard Evergreen installation, must be installed and the extension must be added to your Evergreen database:

Install pgtap from source, the versions in Debian/Ubuntu are not well maintained and are from 2011 when they exist at all. Old versions don't contain all the functions used by Evergreen pgtap tests.

  * Download from - http://pgxn.org/dist/pgtap/
  * Install Instructions - http://pgtap.org/documentation.html#installation
  * Install the package for libtap-parser-sourcehandler-pgtap-perl which provides pg_prove.

<code sh>
apt-get install libtap-parser-sourcehandler-pgtap-perl #Debian/Ubuntu provides pg_prove
psql -U evergreen [other DB connection params] -c "CREATE EXTENSION pgtap;"
</code>

To run pgTAP tests:

<code sh>
cd Open-ILS/src/sql/Pg
pg_prove -U evergreen [other DB connection params] t t/regress
</code>

==== pgTAP Live Database Tests ====

Live pgTAP tests are built in the same manner as regular pgTAP tests, with the added requirement / assumption that the concerto data set is installed.

NOTE: The "concerto" dataset can be installed when the database is built using the "--load-all-sample" option.

  * Files are kept in the Evergreen repository under Open-ILS/src/sql/Pg/live_t/

Running live tests:

<code sh>
cd Open-ILS/src/sql/Pg
pg_prove -U evergreen [other DB connection params] live_t
</code>

==== C Unit Tests ====

Like Perl unit tests, C unit tests are used for testing code that does not require a running Evergreen system.
   
  * They are built using http://check.sourceforge.net/
  * Files live in Open-ILS/src/c-apps/tests
  * Tests are automatically executed when running "make check"

At time of writing, there is no equivalent to "live" tests for C code.

{{tag>automated-tests}}