====== Rolling an Evergreen Release ======

**¡This is a work in progress.  It will grow and change!**

==== Helpful videos ====

These recordings show various release building activities.

  * May 2023 [[https://www.youtube.com/watch?v=mdEhjFKMFTI|Cutting a Evergreen Release]]
  * Nov 2023 [[https://drive.google.com/file/d/1G0niyLAh55_mkgJVGn1GLFGHl_zEltcJ/view|Building 3.12, first session]]
  * Dec 2023 [[https://princeton.zoom.us/rec/play/zDAyi7Y852mQys4Y7txxJz8fpLooM9zbz3iosP9Ex-ulHmlQ0iSWXfe-5vB-9ke_WRT-HbMYAOsHvoEO.YiWoiCK3_85oGQxf?canPlayFromShare=true&from=share_recording_detail&continueMode=true&componentName=rec-play&originRequestUrl=https%3A%2F%2Fprinceton.zoom.us%2Frec%2Fshare%2Fwrmq5l5oEMGQyzXFJiXoww0NlAK3cH8sH4amQOrE9jO0hsSjZfcl2GGzTz9y9aL4.LybQWetrF2AEXvje|Building 3.12, second session]]

==== Prerequisites ====

Make sure you are in an environment with your git ssh keys, since you will need to be pushing commits to the git server.

See [[::versioning|Versioning]] for notes on the numbering scheme.

==== Install packager dependencies ====

  * Install the **packager** dependencies using the Evergreen installer makefile:

<code bash>
# sudo make -f Open-ILS/src/extras/Makefile.install <os>-packager
$ sudo make -f Open-ILS/src/extras/Makefile.install ubuntu-focal-packager
</code>

[Note that your Ubuntu version may be something other than 'focal'; substitute the appropriate version keyword]

==== Update relator codes (requires commit bit) ====

  * During **beta** release building, update relator codes and commit the change to main (if necessary).  This happens before string freeze for translations, see https://bugs.launchpad.net/evergreen/+bug/1407507 for more details.

<code bash>
$ perl build/tools/relator_map > Open-ILS/src/templates/opac/parts/relators.tt2
$ cp Open-ILS/src/templates/opac/parts/relators.tt2 Open-ILS/src/templates-bootstrap/opac/parts/relators.tt2
# if 'git status' shows changes, commit them.
$ git commit -sm "Updating relator codes for XXX" Open-ILS/src/templates/opac/parts/relators.tt2 Open-ILS/src/templates-bootstrap/opac/parts/relators.tt2
</code>


==== Prepare release notes (for major version bumps only) ====

See also: [[evergreen-docs:release_notes_process|Release Notes Process for DIG Release Coordinator]]

Cut a new release branch (such as rel_3_13):

<code> git checkout -b rel_3_13 main </code>

Create release notes:

First, the RELEASE_NOTE_NEXT piece.

<code bash>$ cd docs/RELEASE_NOTES_NEXT
$ ./create_release_notes.sh -r 3.13 </code>

After that script runs, you can remove 3.13-specific files from docs/RELEASE_NOTES_NEXT. Running the following in the docs/RELEASE_NOTES_NEXT directory should work:

<code bash>
$ find . -mindepth 2 -name '*.txt' -exec git rm {} \;
$ find . -mindepth 2 -name '*.adoc' -exec git rm {} \;
$ echo "" > miscellaneous.adoc
</code>

Then, the extract release notes from the commit piece.

Next, use the `extract_release_notes_from_commits.pl` script to get the rest of the release notes.

  * The script parses commits to pull the following: Release notes for bugs that use the <Release-note:> tag; Contributors; Sponsors
  * See [[https://bugs.launchpad.net/evergreen/+bug/2051874|Bug #2051874]] for details.

<code bash>
$ ../tools/extract_release_notes_from_commits.pl --help
$ ../tools/extract_release_notes_from_commits.pl --current-branch=rel_3_13_beta --prev-release-branch=main
</code>

  * Integrate the results into the release notes. 
  * Make sure all directories in RELEASE_NOTES_NEXT are empty after the two scripts are run. If they are not, remove any lingering adoc files.
  * Stage your changes and commit the release notes: <code>git commit -sm "3.13 release notes"</code>
  * <fixme> in the future, combine these two processes for release notes in major versions.

For an alpha or beta release, this rough process is probably enough.  For a real #.0 release, the resulting compiled release notes will need some TLC/cleanup.

Once you are at the .0 release, update line 3 of the antora.yml configuration file in your release branch so it reads 'version: 3.13' instead of 'version: latest'.

<code bash>$ vi docs/antora.yml</code>

Then, update the version in the site.yml file on main branch to include rel_3_13 on line 9.

<code bash>$ vi docs/site.yml</code>

==== Translations, Part 0: New Series Branch ====

This needs to be done once during the release cycle, typically at alpha or beta

Set up the new series branch using the instructions here: [[i18n:setting_up_a_new_series_translation|Setting Up a New Series Translation]]

==== Translations, Part 1: newpot (requires commit bit) ====

During each release starting with beta, get the latest template changes and apply them to the main branch you are working from in origin (main/rel_3_13/etc.).  It is important to merge any new pot changes to origin so that Launchpad translations will get the latest strings that require translation.

  * Update pot in Evergreen <code bash>$ cd build/i18n && make newpot</code>
  * From the Evergreen directory, run the python script to commit any changes that do more than modify metadata <code bash>python3 build/i18n/scripts/add_translations.py -c # use -h option for help</code>

==== Translations, Part 1a: Angular Staff Client Updates (requires a POEditor account) ====

Export strings for the Angular staff client and import them into POEditor, following the [[:poeditor|POEditor instructions]].  You will need to be a poeditor.com admin to do this.

==== Setting up your git branch ====

Create a local working branch for the release.  This will contain changes applicable to only this release, and become the release "tag": 

<code bash>$ git checkout -b tags/rel_3_13_1 rel_3_13</code>

==== Translations, Part 2: update_pofiles ====

  * Next, clone (or update) the translations repository in a new directory as a peer to Evergreen. This pulls in translations from Launchpad:

<code bash>
$ cd ../; 
# Clone the repository if necessary
$ bzr branch lp:~evergreen-bugs/evergreen/translation-export-main # for main branch
$ bzr branch lp:~evergreen-bugs/evergreen/translation-export-3.12 # for 3.12.x, e.g.
# If already cloned, simply pull in the latest changes
$ cd translation-export-SERIES; bzr pull; cd ../
</code>

  * NOTE: See here to find the correct translation branch for the version you are building: https://code.launchpad.net/evergreen/+branches

  * Then, sync translated files with Evergreen. Update PO files in the appropriate branch in origin:

<code bash>
# series here is "main" or "3.10" or whatever corresponding to the release
# that you are building
$ Evergreen/build/i18n/scripts/update_pofiles -l translation-export-SERIES -e Evergreen
$ cd Evergreen
$ git add -A build # pick up updates
$ git commit -sm "Translation updates - po files"
</code>

==== Translations, Part 2a: Update Angular Staff Client Translations (requires a POEditor account) ====
  - [[https://wiki.evergreen-ils.org/doku.php?id=poeditor|Export (as xtb) any translations]] from the POEditor site that have over 50% translations available.
  - Save them to ''Open-ILS/src/eg2/src/locale/'' and commit using the following steps:
<code bash>
$ cp ~/download/path/messages.*.xtb ./Open-ILS/src/eg2/src/locale/
$ git add -A Open-ILS/src/eg2/src/locale/
$ git commit -asm "Translation updates - xtb files"
</code>

==== Other version number changes ====

  * Edit ./Open-ILS/src/perlmods/lib/OpenILS.pm and set (e.g., for 3.13.0):
<code perl>
our $VERSION = '3.1300';
</code>

  * NOTE: The version string is two digits for each level after the decimal, so for example "3.0.5" would become "3.0005".  "3.12.4" would be "3.1204".  If it is a beta or release candidate, do not include "beta" or "rc", just use the numeric version. 

  * Commit the version change

<code bash>
$ git commit -asm "bumping Perl version string for 3.13"
</code>

==== Update server upgrade documentation ====

  * Replace release version placeholders (X.X.X) with the actual version numbers in docs/modules/installation/pages/server_upgrade.adoc
    * In vim, '':%s/3.X.X/3.13.0/'' will do the trick
  * If you are creating a beta or rc, add a hyphen and then beta or rc.  For example, 3.13-rc.
  * Replace the previous version placeholder (e.g. 3.X.W) with the last release number.
    * In vim, '':%s/3.X.W/3.13.0/'' will do the trick
  * Double check the correct minimum version of OpenSRF.
  * If the minimum version of OpenSRF has changed, also update the OpenSRF version number in docs/modules/installation/pages/server_installation.adoc

==== Run the release builder script ====

Run the release builder script:
<code bash>
$ export PATH=$PATH:/openils/bin
$ build/tools/make_release -v 3.13-beta -f origin/tags/rel_3_12_3
</code>

Additional notes on make_release options:
  * If you are building beta, use -beta for the first version value, for example: 3.13-beta.
  * Use the highest numbered previous version for the second version value.
    * For example, building 3.13-beta, the script should start from the highest 3.12 release at the time of branching: <code bash>$ build/tools/make_release -v 3.13-beta -f origin/tags/rel_3_12_3</code>
    * For a .0 release, the script should start from (probably) the release candidate: <code bash>$ build/tools/make_release -v 3.13.0 -f origin/tags/rel_3_13_rc</code>
    * For a point release, the script should start from the previous point release in the series: <code bash>$ build/tools/make_release -v 3.13.1 -f origin/tags/rel_3_13_0</code>
  * Use the -v flag when your branch name does not match the normal pattern, otherwise auto-detect works
  * Use the -j flag to point at wherever your OpenSRF javascript lib source is, if not already installed
    * -j ~/Git/OpenSRF/src/javascript/
  * The -f flag sets the branch "from" which this release this being made from, aka what was the last version of Evergreen prior to the one you're making. This guides the make_release towards creating a version-upgrade from X to Y. 
  * You can add the -r option to edit the upgrade script before it goes into the tarball. This may be necessary if tables that get altered in the upgrade scripts also have updates or inserts of new data.
  * Enhanced Concerto Dataset upgrade *experimental
    * This is optional, but if enough of the database tables have changed, it's a good idea.
    * -H <postgres server hostname>
    * -U <postgres username> # needs SUPERUSER and CREATEDB
    * -P <database password>
    * -O <database port>
    * Omitting -H will cause make_release to skip the enhanced concerto upgrade steps.

Note that you will get a lot of warnings related to the po translation files.  These can be safely ignored.


==== Generate release docs ====

From the Evergreen directory:

<code bash>
# copy README and release notes to ../release/
$ export RELEASE=3_12
$ cp README ../release/README_$RELEASE
$ cp docs/RELEASE_NOTES_$RELEASE.adoc ../release/
$ cd ../release/
# in ../release/
$ asciidoc -a toc -a numbered README_$RELEASE
$ asciidoc -a toc -a numbered RELEASE_NOTES_$RELEASE.adoc
# delete the originals
$ rm README_$RELEASE RELEASE_NOTES_$RELEASE.adoc
</code>



==== Upload to web server (requires access to the evergreen-ils.org web server) ====

Move the release files into the correct download directories on the evergreen-ils.org web server (which is also known as lupin):
    * Previews / alphas / betas / release candidates go in ''/var/www/open-ils.org/downloads/previews/'' 
    * Final releases and ChangeLog-* go in ''/var/www/open-ils.org/downloads/''
    * Install docs / README_* go in ''/var/www/open-ils.org/documentation/install/''
    * Release notes go in ''/var/www/open-ils.org/documentation/release/''

Because the release notes document is cumulative for each release series, a symbolic link exists for each release series so that (e.g.) ''RELEASE_NOTES_3_666.html'' points to the latest release in the series (e.g., ''RELEASE_NOTES_3_666.html'' → ''RELEASE_NOTES_3_666_7.html''). Therefore, when uploading the release notes, update that symbolic link. When uploading the first alpha or beta for a series, it will be necessary to create the symbolic link.

When updating the downloads page to the website, the URL should use the symbolic link, not the release notes for the specific release, e.g., ''https://evergreen-ils.org/documentation/release/RELEASE_NOTES_3_666.html'', not ''https://evergreen-ils.org/documentation/release/RELEASE_NOTES_3_666_7.html''.
==== "Tag" Working Branch (requires commit bit) ====

<code bash>
git push origin tags/rel_3_13_0:tags/rel_3_13_0
</code>

==== "Forward-port" Version Upgrade Script (and, optionally, po files updates) (requires commit bit) ====

If doing a final release, you will need to copy the upgrade script you created to main and your current rel_x_y branch.  If you skip copying to the rel_x_y branch in particular, your next point release will be missing an upgrade script in the middle of the chain!

==== Announcing and recording the release ====
    * Update evergreen_history.txt file in history repo with release dates.
    * Release team / outreach committee make announcements following [[https://docs.google.com/document/d/1FpCvnAgL0TiHL3L-lNWbEd7FLBzVs6UV0L1kIoN70Jw/edit?usp=sharing|publicity checklist]]