Rolling an Evergreen Release

¡This is a work in progress. It will grow and change!

These recordings show various release building activities.

• May 2023 Cutting a Evergreen Release
• Nov 2023 Building 3.12, first session
• Dec 2023 Building 3.12, second session

Make sure you are in an environment with your git ssh keys, since you will need to be pushing commits to the git server.

To upload translations to Launchpad, you will also need to register an ssh key with Launchpad. See the 3.13 translation page for an example.

See Versioning for notes on the numbering scheme.

• Install the packager dependencies using the Evergreen installer makefile:

# sudo make -f Open-ILS/src/extras/Makefile.install <os>-packager
$ sudo make -f Open-ILS/src/extras/Makefile.install ubuntu-focal-packager

[Note that your Ubuntu version may be something other than 'focal'; substitute the appropriate version keyword]

• During beta release building, update relator codes and commit the change to main (if necessary). This happens before string freeze for translations, see https://bugs.launchpad.net/evergreen/+bug/1407507 for more details.

$ perl build/tools/relator_map > Open-ILS/src/templates/opac/parts/relators.tt2
$ cp Open-ILS/src/templates/opac/parts/relators.tt2 Open-ILS/src/templates-bootstrap/opac/parts/relators.tt2
# if 'git status' shows changes, commit them.
$ git commit -sm "Updating relator codes for XXX" Open-ILS/src/templates/opac/parts/relators.tt2 Open-ILS/src/templates-bootstrap/opac/parts/relators.tt2

See also: Release Notes Process for DIG Release Coordinator

Cut a new release branch (such as rel_3_13):

 git checkout -b rel_3_13 main 

Create release notes:

First, the RELEASE_NOTE_NEXT piece.

$ cd docs/RELEASE_NOTES_NEXT
$ ./create_release_notes.sh -r 3.13 

After that script runs, you can remove 3.13-specific files from docs/RELEASE_NOTES_NEXT. Running the following in the docs/RELEASE_NOTES_NEXT directory should work:

$ find . -mindepth 2 -name '*.txt' -exec git rm {} \;
$ find . -mindepth 2 -name '*.adoc' -exec git rm {} \;
$ echo "" > miscellaneous.adoc

Then, the extract release notes from the commit piece.

Next, use the `extract_release_notes_from_commits.pl` script to get the rest of the release notes.

• The script parses commits to pull the following: Release notes for bugs that use the <Release-note:> tag; Contributors; Sponsors
• See Bug #2051874 for details.

$ ../tools/extract_release_notes_from_commits.pl --help
$ ../tools/extract_release_notes_from_commits.pl --current-branch=rel_3_13_beta --prev-release-branch=main

• Integrate the results into the release notes.
• Make sure all directories in RELEASE_NOTES_NEXT are empty after the two scripts are run. If they are not, remove any lingering adoc files.
• Stage your changes and commit the release notes:

  git commit -sm "3.13 release notes"

• <fixme> in the future, combine these two processes for release notes in major versions.

For an alpha or beta release, this rough process is probably enough. For a real #.0 release, the resulting compiled release notes will need some TLC/cleanup.

Once you are at the .0 release, update line 3 of the antora.yml configuration file in your release branch so it reads 'version: 3.13' instead of 'version: latest'.

$ vi docs/antora.yml

Then, update the version in the site.yml file on main branch to include rel_3_13 on line 9.

$ vi docs/site.yml

This needs to be done once during the release cycle, typically at alpha or beta

Set up the new series branch using the instructions here: Setting Up a New Series Translation

During each release starting with beta, get the latest template changes and apply them to the main branch you are working from in origin (main/rel_3_13/etc.). It is important to merge any new pot changes to origin so that Launchpad translations will get the latest strings that require translation.

• Update pot in the Evergreen directory as a non-root user

  $ cd build/i18n && make newpot

• From the Evergreen directory, run the python script to commit any changes that do more than modify metadata

  python3 build/i18n/scripts/add_translations.py -c # use -h option for help

Export strings for the Angular staff client and import them into POEditor, following the POEditor instructions. You will need to be a poeditor.com admin to do this.

Create a local working branch for the release. This will contain changes applicable to only this release, and become the release "tag":

$ git checkout -b tags/rel_3_13_1 rel_3_13

• Next, clone (or update) the translations repository in a new directory as a peer to Evergreen. This pulls in translations from Launchpad:

$ cd ../; 
# Clone the repository if necessary
$ bzr branch lp:~evergreen-bugs/evergreen/translation-export-main # for main branch
$ bzr branch lp:~evergreen-bugs/evergreen/translation-export-3.12 # for 3.12.x, e.g.
# If already cloned, simply pull in the latest changes
$ cd translation-export-SERIES; bzr pull; cd ../

• NOTE: See here to find the correct translation branch for the version you are building: https://code.launchpad.net/evergreen/+branches
• It will give you an error "You have not informed bzr of your Launchpad ID, and you must do this to write to Launchpad or access private data." You can safely ignore this, since you will just need to read public data for the next step – no writing or private data needed.

• Then, sync translated files with Evergreen. Update PO files in the appropriate branch in origin:

# series here is "main" or "3.10" or whatever corresponding to the release
# that you are building
$ Evergreen/build/i18n/scripts/update_pofiles -l translation-export-SERIES -e Evergreen
$ cd Evergreen
$ git add -A build # pick up updates
$ git commit -sm "Translation updates - po files"

1. Export (as xtb) any translations from the POEditor site that have over 50% translations available.
2. Save them to Open-ILS/src/eg2/src/locale/ and commit using the following steps:

$ cp ~/download/path/messages.*.xtb ./Open-ILS/src/eg2/src/locale/
$ git add -A Open-ILS/src/eg2/src/locale/
$ git commit -asm "Translation updates - xtb files"

• Replace release version placeholders (X.X.X) with the actual version numbers in docs/modules/installation/pages/server_upgrade.adoc
  • In vim, :%s/3.X.X/3.13.0/ will do the trick
• If you are creating a beta or rc, add a hyphen and then beta or rc. For example, 3.13-rc.
• Replace the previous version placeholder (e.g. 3.X.W) with the last release number.
  • In vim, :%s/3.X.W/3.13.0/ will do the trick
• Double check the correct minimum version of OpenSRF.
• If the minimum version of OpenSRF has changed, also update the OpenSRF version number in docs/modules/installation/pages/server_installation.adoc

Run the release builder script:

$ export PATH=$PATH:/openils/bin
$ build/tools/make_release -v 3.13-beta -f origin/tags/rel_3_12_3

Additional notes on make_release options:

• If you are building beta, use -beta for the first version value, for example: 3.13-beta.
• Use the highest numbered previous version for the second version value.
  • For example, building 3.13-beta, the script should start from the highest 3.12 release at the time of branching:

    $ build/tools/make_release -v 3.13-beta -f origin/tags/rel_3_12_3

  • For a .0 release, the script should start from the highest point release in the previous series:

    $ build/tools/make_release -v 3.13.0 -f origin/tags/rel_3_12_3

  • For a point release, the script should start from the previous point release in the series:

    $ build/tools/make_release -v 3.13.1 -f origin/tags/rel_3_13_0

• Use the -v flag when your branch name does not match the normal pattern, otherwise auto-detect works
• Use the -j flag to point at wherever your OpenSRF javascript lib source is, if not already installed
  • -j ~/Git/OpenSRF/src/javascript/
• The -f flag sets the branch "from" which this release this being made from, aka what was the last version of Evergreen prior to the one you're making. This guides the make_release towards creating a version-upgrade from X to Y.
• You can add the -r option to edit the upgrade script before it goes into the tarball. This may be necessary if tables that get altered in the upgrade scripts also have updates or inserts of new data.
• Enhanced Concerto Dataset upgrade *experimental
  • This is optional, but if enough of the database tables have changed, it's a good idea.
  • -H <postgres server hostname>
  • -U <postgres username> # needs SUPERUSER and CREATEDB
  • -P <database password>
  • -O <database port>
  • Omitting -H will cause make_release to skip the enhanced concerto upgrade steps.

Note that you will get a lot of warnings related to the po translation files. These can be safely ignored.

From the Evergreen directory:

# copy README and release notes to ../release/
$ export RELEASE=3_12
$ cp README ../release/README_$RELEASE
$ cp docs/RELEASE_NOTES_$RELEASE.adoc ../release/
$ cd ../release/
# in ../release/
$ asciidoc -a toc -a numbered README_$RELEASE
$ asciidoc -a toc -a numbered RELEASE_NOTES_$RELEASE.adoc
# delete the originals
$ rm README_$RELEASE RELEASE_NOTES_$RELEASE.adoc

Move the release files into the correct download directories on the evergreen-ils.org web server (which is also known as lupin):

• Previews / alphas / betas / release candidates go in /var/www/open-ils.org/downloads/previews/
• Final releases and ChangeLog-* go in /var/www/open-ils.org/downloads/
• Install docs / README_* go in /var/www/open-ils.org/documentation/install/
• Release notes go in /var/www/open-ils.org/documentation/release/

Because the release notes document is cumulative for each release series, a symbolic link exists for each release series so that (e.g.) RELEASE_NOTES_3_666.html points to the latest release in the series (e.g., RELEASE_NOTES_3_666.html → RELEASE_NOTES_3_666_7.html). Therefore, when uploading the release notes, update that symbolic link. When uploading the first alpha or beta for a series, it will be necessary to create the symbolic link.

When updating the downloads page to the website, the URL should use the symbolic link, not the release notes for the specific release, e.g., https://evergreen-ils.org/documentation/release/RELEASE_NOTES_3_666.html, not https://evergreen-ils.org/documentation/release/RELEASE_NOTES_3_666_7.html.

git push origin tags/rel_3_13_0:tags/rel_3_13_0

If doing a final release, you will need to copy the upgrade script you created to main and your current rel_x_y branch. If you skip copying to the rel_x_y branch in particular, your next point release will be missing an upgrade script in the middle of the chain!

For example, if you just created a release 3.12.6 and were on the branch tags/rel_3_12_6:

git checkout rel_3_12
git checkout tags/rel_3_12_6 -- Open-ILS/src/sql/Pg/version-upgrade/3.12.5-3.12.6-upgrade-db.sql
git commit -sm "Forward-port 3.12.5-3.12.6 upgrade script"
git checkout main
git cherry-pick rel_3_12

• Update evergreen_history.txt file in history repo with release dates.
• Release team / outreach committee make announcements following publicity checklist