Differences

This shows you the differences between two versions of the page.

--- dev:release_process:evergreen:how_to_build [2024/02/02 13:52] – [Generate release docs] sandbergja
+++ dev:release_process:evergreen:how_to_build [2025/02/10 16:53] (current) – [Helpful videos] aneiman
@@ Line 1: / Line 1: @@
-===== Rolling an Evergreen Release =====
+====== Rolling an Evergreen Release ======
 **¡This is a work in progress.  It will grow and change!**
+==== Helpful videos ====
+These recordings show various release building activities.
+  * May 2023 [[https://www.youtube.com/watch?v=mdEhjFKMFTI|Cutting a Evergreen Release]]
+  * Nov 2023 [[https://drive.google.com/file/d/1G0niyLAh55_mkgJVGn1GLFGHl_zEltcJ/view|Building 3.12, first session]]
+  * Dec 2023 [[https://princeton.zoom.us/rec/play/zDAyi7Y852mQys4Y7txxJz8fpLooM9zbz3iosP9Ex-ulHmlQ0iSWXfe-5vB-9ke_WRT-HbMYAOsHvoEO.YiWoiCK3_85oGQxf?canPlayFromShare=true&from=share_recording_detail&continueMode=true&componentName=rec-play&originRequestUrl=https%3A%2F%2Fprinceton.zoom.us%2Frec%2Fshare%2Fwrmq5l5oEMGQyzXFJiXoww0NlAK3cH8sH4amQOrE9jO0hsSjZfcl2GGzTz9y9aL4.LybQWetrF2AEXvje|Building 3.12, second session]]
+  * Feb 2025 [[https://youtu.be/PV1R8o8GFoI?si=ZLBRfKFWqOPgz4CV|3.15 Build Q&A]]
 ==== Prerequisites ====
 Make sure you are in an environment with your git ssh keys, since you will need to be pushing commits to the git server.
+Make sure that you have OpenSRF installed in your environment.
+To upload translations to Launchpad, you will also need to register an ssh key with Launchpad. See the [[https://code.launchpad.net/~evergreen-bugs/evergreen/translation-export-3.13|3.13 translation page for an example]].
+See [[::versioning|Versioning]] for notes on the numbering scheme.
 ==== Install packager dependencies ====
@@ Line 14: / Line 28: @@
 <code bash>
 # sudo make -f Open-ILS/src/extras/Makefile.install <os>-packager
-$ sudo make -f Open-ILS/src/extras/Makefile.install ubuntu-trusty-packager
+$ sudo make -f Open-ILS/src/extras/Makefile.install ubuntu-jammy-packager
 </code>
-[Note that your Ubuntu version may be something other than 'trusty'; substitute the appropriate version keyword]
+[Note that your Ubuntu version may be something other than 'jammy', or you may be using a different OS altogether. Substitute the appropriate version keyword -- if you don't know your Ubuntu version off the top of your head, you can check the /etc/lsb-release file]
+==== Confirm git credentials and username/email ====
+The following steps will require you to commit your work and push to a remote git repository.  If you are using a different environment than you usually use (for example, a fresh Docker container or new VM), make sure the following are correct.
+If you are using a docker dev container, You can commit from inside the container, but push from the host machine -- if you choose this approach, you should make sure the first two commands work from within the container, and the last two commands work from your host machine.
+^ Setting                                                                                                                      ^ How to check it                                                                                                                                                                  ^ How to fix it                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
+| Your user name                                                                                                               | Run the command: ''git config user.name'' and confirm that your name displays                                                                                                    | Run the command: ''git config --global user.name "My Name"''                                                                                                                                                                                                                                                                                                                                                                                                                        |
+| Your email address                                                                                                           | Run the command: ''git config user.email'' and confirm that your email displays                                                                                                  | Run the command: ''git config --global user.email me@example.com''                                                                                                                                                                                                                                                                                                                                                                                                                  |
+| The git remote you intend to push to (the Evergreen repo if you are a committer, the working/Evergreen repo if you are not)  | Run the command: ''git remote -v''.  Confirm that it includes ''git@git.evergreen-ils.org:Evergreen.git''  or ''git@git.evergreen-ils.org:working/Evergreen.git'' as a push url  | Run a command like: ''git remote add working git@git.evergreen-ils.org:working/Evergreen.git''                                                                                                                                                                                                                                                                                                                                                                                      |
+| Your SSH key to connect to the git remote                                                                                    | Run the command ''git ls-remote''.  Confirm that it does not ask for ''git@git.evergreen-ils.org's password''                                                                    | Securely copy or mount your private SSH key into the environment you will be using to push.  For example, if you are creating a docker container for your work, you can add the flag ''-v ~/.ssh:/home/opensrf/.ssh'' to your ''docker run'' command to mount your .ssh directory.  If you copy your private key to the VM or container, be sure to remove it or destroy the VM/container when you are done, since this key could allow somebody else to impersonate you.  |
 ==== Update relator codes (requires commit bit) ====
@@ Line 33: / Line 58: @@
 ==== Prepare release notes (for major version bumps only) ====
-=> See also the [[evergreen-docs:release_notes_process|Release Notes Process for DIG Release Coordinator]]
+See also: [[evergreen-docs:release_notes_process|Release Notes Process for DIG Release Coordinator]]
-  * Cut a new release branch (such as rel_3_7):
+Cut a new release branch (such as rel_3_13):
-<code>
-git checkout -b rel_3_7 main
-</code>
-  * Update the version in the antora configuration
+<code> git checkout -b rel_3_13 main </code>
-<code bash>
+Create release notes:
-$ vi docs/antora.yml
-</code>
-edit 'latest' -> '3.7'
+First, the RELEASE_NOTE_NEXT piece.
-  * Create release notes
+<code bash>$ cd docs/RELEASE_NOTES_NEXT
+$ ./create_release_notes.sh -r 3.13 </code>
-<code bash>
+After that script runs, you can remove 3.13-specific files from docs/RELEASE_NOTES_NEXT. Running the following in the docs/RELEASE_NOTES_NEXT directory should work:
-$ cd docs/RELEASE_NOTES_NEXT
-$ ./create_release_notes.sh -r 2_8
-</code>
-  * Remove 2.8-specific files from docs/RELEASE_NOTES_NEXT.
-    * Running the following in the docs/RELEASE_NOTES_NEXT directory should work:
 <code bash>
 $ find . -mindepth 2 -name '*.txt' -exec git rm {} \;
 $ find . -mindepth 2 -name '*.adoc' -exec git rm {} \;
+$ echo "" > miscellaneous.adoc
 </code>
-  * Stage your changes and commit the release notes: `git commit -sm "3.7 release notes"`
+Then, the extract release notes from the commit piece.
+Next, use the `extract_release_notes_from_commits.pl` script to get the rest of the release notes.
+  * The script parses commits to pull the following: Release notes for bugs that use the <Release-note:> tag; Contributors; Sponsors
+  * See [[https://bugs.launchpad.net/evergreen/+bug/2051874|Bug #2051874]] for details.
+<code bash>
+$ ../tools/extract_release_notes_from_commits.pl --help
+$ ../tools/extract_release_notes_from_commits.pl --current-branch=rel_3_13_beta --prev-release-branch=main
+</code>
+  * Integrate the results into the release notes.
+  * Make sure all directories in RELEASE_NOTES_NEXT are empty after the two scripts are run. If they are not, remove any lingering adoc files.
+  * Stage your changes and commit the release notes: <code>git commit -sm "3.13 release notes"</code>
+  * <fixme> in the future, combine these two processes for release notes in major versions.
 For an alpha or beta release, this rough process is probably enough.  For a real #.0 release, the resulting compiled release notes will need some TLC/cleanup.
+Once you are at the .0 release, update line 3 of the antora.yml configuration file in your release branch so it reads 'version: 3.13' instead of 'version: latest'.
+<code bash>$ vi docs/antora.yml</code>
+Then, update the version in the site.yml file on main branch to include rel_3_13 on line 9.
+<code bash>$ vi docs/site.yml</code>
+==== Translations, Part 0: New Series Branch ====
+This needs to be done once during the release cycle, typically at alpha or beta
+Set up the new series branch using the instructions here: [[i18n:setting_up_a_new_series_translation|Setting Up a New Series Translation]]
 ==== Translations, Part 1: newpot (requires commit bit) ====
-  * During each release starting with beta, get the latest template changes and apply them to the main branch you are working from in origin (main/rel_3_1/etc.).  It is important to merge any new pot changes to origin so that Launchpad translations will get the latest strings that require translation.
+During each release starting with beta, get the latest template changes and apply them to the main branch you are working from in origin (main/rel_3_13/etc.).  It is important to merge any new pot changes to origin so that Launchpad translations will get the latest strings that require translation.
-    * Update pot in Evergreen <code bash>$ cd build/i18n && make newpot</code>
-    * From the Evergreen directory, run the python script to commit any changes that do more than modify metadata <code bash>python3 build/i18n/scripts/add_translations.py -c # use -h option for help</code>
+  * Update pot in the Evergreen directory as a non-root user <code bash>$ cd build/i18n && make newpot</code>
+  * From the Evergreen directory, run the python script to commit any changes that do more than modify metadata <code bash>python3 build/i18n/scripts/add_translations.py -c # use -h option for help</code>
 ==== Translations, Part 1a: Angular Staff Client Updates (requires a POEditor account) ====
-    * Export strings for the Angular staff client and import them into POEditor, following these [[:poeditor|instructions]].  You will need to be a poeditor.com admin to do this.
+Export strings for the Angular staff client and import them into POEditor, following the [[:poeditor|POEditor instructions]].  You will need to be a poeditor.com admin to do this.
 ==== Setting up your git branch ====
-  * Create a local working branch for the release.  This will contain changes applicable to only this release, and become the release "tag":
+Create a local working branch for the release.  This will contain changes applicable to only this release, and become the release "tag":
 <code bash>$ git checkout -b tags/rel_3_13_1 rel_3_13</code>
@@ Line 91: / Line 137: @@
 # Clone the repository if necessary
 $ bzr branch lp:~evergreen-bugs/evergreen/translation-export-main # for main branch
-$ bzr branch lp:~evergreen-bugs/evergreen/translation-export-3.10 # for 3.10.x, e.g.
+$ bzr branch lp:~evergreen-bugs/evergreen/translation-export-3.12 # for 3.12.x, e.g.
 # If already cloned, simply pull in the latest changes
 $ cd translation-export-SERIES; bzr pull; cd ../
@@ Line 97: / Line 143: @@
   * NOTE: See here to find the correct translation branch for the version you are building: https://code.launchpad.net/evergreen/+branches
+  * It will give you an error "You have not informed bzr of your Launchpad ID, and you must do this to write to Launchpad or access private data."  You can safely ignore this, since you will just need to read public data for the next step -- no writing or private data needed.
   * Then, sync translated files with Evergreen. Update PO files in the appropriate branch in origin:
 <code bash>
-# series here is "main" or "3.10" or whatever corresponding to the release
+# series here is "main" or "3.13" or whatever corresponding to the release
 # that you are building
 $ Evergreen/build/i18n/scripts/update_pofiles -l translation-export-SERIES -e Evergreen
@@ Line 116: / Line 163: @@
 $ git add -A Open-ILS/src/eg2/src/locale/
 $ git commit -asm "Translation updates - xtb files"
-</code>
-==== Other version number changes ====
-  * Edit ./Open-ILS/src/perlmods/lib/OpenILS.pm and set (e.g., for 3.1.1):
-<code perl>
-our $VERSION = '3.0101';
-</code>
-  * NOTE: The version string is two digits for each level after the decimal, so for example "3.0.5" would become "3.0005".  "3.12.0" would be "3.1200".  If it is a beta or release candidate, do not include "beta" or "rc", just use the numeric version.
-  * Commit the version change
-<code bash>
-$ git commit -asm "bumping Perl version string for 2.8"
 </code>
 ==== Update server upgrade documentation ====
-  * Replace release version placeholders with the actual version numbers in docs/modules/installation/pages/server_upgrade.adoc (X.X.X and X_X_X versions)
+  * Replace release version placeholders (X.X.X) with the actual version numbers in docs/modules/installation/pages/server_upgrade.adoc
-  * If you are creating a beta or rc, add a hyphen and then beta or rc.  For example, 3.12-rc.
+    * In vim, '':%s/3.X.X/3.13.0/'' will do the trick
+  * If you are creating a beta or rc, add a hyphen and then beta or rc.  For example, 3.13-rc.
   * Replace the previous version placeholder (e.g. 3.X.W) with the last release number.
+    * In vim, '':%s/3.X.W/3.13.0/'' will do the trick
   * Double check the correct minimum version of OpenSRF.
   * If the minimum version of OpenSRF has changed, also update the OpenSRF version number in docs/modules/installation/pages/server_installation.adoc
@@ Line 143: / Line 177: @@
 ==== Run the release builder script ====
-  * Run the release builder script
+Run the release builder script:
 <code bash>
 $ export PATH=$PATH:/openils/bin
-$ build/tools/make_release -v 2.8.0 -f origin/tags/rel_2_7_3
+$ build/tools/make_release -v 3.13-beta -f origin/tags/rel_3_12_3
 </code>
-  * Additional notes on make_release options:
+Additional notes on make_release options:
-    * Use the -v flag when your branch name does not match the normal pattern, otherwise auto-detect works
+  * If you are building beta, use -beta for the first version value, for example: 3.13-beta.
-    * Use the -j flag to point at wherever your OpenSRF javascript lib source is, if not already installed
+  * Use the highest numbered previous version for the second version value.
-      * -j ~/Git/OpenSRF/src/javascript/
+    * For example, building 3.13-beta, the script should start from the highest 3.12 release at the time of branching: <code bash>$ build/tools/make_release -v 3.13-beta -f origin/tags/rel_3_12_3</code>
-    * The -f flag sets the branch "from" which this release this being made from, aka what was the last version of Evergreen prior to the one you're making. This guides the make_release towards creating a version-upgrade from X to Y.
+    * For a .0 release, the script should start from the highest point release in the previous series: <code bash>$ build/tools/make_release -v 3.13.0 -f origin/tags/rel_3_12_3</code>
-    * You can add the -r option to edit the upgrade script before it goes into the tarball. This may be necessary if tables that get altered in the upgrade scripts also have updates or inserts of new data.
+    * For a point release, the script should start from the previous point release in the series: <code bash>$ build/tools/make_release -v 3.13.1 -f origin/tags/rel_3_13_0</code>
-    * Enhanced Concerto Dataset upgrade *experimental
+  * Use the -v flag when your branch name does not match the normal pattern, otherwise auto-detect works
-      * This is optional, but if enough of the database tables have changed, it's a good idea.
+  * Use the -j flag to point at wherever your OpenSRF javascript lib source is, if not already installed
-      * -H <postgres server hostname>
+    * -j ~/Git/OpenSRF/src/javascript/
-      * -U <postgres username> # needs SUPERUSER and CREATEDB
+  * The -f flag sets the branch "from" which this release this being made from, aka what was the last version of Evergreen prior to the one you're making. This guides the make_release towards creating a version-upgrade from X to Y.
-      * -P <database password>
+  * You can add the -r option to edit the upgrade script before it goes into the tarball. This may be necessary if tables that get altered in the upgrade scripts also have updates or inserts of new data.
-      * -O <database port>
+  * Enhanced Concerto Dataset upgrade *experimental
-      * Omitting -H will cause make_release to skip the enhanced concerto upgrade steps.
+    * This is optional, but if enough of the database tables have changed, it's a good idea.
+    * -H <postgres server hostname>
+    * -U <postgres username> # needs SUPERUSER and CREATEDB
+    * -P <database password>
+    * -O <database port>
+    * Omitting -H will cause make_release to skip the enhanced concerto upgrade steps.
 Note that you will get a lot of warnings related to the po translation files.  These can be safely ignored.
@@ Line 171: / Line 210: @@
 <code bash>
-# copy README and docs/RELEASE_NOTE_2_x.adoc to ../release/
+# copy README and release notes to ../release/
-$ cp README ../release/README_2_8
+$ export RELEASE=3_12_0 # Increment for later release, 3_12_1 for example.
-$ cp docs/RELEASE_NOTES_2_8.adoc ../release/
+$ cp README ../release/README_$RELEASE
+$ cp docs/RELEASE_NOTES_$RELEASE.adoc ../release/
 $ cd ../release/
 # in ../release/
-$ asciidoc -a toc -a numbered README_2_8
+$ asciidoc -a toc -a numbered README_$RELEASE
-$ asciidoc -a toc -a numbered RELEASE_NOTES_2_8.adoc
+$ asciidoc -a toc -a numbered RELEASE_NOTES_$RELEASE.adoc
 # delete the originals
-$ rm README_2_8
+$ rm README_$RELEASE RELEASE_NOTES_$RELEASE.adoc
-$ rm RELEASE_NOTES_2_8.adoc
 </code>
-==== Upload to web server (requires lupin access) ====
+==== Upload to web server (requires access to the evergreen-ils.org web server) ====
-Move the release files into the correct download directories:
+Move the release files into the correct download directories on the evergreen-ils.org web server (which is also known as lupin):
     * Previews / alphas / betas / release candidates go in ''/var/www/open-ils.org/downloads/previews/''
-    * Final releases and ChangeLog-* go in ''/var/www/open-ils.org/downloads/''
+    * Final releases go in ''/var/www/open-ils.org/downloads/''
     * Install docs / README_* go in ''/var/www/open-ils.org/documentation/install/''
     * Release notes go in ''/var/www/open-ils.org/documentation/release/''
+Because the release notes document is cumulative for each release series, a symbolic link exists for each release series so that (e.g.) ''RELEASE_NOTES_3_666.html'' points to the latest release in the series (e.g., ''RELEASE_NOTES_3_666.html'' → ''RELEASE_NOTES_3_666_7.html''). Therefore, when uploading the release notes, update that symbolic link. When uploading the first alpha or beta for a series, it will be necessary to create the symbolic link.
+When updating the downloads page to the website, the URL should use the symbolic link, not the release notes for the specific release, e.g., ''https://evergreen-ils.org/documentation/release/RELEASE_NOTES_3_666.html'', not ''https://evergreen-ils.org/documentation/release/RELEASE_NOTES_3_666_7.html''.
 ==== "Tag" Working Branch (requires commit bit) ====
 <code bash>
-git push origin rel_2_8_0:tags/rel_2_8_0
+git push origin tags/rel_3_13_0:tags/rel_3_13_0
 </code>
@@ Line 203: / Line 245: @@
 If doing a final release, you will need to copy the upgrade script you created to main and your current rel_x_y branch.  If you skip copying to the rel_x_y branch in particular, your next point release will be missing an upgrade script in the middle of the chain!
+For example, if you just created a release 3.12.6 and were on the branch tags/rel_3_12_6:
+<code bash>
+git checkout rel_3_12
+git checkout tags/rel_3_12_6 -- Open-ILS/src/sql/Pg/version-upgrade/3.12.5-3.12.6-upgrade-db.sql
+git commit -sm "Forward-port 3.12.5-3.12.6 upgrade script"
+git checkout main
+git cherry-pick rel_3_12
+</code>
 ==== Announcing and recording the release ====
     * Update evergreen_history.txt file in history repo with release dates.
     * Release team / outreach committee make announcements following [[https://docs.google.com/document/d/1FpCvnAgL0TiHL3L-lNWbEd7FLBzVs6UV0L1kIoN70Jw/edit?usp=sharing|publicity checklist]]