User Tools

Site Tools


dev:sqitch

Using Sqitch with Evergreen

Getting Started

Install Sqitch

# Works on most systems.  See sqitch.org for more.
$ sudo cpan App::Sqitch

Install Base Schema

Install the latest working branch from LP 1521693 using the standard DB install process. The branch is set up to use Sqitch internally. No special considerations are required for the base install.

Creating an Upgrade Script

For this example, we're cross-porting an existing SQL upgrade script for LP #1319998.

Full code is tracked in commit c999d04.

Override Sqitch DB connections parameters the sqitch flags –db-host, –db-name, –db-port, and –db-user. The PGPASSWORD environment variable is also honored.

$ cd Open-ILS/src/sql/schema
$ sqitch add schema.materialized_billing_summary_delete_trigger \
    --requires schema.no_delete_acq_cancel_reasons \
    -n 'Cross porting LP#1319998 schema.materialized_billing_summary_delete_trigger' 
  • The –requires flag specifies which existing target our new target depends on. For consistency with existing SQL upgrades, I apply all changes sequentially, so the –requires value points to the previous Sqitch install target. Or, in other words, the last/bottom target in the sqitch.plan file.
  • -n specifies a Sqitch "commit" message which appears in the Sqitch log output.

'sqitch add' adds a line to sqitch.plan to track the new install target and creates 3 new files:

deploy/schema.materialized_billing_summary_delete_trigger.sql
revert/schema.materialized_billing_summary_delete_trigger.sql
verify/schema.materialized_billing_summary_delete_trigger.sql
  • The deploy/ file includes our new change. The structure of the file is practically identical to our existing upgrade/XXXX.foo.sql files, except they do not include the inline dependency check (evergreen.upgrade_deps_block_check(XXXX)).
  • The revert/ file includes SQL that allows us to roll back the changes in the deploy script.
    • For example, when deploying a modified function, as is the case with this example, the revert/ file will include the original (well, previous) function definition.
  • The verify/ file contains SQL to confirm the changes in the deploy/ file were properly deployed.

For example, see the contents of these 3 files at commit c999d04.

Deploying an Upgrade Script

  • Sanity check the status first
$ sqitch status
# On database evergreen
# Project:  evergreen
# Change:   a632d6b112c3770ad8bd3efff80b9f5700212166
# Name:     schema.no_delete_acq_cancel_reasons
# Deployed: 2015-12-29 10:28:02 -0500
# By:       Bill Erickson <berickxx@gmail.com>
# 
Undeployed change:
  * schema.materialized_billing_summary_delete_trigger
  • Deploy the change
$ sqitch deploy
Deploying changes to evergreen
  + schema.materialized_billing_summary_delete_trigger .. ok

* If necessary, revert the change.

  • This is particularly useful during development and testing.
  • –to represents the last target that will still be deployed after the revert runs. In other words, it's the point in time you want to return to.
  • In this case, I want to return to the target that occurred just before my new target/upgrade.
$ sqitch revert --to schema.no_delete_acq_cancel_reasons
Revert changes to schema.no_delete_acq_cancel_reasons from evergreen? [Yes] 
  - schema.materialized_billing_summary_delete_trigger .. ok

Resolving Conflicts / Deploying Multiple Plan Files

AKA "Cannot find this change in sqitch.plan"

When deploying changes, Sqitch inspects both the name and position of each target in the sqitch.plan file. A Sqitch install target is a product of its name and which target precedes it. (This is similar to Git commits). I had originally assumed only the name of the change mattered, so this was unexpected.

In an environment where changes are always deployed linearly, like a production database, this has no bearing. However, when using Sqitch on a development database, where changes from different Git branches may be deployed in a single database, this can cause confusion for Sqitch. When the sqitch.plan file does not match the events tracked in the Sqitch database tables, a sqitch deploy may result in an error like this:

# On database evergreen
# Project:  evergreen
# Change:   e29fe26ba146d263e3494cc07960ecf2b26b620d
# Name:     schema.fix_vandelay_stored_procs
# Deployed: 2016-03-30 10:16:33 -0400
# By:       Bill Erickson <foo@example.org>
#
Cannot find this change in sqitch.plan
Make sure you are connected to the proper database for this project.

The solution is to revert any Sqitch targets that are not shared by both branches. The offending targets can be completely rolled back using

sqitch revert --to <last-shared-change>

Or they can be virtually rolled back by telling Sqitch to pretend like they were reverted using

sqitch revert --log-only --to <last-shared-change>

The –log-only option tells Sqitch to update its tracking tables without executing any deploy/verify/revert scripts. This is particularly useful when bouncing between branches where you may not want to revert changes, because it means deleting test data.

Example

  1. Deploy sqitch.plan with changes A, B, and C.
  2. Checkout another Git branch with a sqitch.plan file containing changes A, B, D.
  3. 'sqitch deploy' will result in error "Cannot find this change in sqitch.plan" with Name: C
  4. Revert all offending changes:
# roll back the change.
sqitch revert --to B

# OR just tell Sqitch to ignore the change.
sqitch revert --to B --log-only
  1. Deploy change D
sqitch deploy

The same steps will work when returning to the original branch w/ changes A, B, and C.

The --merge option

Version 1.1.0 of Sqitch will support a –merge option to better handle situations where the plan file does not match the exact order of the events tracked in the Sqitch database. (At time of writing, CPAN installs version 0.9994). In theory, this will make it easier to use a single database for developing across multiple branches.

dev/sqitch.txt · Last modified: 2016/03/30 11:17 by erickson

© 2008-2017 GPLS and others. Evergreen is open source software, freely licensed under GNU GPLv2 or later.
The Evergreen Project is a member of Software Freedom Conservancy.