Sustained.py

A Python query builder and schema migration tool, inspired by Objection.js

API reference

Command line reference

The sustained console script installs with the package. It also runs as python -m sustained.

sustained <command> [--config MODULE] [command options]

Every command imports a config module from the current directory, sustained_config by default. Pass --config MODULE to name a different module.

Guide: Schema and Migrations.

Commands

Command Options Does
plan --json Shows the pending migrations, the problems, and the model drift.
status --json Shows every migration’s state: applied, pending, or changed.
rehearse --json Runs the pending migrations up and back down, then rolls it all back.
migrate --target ID, --no-validate, --allow-out-of-order, --unrehearsed Applies pending migrations in order.
down --steps N (default 1) or --to ID Reverts applied migrations, newest first.
validate --json Checks the tracking table against the migrations.
repair   Fixes tracking rows after failures or intentional edits.
script up or down (default up) Prints the SQL a run would execute, without running it.
baseline TARGET (required) Records migrations as applied without running them.

--steps and --to are mutually exclusive.

Exit codes

Code Means
0 Success, or nothing to do.
1 A failure: config, connection, validation problems, or a migration error. Details on stderr.
2 plan only: work is waiting.
3 plan and migrate: a guard blocked a statement.
4 migrate only: the run removes data and no rehearsal proved it.

plan uses all of these codes. It exits 0 when the database is current, 2 when migrations are pending or the models have drifted, 3 when a guard blocked a statement, and 1 when validation found problems. Problems outrank a blocked statement, and a blocked statement outranks pending work.

argparse also exits 2 on a usage error. If your script treats 2 as “work is waiting”, check stderr for an error: line first.

rehearse exits 1 when an up step or a down step failed, when the models did not land, or when the schema did not come back. A migration with no down step is not a failure, so rehearse exits 0 for it.

migrate exits 4 when the run would remove data and no passing rehearsal covers those statements. The message names the statements and both ways forward, and repeats --target when the run had one.

A block or a refusal on the migration generated from the models happens after the registered migrations have applied. Those ids print on stdout before the error, and they stay applied.

validate exits 1 when it finds problems, and 0 when it finds none. The exit codes are the same with and without --json.

The config module

Attribute Required Type Default
connection one of the two A DB-API 2.0 connection checked first
get_connection one of the two () -> Connection used when connection is absent
migrations no list[Migration] []
migrations_dir no Path to .up.sql / .down.sql / .repeat.sql files None
placeholders no dict[str, str] filling ${key} markers None
models no List of model classes None
dialect no A Dialects member, or its name: 'postgres', 'MSSQL' Dialects.DEFAULT
table no Tracking table name 'sustained_migrations'
rehearsal_table no Rehearsal table name 'sustained_rehearsals'
tracking_table_options no TableOptions None
guards no list[Guard] from sustained.guards []
get_rehearsal_connection no () -> Connection, a scratch database None
before_migrate no (connection) -> None not called
after_migrate no (connection, applied) -> None not called
on_error no (connection, migration_id, error) -> None not called

A config module that defines neither connection nor get_connection raises ValueError. An unknown dialect name raises ValueError listing the valid names.

Migrations from migrations_dir are appended after the ones in migrations, so you can use both sources together.

# sustained_config.py
import psycopg

from models import Show, Venue


def get_connection():
    return psycopg.connect('postgresql://localhost/app')


migrations_dir = 'migrations'
models = [Venue, Show]
dialect = 'postgres'

Guards

plan runs the guards over every statement it lists, the pending migrations and the drift together, and prints a guards section beside the other sections. With --json, each verdict appears on the statement object it flags, as {"rule", "verdict"}.

migrate refuses a blocked run before any statement executes and exits 3, with the rule and the statement on stderr. A warning verdict prints on stderr and the run continues. rehearse does not enforce guards.

No flag skips a guard for one run. Fix the statement, or take the rule out of the config module.

Callbacks

Only migrate calls the callbacks. rehearse does not call them, because it rolls everything back. The CLI collects the callbacks into a Callbacks object and hands that object to the migrator, and the migrator makes the calls, so the same hooks are available through the API.

before_migrate runs before the run starts, which is before validation and before the advisory lock. after_migrate runs only when at least one migration applied, so a run with nothing to do calls nothing. on_error runs after a failure and before the failure reaches the shell. Its migration_id argument is None when the run failed before it reached a migration, as it does for a guard block or a validation problem.

The CLI skips a callback that is not callable. When on_error itself raises, its error prints to stderr, and the original migration error still decides the exit code.

Rehearsal connection

When the config defines get_rehearsal_connection(), rehearse builds a second migrator on that connection and rehearses on the scratch database instead. The dialect check does not apply there, the changes may remain after the rollback, and the footer says so. The scratch connection closes when the command ends.

The rehearsal row goes on the real database rather than the scratch one, keyed against the real database’s applied history and pending set. Sustained writes the row only when the scratch run applied every migration pending on the real database. Otherwise the output says the row was not recorded.

Output

Output is plain text, one record per line, with no colour.

$ sustained status
applied  001_create_venues
pending  002_create_shows
changed  upcoming_shows
$ sustained plan
pending
  003_sessions  2 statements
  004_trim      1 statement
    destructive  ALTER TABLE users DROP COLUMN legacy
  vw_active     1 statement  repeat changed

drift
  ALTER TABLE users ADD COLUMN bio TEXT

2 pending migrations, 1 drift statement
run: sustained rehearse

The footer points at rehearse when a pending migration removes data, because migrate refuses that run without a rehearsal row. Otherwise the footer points at migrate. A blocked statement replaces the footer with blocked: fix the statement, or take the rule out of guards.

A guards section follows the other sections when the config names guards, with one line per verdict:

guards
  block  no_drops          ALTER TABLE users DROP COLUMN legacy
  warn   no_table_rewrite  ALTER TABLE users ALTER COLUMN age TYPE BIGINT

The drift section appears only when the config names models. It reports every difference, drops included, and migrate never generates a drop. A drift section of only drops says so instead of offering the command. The run: line prints only when validation found no problems.

$ sustained rehearse
rehearsed 003_sessions  up ok, down ok, reversed
rehearsed 004_trim      up ok, down ok, reversed
rehearsed vw_active     up ok, no down step (repeatable)
rollback complete, database unchanged
rehearsal row recorded

rehearsal row recorded means Sustained wrote the row where migrate will read it. The words after the id are the checks that passed, in order: up ok, landed for the migration generated from the config’s models, down ok, and reversed. A check that failed reads not landed or not reversed, with the objects listed underneath and run: sustained plan at the end.

A failure names the statement that failed, and the migrations under it that never ran:

$ sustained rehearse
rehearsed 003_sessions  up ok, down not rehearsed: the run stopped
failed    004_trim      up: column "legacy" of relation "users" does not exist
rollback complete, database unchanged
run: sustained plan

When the config names models, migrate reads the schema back after a successful run. It prints schema matches the models, or one drift <gap> line per difference that is left. This report does not change the exit code.

The other commands print applied <id>, reverted <id>, repaired <action>, or baselined <id>, one per line. With nothing to do they print Nothing to apply., Nothing to revert., Nothing to repair., Nothing to baseline., or Nothing to rehearse. validate prints OK, or one problem <text> line per problem.

Errors go to stderr as error: <message>, or as error in '<migration id>': <message> when the failure came from a known migration.

JSON output

status, validate, plan, and rehearse take --json and print one object to stdout.

$ sustained plan --json
{
  "pending": [
    {
      "id": "004_trim",
      "state": "pending",
      "repeatable": false,
      "statements": [
        {
          "sql": "ALTER TABLE users DROP COLUMN legacy",
          "destructive": true,
          "guards": [{"rule": "no_drops", "verdict": "block"}]
        }
      ],
      "destructive": ["ALTER TABLE users DROP COLUMN legacy"]
    }
  ],
  "problems": [],
  "drift": null
}

Every place a command reports SQL uses that statement object, including drift. When the config names no models, drift is null rather than [], so a caller can tell “nothing was compared” from “compared and found no gap”. statements is null for a callable step, which renders no SQL; before version 2.13.0 statements was a count. A guard verdict appears on the statement it flags, as {"rule", "verdict"}, and a statement no guard flagged has []. The guards key is present from version 2.15.0 onward.

rehearse --json prints:

$ sustained rehearse --json
{
  "rehearsed": [
    {
      "id": "004_trim",
      "up_ok": true,
      "down_ok": true,
      "error": null,
      "landed": null,
      "reversed": []
    }
  ],
  "scratch": false,
  "key": "9c1f...",
  "recorded": true,
  "ok": true
}

landed and reversed are null when the check did not run, [] when the check passed, and the lines naming the trouble when the check failed. key names the content the run covered. recorded says whether Sustained wrote the row where migrate will read it.

status --json prints {"migrations": [{"id": ..., "state": ...}]}. validate --json prints {"ok": ..., "problems": [...]}.