hectic-lab/util.nix
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(postgres-hooks): document hectic bundle + responsibility split

Browse Source 
Add lib/hook/sql/README.md describing bundle layout, apply order, Nix API
(self.lib.hectic.*), shell helper contract, and the steps for adding a new
SQL file. Rewrite db-tool README's hectic section: drop stale
PG_HECTIC_INHERITANCE / HECTIC_INHERITANCE_SQL env vars, add
HECTIC_DOTENV_FILE, document the postgres-init / migrator init / database
hydrate responsibility split.
This commit is contained in:
yukkop
2026-04-30 22:12:18 +00:00
parent 1a209f6960
commit d3cdbdf3e2
2 changed files with 147 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files

101
lib/hook/sql/README.md Normal file
View File

@@ -0,0 +1,101 @@
# hectic SQL bundle
Single source of truth for every object created in the `hectic` PostgreSQL
schema. Consumed by:
- `package/migrator` — applies the bundle on `migrator init` (mandatory).
- `package/db-tool` — applies the bundle in `database hydrate` (default; opt
out with `--no-hook`).
- External consumers (e.g. `proxydoe`) — invoke `psql -f` directly against the
paths exposed via `self.lib.hectic.*.path`.
## Layout
| File | Purpose |
| --- | --- |
| `HECTIC_VERSION` | Single version string for the whole bundle (e.g. `0.1.0`). Read via `lib.fileContents`. |
| `hectic-version.sql` | Templated. Creates `hectic.version`, inserts the current `versionString`, raises on mismatch. |
| `hectic-secret.sql` | Creates `hectic.secret`, `hectic.load_secrets_from_env(text)`, `hectic.get_secret(text)`. |
| `hectic-migration.sql` | Creates the `hectic.migration` table and supporting domains/triggers used by `migrator`. |
| `hectic-inheritance.sql` | Creates `hectic.created_at`, `hectic.updated_at`, `hectic.immutable` parent tables and the DDL event triggers that enforce inheritance, attach `BEFORE UPDATE` triggers, and block DML on immutable tables outside `migration_mode`. |
`hectic-version.sql` is templated at Nix evaluation time: `@HECTIC_VERSION@`
is substituted with the contents of `HECTIC_VERSION`. All other files are
applied verbatim.
## Apply order
The bundle MUST be applied in this order (enforced by
`apply-hectic-bundle.sh`):
1. `hectic-version.sql` — version check first; aborts the rest on mismatch.
2. `hectic-secret.sql`
3. `hectic-migration.sql`
4. `hectic-inheritance.sql`
Re-applying the bundle is idempotent — every CREATE uses
`IF NOT EXISTS` / `CREATE OR REPLACE`, and the version check accepts a row
that already matches.
## Nix API (`self.lib.hectic`)
```nix
self.lib.hectic = {
versionString; # e.g. "0.1.0"
version = { sql; }; # templated
secret = { sql; path; };
migration = { sql; path; };
inheritance = { sql; path; };
applyBundleScript; # ./hook/apply-hectic-bundle.sh
};
```
`.sql` is the file contents as a string. `.path` is the Nix store path of the
verbatim source (only available on non-templated entries; consumers needing a
materialized version of `version.sql` must do
`pkgs.runCommand "hectic-version.sql" { text = self.lib.hectic.version.sql; passAsFile = ["text"]; } ''cp "$textPath" "$out"''`).
## Shell helper (`apply-hectic-bundle.sh`)
`lib/hook/apply-hectic-bundle.sh` is a dash-compatible helper sourced by both
`migrator` and `db-tool`. Public entry point:
```sh
apply_hectic_bundle <PGURL> [<DOTENV_CONTENT>]
```
- `<PGURL>` — full PostgreSQL connection string.
- `<DOTENV_CONTENT>` — optional. When present, after applying the bundle the
helper invokes `hectic.load_secrets_from_env(<dotenv>)` inside a
dollar-quoted (`$ps_env$`) string so secret values cannot terminate the
literal.
Required environment (paths to the SQL files):
- `HECTIC_VERSION_SQL`
- `HECTIC_SECRET_SQL`
- `HECTIC_MIGRATION_SQL`
- `HECTIC_INHERITANCE_SQL`
`migrator` and `db-tool` set these via Nix at build time. External consumers
typically invoke `psql -f` against the paths directly instead of sourcing the
helper.
## Adding a new SQL file
1. Add `lib/hook/sql/hectic-<name>.sql`.
2. Wire it into `lib/default.nix` under `lib.hectic.<name>`.
3. Inject `HECTIC_<NAME>_SQL` in both `package/migrator/default.nix` and
`package/db-tool/default.nix`.
4. Append a `psql -f "$HECTIC_<NAME>_SQL"` step to
`lib/hook/apply-hectic-bundle.sh` in the correct order.
5. Bump `HECTIC_VERSION` if the new content changes existing semantics.
6. Update tests in `test/package/migrator/test/postgresql/init-hectic-bundle/`
and `test/package/db-tool/test/postgresql/hydrate-hook/`.
## Versioning
`HECTIC_VERSION` is a single global version for the bundle, not per-file.
Bump it on any breaking change to the schema. `hectic-version.sql` raises an
exception when the database row diverges from the bundle version, forcing a
deliberate migration before the rest of the bundle runs.

75
package/db-tool/README.md
View File

@@ -32,8 +32,7 @@ These variables must be set for `db-tool` to function.
| `PG_CONF_FILE` | (unset) | Path to a `postgresql.conf` file. When set, replaces the script-generated config entirely on fresh init. `port` and `unix_socket_directories` are still appended at runtime (always overridden). When set, `PG_DISABLE_LOGGING` and `PG_SHARED_PRELOAD_LIBRARIES` are ignored. |
| `PG_SHARED_PRELOAD_LIBRARIES` | `pg_cron` | Comma-separated `shared_preload_libraries` value. Set to empty string to disable. Ignored when `PG_CONF_FILE` is set. |
| `PG_DISABLE_LOGGING` | `0` | Set to `1` to disable PostgreSQL logging collector. Ignored when `PG_CONF_FILE` is set. |
| `PG_HECTIC_INHERITANCE` | `1` | Apply the [`hectic` inheritance bundle](#hectic-inheritance-bundle) to the target database after init. Set to `0` to disable. |
| `HECTIC_INHERITANCE_SQL` | (auto) | Override path to the SQL file applied by `PG_HECTIC_INHERITANCE=1`. Defaults to the SQL shipped with `postgres-init`. |
| `HECTIC_DOTENV_FILE` | (unset) | Optional dotenv file. When set and readable, `database hydrate` passes its contents to `hectic.load_secrets_from_env(...)` after applying the bundle. Falls back to `${LOCAL_DIR}/.env.${ENVIRONMENT}` when unset. |
| `PATCH_LOG` | (stdout) | Path to log the output of database patches. |
| `HYDRATE_LOG` | (stdout) | Path to log the output of database hydration. |
@@ -109,24 +108,36 @@ To use `db-tool` in a Nix development shell, add the following to your `flake.ni
}
```
## hectic Inheritance Bundle
## hectic Bundle
`pkgs.hectic.hectic-inheritance` ships a SQL artifact that bootstraps a `hectic`
schema with three parent tables and DDL event triggers:
`db-tool` and `migrator` apply a single bundle of SQL files that bootstrap the
`hectic` schema. The bundle lives in
[`lib/hook/sql/`](../../lib/hook/sql/README.md) — see that README for full
contract, file layout, and the `self.lib.hectic.*` Nix API.
The bundle creates:
- `hectic.version` — single version row for the entire hectic system.
Mismatch between database and bundle raises an exception.
- `hectic.secret` + `hectic.load_secrets_from_env(text)` +
`hectic.get_secret(text)` — encrypted secret storage and dotenv loader.
- `hectic.migration` — table consumed by `migrator`.
- `hectic.created_at` / `hectic.updated_at` / `hectic.immutable` parent tables
and the DDL event triggers that enforce inheritance, attach
`BEFORE UPDATE` triggers, and block DML on immutable tables outside
`migration_mode`.
Inheritance details:
- `hectic.created_at(created_at TIMESTAMPTZ NOT NULL DEFAULT NOW())` — every
user table must `INHERITS (hectic.created_at)`. The event trigger
`hectic_enforce_created_at_inheritance` raises an exception on `CREATE TABLE`
otherwise.
- `hectic.updated_at(updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW())` — optional.
Any table that inherits from it automatically gets a `BEFORE UPDATE FOR EACH
ROW` trigger calling `hectic.set_updated_at()` attached by
`hectic_attach_updated_at_trigger`.
- `hectic.immutable()` — pure marker. Tables inheriting it are blocked from
`INSERT`/`UPDATE`/`DELETE`/`TRUNCATE` outside migration mode by triggers
attached by `hectic_attach_immutable_triggers`. Useful for reference data
that must only change via migrations. To allow DML inside a migration, wrap
it in a transaction:
`hectic_enforce_created_at_inheritance` raises on `CREATE TABLE` otherwise.
- `hectic.updated_at(updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW())` —
optional. Any inheriting table automatically gets a
`BEFORE UPDATE FOR EACH ROW` trigger calling `hectic.set_updated_at()`.
- `hectic.immutable()` — pure marker. Inheriting tables are blocked from
`INSERT`/`UPDATE`/`DELETE`/`TRUNCATE` outside migration mode. To allow DML
inside a migration, wrap it in a transaction:
```sql
BEGIN;
@@ -148,6 +159,16 @@ Per-database opt-out for additional schemas via the
ALTER DATABASE mydb SET hectic.inheritance_extra_excluded_schemas = 'legacy,etl';
```
### Responsibility split
| Component | Applies bundle? |
| --- | --- |
| `postgres-init` | **No.** Pure PostgreSQL provisioner — starts a vanilla cluster, nothing more. |
| `migrator init` | **Yes, mandatory.** The bundle is a hard prerequisite for `hectic.migration`. |
| `database hydrate` | **Yes, by default.** Re-applied on every hydrate. Skip with `--no-hook`. After applying the bundle, hydrate also calls `hectic.load_secrets_from_env(<dotenv>)` if `HECTIC_DOTENV_FILE` (or `${LOCAL_DIR}/.env.${ENVIRONMENT}`) is readable. |
The bundle is idempotent — repeated application is safe.
### `db-tool diff` and immutable tables
`database diff` already includes immutable tables in its schema-level
@@ -158,25 +179,21 @@ diff of the rows of every table inheriting `hectic.immutable`. Drift in
"frozen" reference data therefore surfaces in the same pager view as schema
drift, and the subcommand exits non-zero when either differs.
### Apply via `postgres-init`
### Apply manually via `psql`
Applied automatically. Set `PG_HECTIC_INHERITANCE=0` to opt out.
### Apply via `migrator` or any psql pipeline
For external consumers (e.g. NixOS modules) bypass the helper and call `psql`
directly against the paths exposed by `self.lib.hectic.*.path`:
```nix
# in your devshell
shellHook = ''
export HECTIC_INHERITANCE_SQL=${pkgs.hectic.hectic-inheritance}/share/hectic/hectic-inheritance.sql
services.postgresql.initialScript = pkgs.writeText "hectic-init.sql" ''
\i ${self.lib.hectic.secret.path}
\i ${self.lib.hectic.migration.path}
\i ${self.lib.hectic.inheritance.path}
'';
```
```sh
psql "$DB_URL" -v ON_ERROR_STOP=1 -f "$HECTIC_INHERITANCE_SQL"
```
The SQL is also exposed via `self.lib.hecticInheritance.sql` (string) and
`self.lib.hecticInheritance.path` (Nix path) for inline pipelines.
The version file (`self.lib.hectic.version`) is templated and only exposes
`.sql` (a string). Materialize it with `pkgs.writeText` if a path is needed.
## Exit Codes