ADR-005: Mainark's control plane moves to Postgres
Date: 2026-10-10
Status: Accepted (2026-10-10); amended 2026-10-10 (the Supabase plan stays Free; P1 is a recovery gate, not a Pro upgrade). Implementation not started; cutover prerequisites P1–P4 outstanding (Migration sequence)
Supersedes: ADR-001 storage choice and its offline requirement; the conclusion of the backend spike ("SQLite prior stands")
Amends: ADR-004 (one database per Product, storage names, I4/I8/I9/I10 enforcement, phase order), ADR-003 (DDL dialect), plan inbox ("only a mailbox"), dual task graphs (retired)
Decision
One Postgres database is the single source of truth for every Product's control-plane data. SQLite remains only as a rollback path during a bounded compatibility window, then is removed.
- One database, many Products. Fulreach, Mainark and future Products live
in one database.
product_idpartitions every domain row. Product is a table, not an identifier prefix or a file path. - Where: Supabase project
okznyuneokewngghvziq("Mainark", organisation "Mainark",eu-central-1, Postgres 17.6), in a schemamainarkthat the Supabase Data API does not expose. Never Fulreach's project (qvqvpnog…, organisation "Adaminds"). The project stays on the Free plan for implementation and rehearsals. It does not become the sole production authority until the recovery gate passes. - Who: daemon, coordinator, HTTP API (and therefore the web UI) and CLI read and write Postgres. Agents never get database credentials.
- No dual write. Data moves once, under a writer freeze and fence. A reverse export exists only for the rollback window.
- Invariants move into the schema (composite foreign keys, row checks, locked triggers).
- Merge, not pick. The existing SQLite copies have diverged; the cutover merges their legitimate history by documented rules (Reconciliation).
- Order: storage moves before ADR-004 phases 3–5 and ADR-003's tables.
productsandproduct_id(ADR-004 phase 5) move into this migration. - Client: postgres.js (evaluation).
Context
Measured 2026-10-10.
| Fact | Detail |
|---|---|
| Live writers | Hetzner box (fulreach-workers): one process, mainark-coordinate (coordinate --watch --auto-review, since 2026-09-29), using /opt/mainark/data/fulreach-tasks.db (FUL). Laptop: CLI and dev daemon using data/orchestrator.db (FUL) and ~/mainark-data/mainark-tasks.db (MAI, symlinked as data/mainark-tasks.db; the only MAI writer since the MAI authority handover, with the box MAI file frozen). |
| Copies | Seven SQLite files hold Mainark data: laptop FUL, laptop MAI, laptop mainark-tasks-mycopy.db; box FUL (live), box MAI (idle since 2026-09-28), two older copies in /home/abdullah; plus an empty /opt/mainark/mainark/data/orchestrator.db. |
| Divergence | Both machines allocate FUL ids from their own counters. FUL-1061…FUL-1076 name two different sets of 16 items, and both sets are referenced on GitHub. The box's counter (1077) is already behind the laptop's (1116), so its next import collides again. Interim partition until cutover: see Interim measures. |
| Accidental migration | On 2026-10-10 a feature-branch dev daemon applied 012_plan_inbox (15:16) and 013_work_item_kind (15:53) to the laptop's real FUL database, because every process migrates on startup. The laptop FUL file now has the kind column and eight phase 2 triggers; the box is at 012. |
| Backups today | Until 2026-10-10 the box's nightly backup copied only mainark-tasks.db, idle since 2026-09-28, so the live FUL database had no backup; it now covers both files (see Interim measures). Box backups sit on the same disk as the databases. The Supabase project has none (Free plan). |
| Size | ~10k rows across 10 tables. |
| Code coupling | 78 synchronous repository functions; 365 call sites in 54 source files, 636 in tests. |
| Web / API | Web reads only through the HTTP API. |
| Bun | Laptop 1.2.2, box 1.4.2. |
Separate files per Product and per machine produced the collision above. One shared database removes the cause rather than the symptom.
Why Postgres, and what we give up
Gains: one truth for every machine; real concurrency (row locks,
SKIP LOCKED, advisory locks); declarative invariants; managed backups;
logical dumps portable to any Postgres; plan inbox and work items in one
transaction.
Losses and costs
- Offline operation ends. ADR-001 required issue CRUD,
implement,queueand dispatch to work with no network. After cutover every Mainark command that reads or writes work needs the database. There is no offline mode and no local cache that can accept writes; local Postgres exists only for development against non-production data. Accepted: central consistency matters more, and nearly every useful action already needs GitHub, Cursor or a model API. - Supabase is a hard dependency. If it is unreachable, Mainark stops dispatching and the CLI fails. Running agents continue; their results are pulled later because every external input is pull-based.
- Async conversion of every repository call (the largest piece of work).
- Network latency per query (laptop in Zurich → Frankfurt ≈ 10 ms; box → Frankfurt measured in phase B). Hot-path loops that query per row are batched.
- Cost: none while on Free. The recovery gate may choose Pro (managed backups) or another tested recovery mechanism; PITR later, if ever.
Where: the Mainark Supabase project
| Question | Finding (2026-10-10) |
|---|---|
| Separate from Fulreach? | Yes: own organisation ("Mainark"), own project. Fulreach production and dev are in organisation "Adaminds". |
| Contents | Only the plan inbox: public.plan_inbox (2 rows, both imported), private.inbox_secret (1 row), RPCs. No mainark schema and no mainark_* roles yet. The box's inbox client points at this project. |
| Region / version | eu-central-1 (Frankfurt), Postgres 17.6.1.127, status ACTIVE_HEALTHY. |
| Plan | Free (Management API GET /v1/organizations/uixmcsjrdcatlgcnshgm: plan: free; re-verified 2026-10-10 17:15 UTC). |
| Backups | None. GET /v1/projects/okznyuneokewngghvziq/database/backups: walg_enabled: true, pitr_enabled: false, 0 backups; supabase backups list agrees. Add-ons: none selected (pitr, compute_instance, ipv4 available). |
| Tooling hazard | The Supabase MCP connector configured in Cursor points at Fulreach production (qvqvpnog…), not this project. Mainark tooling and agents must never use it for Mainark work. |
| Exposure | The anon key is distributed to Cursor environments; the Data API exposes public and graphql_public. |
| Connectivity | Direct host is IPv6-only (box: fine). Laptop uses the Supavisor session pooler (IPv4). |
Recommendation: reuse this project, with:
- Stay on Free for now. Schema, adapters, importer/exporter, tests and rehearsals use the current Free project. Free has no managed backups and can pause after inactivity, so the project becomes the sole production authority only after the recovery gate passes. No plan upgrade or paid add-on without explicit approval.
- Schema
mainark, not in the Data API's exposed schemas;revoke all on schema mainark from public, anon, authenticated. The inbox stays inpublicbehind its RPCs. CI runs Supabase's security advisor. - Roles:
mainark_owner(owns the schema; migrations only) andmainark_app(DML onmainark.*,select, updateonpublic.plan_inbox). Notpostgres, not the service key. The service key leaves the box after cutover.
Database layout
mainark (not exposed)
products, product_policies, id_sequences
work_items, work_item_aliases, work_item_dependencies
work_item_events, work_item_changes
agent_runs, review_cycles, events, scheduled_dispatch_slots, inbox_imports
schema_migrations, migration_ledger
public plan_inbox private inbox_secret
Table names. Postgres uses work_items (and work_item_*): a neutral
persistence name for rows whose explicit kind is goal | task. It is not
user-facing vocabulary; the product speaks of Goals and Tasks (ADR-004).
Repository function names and domain types are unchanged by the port.
Products
create table mainark.products (
id text primary key check (id ~ '^[a-z][a-z0-9-]{1,30}$'), -- 'fulreach', 'mainark'
name text not null,
identifier_prefix text not null unique check (identifier_prefix ~ '^[A-Z][A-Z0-9]{1,9}$'), -- 'FUL', 'MAI'
repo_url text not null unique,
default_branch text not null default 'main',
branch_prefix text not null, -- 'feat/ful-'
validate_cmd text,
linear_team_key text unique, -- integration boundary only
status text not null default 'active' check (status in ('active', 'archived')),
created_at timestamptz not null default now(),
updated_at timestamptz not null default now()
);
create table mainark.product_policies (
product_id text primary key references mainark.products(id),
auto_dispatch_enabled boolean not null default false,
auto_review_enabled boolean not null default false,
auto_review_max_rounds int not null default 3,
max_review_cycles int not null default 3,
max_inflight_runs int not null default 1,
daily_budget_usd numeric(10,2),
auto_merge boolean not null default false,
auto_merge_strategy text check (auto_merge_strategy in ('merge', 'squash', 'rebase')),
pr_review_after_push boolean not null default false,
cloud_env text check (cloud_env in ('cloud', 'machine', 'pool')),
cloud_env_name text,
updated_at timestamptz not null default now(),
updated_by text not null
);
create table mainark.id_sequences (
product_id text not null references mainark.products(id),
family text not null check (family in ('item', 'incident', 'run')),
next_value bigint not null check (next_value > 0),
primary key (product_id, family)
);
Where today's per-Product configuration goes
Rule: facts every machine must agree on go into the database; paths on one machine and secrets stay in that machine's environment. While knobs move, env may only restrict database policy (a kill switch can turn auto-dispatch off, never on).
| Today | Moves to | When |
|---|---|---|
SQLITE_PATH (file = Product) | Gone. MAINARK_PRODUCT=<id>; the CLI can infer it from the cwd repo's remote vs products.repo_url | Cutover |
MAINARK_ISSUE_PREFIX, LINEAR_TEAM_KEY as prefix | products.identifier_prefix | Cutover |
REPO_URL, GIT_DEFAULT_BRANCH, MAINARK_BRANCH_PREFIX, MAINARK_VALIDATE_CMD, LINEAR_TEAM_* | products | Cutover |
id_sequences | mainark.id_sequences (Product × family) | Cutover |
scheduled_dispatch_slots.graph, MAINARK_DAILY_GRAPH | product_id | Cutover |
MAINARK_COORDINATE_AUTO_DISPATCH, …_AUTO_REVIEW(_MAX_ROUNDS), MAX_REVIEW_CYCLES, MAINARK_MAX_INFLIGHT, MAX_PARALLEL_AGENTS, AUTO_MERGE(_STRATEGY), MAINARK_PR_REVIEW_AFTER_PUSH, MAINARK_CLOUD_ENV(_NAME), per-Product MAINARK_DAILY_BUDGET_USD | product_policies | After the rollback window, one knob at a time |
.mainark/epics.json, .mainark/services.json, .reviews/epic-*.state.json | Goal columns, services, Goal phase progress (ADR-004 phase 5) | After the window |
REPO_PATH, MAINARK_ROOT, MAINARK_HOME, MAINARK_TARGET(_DIR), MAINARK_MACHINE_* | Stay in env (machine-local paths) | — |
API keys, GITHUB_PAT, SLACK_WEBHOOK_URL, inbox secret, database URL | Stay in env (secrets) | — |
| Executor/model choices | Env for now; product_policies once they differ per Product | Later |
Schema translation
The port is a translation. Allowed changes: Postgres types, product_id,
domain table names, declarative invariants, normalised dependencies, global
run identity. Pipeline columns, statuses and JSON shapes keep their meaning.
| SQLite | Postgres |
|---|---|
issues | work_items + product_id; kind (ADR-004); generated parent_kind |
DATETIME text (two formats seen) | timestamptz; importer parses YYYY-MM-DD HH:MM:SS and ISO-8601 Z as UTC, refuses anything else |
*_json TEXT | jsonb |
| Linear display columns | one linear jsonb column (obsolete per ADR-001, kept as data) |
autonomy_json (from unmerged 011_autonomy.sql) | autonomy jsonb, carried as data (1 non-default row: FUL-1026) |
depends_on CSV | work_item_dependencies (below); the CSV does not exist in Postgres |
agent_runs.id (per-file autoincrement) | id bigint generated always as identity — the only identity; plus run_number (Product-scoped display number) |
issue_events, events, review_cycles ids | new identity ids; migration_ledger keeps source file + old id |
issue_events.issue_id | work_item_events.work_item_id not null FK |
issue_changes (trigger-written change feed; its id is the cursor of GET /issues/changes) | work_item_changes, not imported: it is derived, not history. Its identity starts above the highest issue_changes.id in any source (today 2,809), so no API client holds a cursor ahead of the new feed; clients re-read the list once at go-live |
inbox_imports | + product_id; import is one transaction with the inbox row |
_migrations | Not imported. Postgres has its own schema_migrations; each source's migration level is recorded in migration_ledger |
events.issue_id (no FK; a webhook log whose subject may be unknown) | work_item_ref text (the id as written) + work_item_id nullable FK, set when the ref resolves; product_id from the source file |
| — | work_item_aliases (product_id, alias, work_item_id, reason, created_at) for former ids (see reconciliation) |
Run identity
agent_runs.id is a global bigint identity and the only key anything
references (parent_run_id, scheduled_dispatch_slots.agent_run_id, API
paths). run_number is a Product-scoped display number from
id_sequences(product, 'run'), unique per Product, used in CLI and Slack text
("FUL run 48"). At import, run_number = the run's SQLite id, which the
reconciliation shows is consistent across copies for every shared run.
Dependencies
depends_on becomes a table designed around ADR-004's semantics.
create table mainark.work_item_dependencies (
id bigint generated always as identity primary key,
product_id text not null,
work_item_id text not null, -- the dependent: Goal or Task
depends_on_id text not null, -- the blocker: Goal or Task
created_at timestamptz not null default now(),
created_by text not null, -- 'cli:adrian', 'inbox:42', 'migration'
waived_at timestamptz,
waived_by text,
waiver_reason text,
replaced_by_id bigint references mainark.work_item_dependencies(id), -- set when re-pointed
foreign key (product_id, work_item_id) references mainark.work_items (product_id, id),
foreign key (product_id, depends_on_id) references mainark.work_items (product_id, id),
check (work_item_id <> depends_on_id),
check ((waived_at is null) = (waiver_reason is null)
and (waived_at is null) = (waived_by is null))
);
-- One active edge per pair; waived edges stay as history.
create unique index work_item_dependencies_active
on mainark.work_item_dependencies (product_id, work_item_id, depends_on_id)
where waived_at is null;
-- "What is X blocked by" and "what does X block".
create index on mainark.work_item_dependencies (product_id, work_item_id) where waived_at is null;
create index on mainark.work_item_dependencies (product_id, depends_on_id) where waived_at is null;
| Requirement | How |
|---|---|
| Product isolation | Both FKs include product_id |
| No self-dependency | CHECK |
| No unknown ids | FKs |
| No cycles; no dependency on an own ancestor or descendant (I10) | Insert trigger: recursive CTE over active edges and over the parent chain, under the Product's hierarchy advisory lock (shared with re-parenting, which can create the same conflicts) |
| Cancelled never satisfies; only a waiver removes (I12) | Edges are never deleted: mainark_app has no DELETE on this table. Removing a dependency = setting waived_* with a required reason |
| Re-point | One transaction: waive the old edge (reason repointed), insert the new edge, set replaced_by_id. Events dependency_added, dependency_waived, dependency_repointed in work_item_events |
| Satisfaction (ADR-004) | One SQL function mainark.dependency_satisfied(product, depends_on_id): a Task when work_status in ('done') (later `merged |
| Efficient blocker queries | Partial indexes on active edges; "ready Tasks" = Tasks with no active edge whose blocker is unsatisfied — one query. Goal satisfaction recurses over descendants; if it shows up in query plans, phase 3's Goal rollup provides a stored status |
Goal/Task invariants in Postgres
Verified in a spike (PGlite, Postgres 17): each rule rejected the violating write and accepted the valid one.
create table mainark.work_items (
product_id text not null references mainark.products(id),
id text primary key,
kind text not null check (kind in ('goal', 'task')),
parent_id text,
parent_kind text generated always as (case when parent_id is null then null else 'goal' end) stored,
work_status text not null default 'todo' check (work_status in ('todo','in_progress','paused','done','cancelled')),
pr_number int,
pr_url text,
-- … pipeline columns unchanged …
unique (product_id, id),
unique (product_id, id, kind),
foreign key (product_id, parent_id, parent_kind) references mainark.work_items (product_id, id, kind),
check (kind = 'task' or (pr_number is null and pr_url is null))
);
create table mainark.agent_runs (
id bigint generated always as identity primary key,
product_id text not null,
run_number bigint not null,
work_item_id text not null,
work_item_kind text not null default 'task' check (work_item_kind = 'task'),
-- …
unique (product_id, run_number),
foreign key (product_id, work_item_id, work_item_kind) references mainark.work_items (product_id, id, kind)
);
The id-prefix rule (FUL-… belongs to the Product with prefix FUL) is an
insert trigger, because a CHECK cannot read products.
| # | Rule | SQLite (phase 2) | Postgres |
|---|---|---|---|
| I1 | kind ∈ {goal, task} | CHECK | CHECK |
| I2 | Parent is a Goal | trigger | Composite FK (product_id, parent_id, 'goal') |
| I3 | A Task has no children; a Goal with children cannot become a Task | trigger + repository | Same FK (changing a referenced kind violates it) |
| I4 | No parent cycles | self-parent trigger; deeper only in setParentId | Recursive-CTE trigger, any depth, any writer, under a per-Product advisory lock |
| I5 | Runs only on Tasks; a Task with runs cannot become a Goal | triggers | Composite FK (product_id, work_item_id, 'task') |
| I6 | PRs only on Tasks | trigger | Row CHECK; the future pull_requests table uses the composite-FK pattern |
| I7 | Incidents link to Tasks | — | Composite-FK pattern on incidents |
| I8 | Goal work_status ∈ {todo, paused, cancelled} | deferred | Row CHECK, after phase 3's rollup cleans the data |
| I9 | No cross-Product parent, dependency, run, incident | not enforceable | product_id in every composite FK |
| I10 | Dependency rules | repository + doctor | Dependencies |
| I11 | Only Tasks are dispatched | dispatch policy + I5 | Unchanged |
| I12 | Cancelled never satisfies; waiver only | single predicate | dependency_satisfied + no DELETE grant |
Foreign keys lock the referenced row and are race-free under READ COMMITTED; trigger checks that read other rows are not (SQLite hid this with its single writer). Hence FKs for I2/I3/I5/I9, and an explicit per-Product advisory lock for the remaining triggers.
Concurrency
- Claims: one conditional
update … where kind = 'task' and work_status = 'todo' returning …; queue pickers usefor update skip locked. - Id allocation:
update id_sequences … returninginside the creating transaction. - One loop per Product: daemon/coordinator loops take a session
pg_try_advisory_lockper Product and skip the tick if another process holds it. Requires a session connection (direct or session pooler), never the transaction pooler. - Inbox import:
for update skip lockedonpublic.plan_inbox, create work items, mark the row — one transaction. Plans name their Product.
Postgres client
Evaluated postgres.js 3.4.9 against Bun's built-in Bun.sql on Bun 1.2.2
(laptop) and 1.4.2 (box), against a real Postgres, on this codebase's patterns:
| Check | postgres.js | Bun.sql 1.2.2 | Bun.sql 1.4.2 |
|---|---|---|---|
Positional $1 text queries (unsafe) | ✓ | ✓ | ✓ |
timestamptz→Date, jsonb, bigint as string | ✓ | ✓ | ✓ |
numeric(10,2) | ✓ "12.50" | ✗ corrupted (" �") | ✓ |
Array parameters (= any($1), text[]) | ✓ | ✗ | ✗ (via unsafe) |
| Rollback on throw, savepoints | ✓ | ✓ | ✓ |
| Error exposes SQLSTATE + constraint name | ✓ (code) | ✓ (errno) | ✓ |
| Reserved connection holding a session advisory lock | ✓ | ✓ | ✓ |
FOR UPDATE SKIP LOCKED in a transaction | ✓ | ✓ | ✓ |
| LISTEN / NOTIFY | ✓ | ✗ | ✓ |
| Runs outside Bun (Node scripts, tooling) | ✓ | ✗ | ✗ |
Recommendation: postgres.js. It is the boring choice: mature, behaviour
independent of the Bun version installed on each machine (they already differ),
array parameters for batch and dependency queries, transactions, savepoints,
reserved connections, prepare: false if a transaction pooler is ever needed,
and portable to Node. Bun.sql is improving quickly but corrupted numeric
values two minor versions ago. Behind the Db interface the choice can be
revisited without touching repositories. Bun versions are pinned per machine
regardless.
Access layer
- Repositories remain the only owners of SQL; functions become
async. - A minimal
Dbinterface (query(text, params),transaction(fn),reserve()), with a postgres.js adapter (production) and a PGlite adapter (tests). No ORM. - Connections: box → direct (IPv6), pool ≤ 5. Laptop → session pooler.
no-floating-promiseslint forsrc/during the conversion.- Query count per coordinator tick and per CLI command is instrumented; hot loops are batched.
Migrations and schema versions
- SQL files in
src/db/pg/migrations/, applied by Mainark's migrator in a transaction under an advisory lock, recorded inmainark.schema_migrations. The same files run in PGlite, local Postgres, CI and production. - No process migrates on startup. Production is migrated only by
mainark db migraterun by the box's deploy step. Every process checks on startup that the schema version is within its supported range and refuses to run otherwise. - Expand/contract: laptop and box may run different releases against one database, so every migration works with the previous release.
- The Supabase CLI runs a local stack only; production migrations do not go
through
supabase db push.
Testing
No test depends on a remote Supabase project.
| Tier | What | Where | Speed |
|---|---|---|---|
| Pure | Routing, dispatch policy, plan parsing, queue views with injected data | everywhere | ms |
| Repository (default) | Repositories, constraints, triggers, migrations against PGlite (in-process Postgres); one instance per test process, truncate between tests | pnpm mainark:validate, laptop, CI; no Docker | ~0.5–0.7 s once, ~0.1 ms/query |
| Postgres integration | Concurrent claims, advisory-lock leases, cycle races, pooler behaviour, import/export round trip, production-sized snapshot | pnpm test:pg against Postgres 17 (Docker or supabase start); CI service container | seconds |
PGlite is single-connection, so races belong to the integration tier. CI runs the repository tier on both adapters to catch drift.
Local development and CI
- Local Postgres (
supabase startorpostgres:17);MAINARK_DATABASE_URLpoints at it by default;mainark db migrateandmainark db seed(two Products, a few Goals/Tasks). - Production from the laptop requires an explicit production URL plus
MAINARK_PRODUCTION=1; a laptop daemon only dispatches if it wins the Product lease. - CI: typecheck → pure + PGlite → Postgres integration → security advisor.
Backups and recovery
Today's state, measured: the Mainark Supabase project (Free) has no backups
and no PITR. Since 2026-10-10 the box's nightly SQLite backup covers both
fulreach-tasks.db and mainark-tasks.db (before that, only the idle MAI
file, now frozen). The authoritative MAI file is on the laptop and has no
scheduled backup: SQLite is transitional, and the effort goes into making
Postgres authoritative. Its recovery points are the two verified snapshots
from the handover.
Operational risk: box backups are on the same disk. mainark-backup.sh
writes to /opt/mainark/backups on /dev/sda1, the disk that holds
/opt/mainark/data. Nothing copies them off the box. Losing the disk, the
box or the Hetzner account loses the live SQLite databases and every backup
together; the only other copies are the laptop's divergent files and the
/tmp reconciliation snapshots. Off-site copies are deliberately not set up
yet. The risk is accepted until cutover, where the off-site pg_dump copies
below replace it, and should be revisited if cutover slips.
Free-plan risks (explicit, current):
- No managed automatic backups on Free.
- The project can be paused after a period of inactivity; a paused project stops every Mainark command until it is resumed.
- The project has zero backups today (verified 2026-10-10).
- Therefore it must not become the sole production authority until the recovery gate passes. Implementation and rehearsals on Free are fine: during them the SQLite files remain authoritative.
Recovery gate (P1)
Before Postgres becomes the sole production authority (step 15 of the migration sequence), a recovery strategy is explicitly approved and tested. The gate is about the outcome, not a plan tier:
- Approved: the chosen strategy, its RPO and RTO, and who restores, are written down and approved by the maintainer.
- Tested: a restore from that strategy into a fresh Postgres passes
mainark db reconcile/validation andmainark nextper Product, within the stated RTO. - Availability: the strategy says how a paused or unreachable project is handled (resume procedure, or restore elsewhere from the backups).
Candidates (none predetermined; one or a combination satisfies the gate):
| Option | What it gives | Cost |
|---|---|---|
| Supabase Pro with managed backups | 7 daily backups, no pausing; whole-project restore | Pro plan |
Off-site logical dumps (the hourly pg_dump below) + tested restore | RPO ≤ 1 h; restore to any Postgres | none beyond storage |
| Another tested mechanism | as documented and tested | — |
Layers planned regardless of the gate's choice (no PITR yet):
| Layer | What | RPO | Restores |
|---|---|---|---|
| Supabase daily backups | Only if the gate chooses Pro: 7 daily backups, no pausing | ≤ 24 h | Whole project, to a day (project offline during restore) |
| Mainark logical dumps | pg_dump --schema=mainark --schema=public from the box hourly, kept 48 h locally and 30 days off-site (the database is a few MB) | ≤ 1 h | Any Postgres, any provider |
| Reverse export (rollback window only) | Nightly mainark db export-sqlite to the box, kept for the window | ≤ 24 h | The previous SQLite release |
| Restore drill | Before cutover, then quarterly: restore the latest dump into a local Postgres, run validation and mainark next per Product | — | Proves the dumps work |
PITR (add-on, requires Small compute; ≈ $100/month for 7 days, RPO ≈ 2
min) is not a cutover prerequisite: hourly dumps bound loss to one hour of
control-plane writes, most of which (PR merges, run status) are re-derivable
by polling GitHub and Cursor. Enable PITR when unattended dispatch runs at
volume. After any Supabase restore, reset mainark_app / mainark_owner
passwords (custom-role passwords are not in Supabase backups).
Reconciliation of the existing SQLite copies
Snapshots taken 2026-10-10 with VACUUM INTO (read-only on the source) from
every copy; all pass integrity_check. Re-checked at 17:10 UTC: the live
files are unchanged since the snapshots except the box's FUL counter (interim
partition). Natural keys are used because autoincrement ids are per file:
items by id (created_at detects collisions); runs by (item, role,
created_at, external run id); item events by (item, time, kind, detail);
issue_changes by (item, time); duplicate keys are numbered, not merged. The
read-only comparison (review.ts, kept with the snapshots) becomes
mainark db reconcile in phase C, and its 2026-10-10 output is the baseline
for the T-1 inventory.
Copies
| Copy | Items | Runs | Item events | Schema | Counter | Verdict |
|---|---|---|---|---|---|---|
Laptop FUL data/orchestrator.db | 857 (5 test rows) | 129 | 2,017 | 013 | FUL 1116 | Source |
Box FUL /opt/mainark/data/fulreach-tasks.db (live) | 818 (5 test rows) | 14 | 1,789 | 012 | FUL 1077 (2000 since the partition) | Source |
Laptop MAI data/mainark-tasks.db | 46 | 61 | 152 | 011 (no inbox_imports, no auto_dispatch) | MAI 47 | Strict subset of box MAI |
Box MAI /opt/mainark/data/mainark-tasks.db | 48 | 68 | 161 | 012 | MAI 49 | Source |
/home/abdullah/fulreach-tasks.db, …/mainark-tasks.db, laptop mainark-tasks-mycopy.db | 802 / 40 / 40 | — | — | — | — | No item, run or event absent from the sources: archived, not imported |
Both MAI files also contain an unused FUL counter row (next_value 1; no
FUL items): dropped.
This table and the results below are the reviewed 2026-10-10 baseline. Since
the MAI authority handover the laptop
MAI row describes the archived stale file, the laptop's data/mainark-tasks.db
is the box file's content plus later laptop writes, and the box MAI file is
frozen.
Results by table
FUL, laptop vs box:
| Table | Laptop | Box | Identical | Only laptop | Only box | Same key, different content |
|---|---|---|---|---|---|---|
issues | 857 | 818 | 751 | 39 | 0 | 67: 16 collisions, 50 laptop newer, 1 box newer |
agent_runs | 129 | 14 | 12 | 115 | 0 | 2 (metadata only) |
issue_events | 2,017 | 1,789 | 1,767 | 250 | 22 | 0 |
issue_changes | 2,808 | 2,443 | 2,424 | 384 | 19 | 0 |
events | 457 | 457 | 457 | 0 | 0 | 0 |
review_cycles | 1 | 1 | 1 | 0 | 0 | 0 |
inbox_imports | 0 | 2 | 0 | 0 | 2 | 0 |
scheduled_dispatch_slots | 0 | 0 | — | — | — | — |
id_sequences | FUL 1116 | FUL 1077 | — | — | — | 1 |
_migrations | 14 | 13 | 11 | 1 (013_work_item_kind) | 0 | 2 (applied_at of 011 and 012) |
MAI, laptop vs box:
| Table | Laptop | Box | Identical | Only laptop | Only box | Same key, different content |
|---|---|---|---|---|---|---|
issues | 46 | 48 | 45 | 0 | 2 | 1 (box newer) |
agent_runs | 61 | 68 | 61 | 0 | 7 | 0 |
issue_events | 152 | 161 | 152 | 0 | 9 | 0 |
issue_changes | 174 | 181 | 174 | 0 | 7 | 0 |
id_sequences | MAI 47 | MAI 49 | — | — | — | 1 |
_migrations | 12 | 13 | 12 | 0 | 1 (012_plan_inbox) | 0 |
| others | empty | empty | — | — | — | — |
Shared runs keep the same SQLite id on both sides (FUL 14 of 14, MAI 61 of
61), so run_number = SQLite id is unambiguous.
Unique rows and conflicts, by type
FUL
| Type | Rows | Detail | Treatment |
|---|---|---|---|
| Items only on laptop | 39 | FUL-1077…1115, created 2026-09-17…10-01; 20 todo, 14 done, 5 in_progress. FUL-1077…1081 are children of the laptop's FUL-1076 | Imported |
| Items only on box | 0 | — | — |
| Same item, laptop newer | 50 | 41 reached done on the laptop (23 from todo, 16 from in_progress, 2 from paused); 9 differ only in state_name; 11 carry PR fields the box lacks (FUL-986…996, PRs #1370–#1383); FUL-990 adds a dependency on FUL-1075. No box-only event touches any of them | Laptop row |
| Same item, box newer | 1 | FUL-1325: done, PR #1325 merged, synced on the box 2026-09-29; in_progress on the laptop | Box row |
| Same id, different item | 16 | FUL-1061…1076 | Below |
| Runs only on laptop | 115 | Ids 15–129, 2026-09-15…09-24: 76 review (18 succeeded, 51 failed, 7 cancelled), 29 implement, 10 fix. 20 are on colliding ids, all belonging to the laptop items | Imported; run_number = SQLite id |
| Runs, different content | 2 | Runs 4 (FUL-999) and 5 (FUL-1001): the laptop's metadata_json adds 12 auto-review keys; no key differs | Laptop row (superset) |
| Item events only on laptop | 250 | 101 work_status, 65 updated, 55 created, 15 claimed, 14 comment: 117 on laptop-only items, 64 on laptop-newer items, 67 on the laptop's colliding items, 2 on FUL-999/1001 | Imported |
| Item events only on box | 22 | 6 merge-sync events (work_status + pr_merged for FUL-999, FUL-1001, FUL-1325, 2026-09-29) and 16 created events of the box's colliding items | Imported; the 16 move to the new ids |
issue_changes only on one side | 384 / 19 | Derived feed rows; 87 of the laptop's are the accidental 013 backfill | Not imported |
inbox_imports only on box | 2 | Supabase plan_inbox rows 1 and 2 → box FUL-1061…1067 and FUL-1068…1076 | Imported with the new ids |
| Counter | 1 | 1116 vs 1077 (2000 since the partition) | Neither: recomputed |
| Migration records | 3 | 013 only on the laptop; 011/012 applied_at differ | Not imported; levels in ledger |
MAI
| Type | Rows | Detail | Treatment |
|---|---|---|---|
| Items only on box | 2 | MAI-47 (cancelled), MAI-48 (done, PR #96) | Imported |
| Same item, box newer | 1 | MAI-39: done, PR #56 merged 2026-09-28; in_progress on the laptop | Box row |
| Runs only on box | 7 | Ids 62–68: MAI-47 implement, review, fix, review, review; MAI-48 implement, review | Imported |
| Item events only on box | 9 | MAI-47 created/claimed/cancelled; MAI-48 created/claimed/done/pr_merged; MAI-39 done/pr_merged | Imported |
issue_changes only on box | 7 | Derived | Not imported |
| Anything only on laptop | 0 | — | — |
Integrity findings (identical in both FUL copies; none in MAI): no orphan runs, no missing parents.
| Finding | Detail |
|---|---|
foreign_key_check | 1 row: issue_events 525 |
| Orphan item event 525 | FUL-759 created, 2026-07-31 13:42:27, title "Main"; FUL-759 exists in no copy |
Orphan events row 36 | FUL-207 webhook_ignored_unlabelled, 2026-04-30 (Linear webhook era); events.issue_id has no FK |
| Missing dependency targets | FUL-307 (done) → FUL-206, FUL-207; FUL-696 (paused, child of FUL-688) → FUL-692 (its other blockers FUL-691 and FUL-693 exist). None of the three targets exists in any copy |
| Test rows | PG-1, PG-2, PG-3, SP-1, NEW-1 ("…: Title", 2026-07-20 15:47:11, todo, no runs or events) |
Non-default autonomy_json | FUL-1026 (eligible: false, risk: medium, class: refactor, max_attempts: 2) |
The 16 colliding FUL ids
The laptop created FUL-1061…1076 on 2026-09-15…17: Hiring Signal dashboards,
the CEO digest, the Hiring Snapshot Goal FUL-1064 with children, the Provider
Governor Goal FUL-1070 with children, and the Warm engager Goal FUL-1076. They
have 20 runs, PRs #1343 (FUL-1061), #1344 (FUL-1062), #1359 (FUL-1065) on the
items and #1345, #1385, #1387 on runs of FUL-1063, FUL-1066, FUL-1067, plus
dependents: FUL-990 → FUL-1075; FUL-1077…1081 are children of FUL-1076 and
depend on FUL-1075. The box created a different FUL-1061…1076 on
2026-10-03/04 from inbox plans 1 ("Workspace lead ownership", 7 steps) and 2
("Inbound analysis", 9 steps): todo, no parent, no runs, 17 dependency edges
among themselves. GitHub PRs #1502 (fix/ful-1063-…), #1504
(fix/ful-1064-…) and #1505 ("FUL-1066: …") use the box's ids. Both sets are
legitimate, and both are referenced on GitHub.
Attribution. A row referencing FUL-1061…1076 belongs to the laptop item
if it is dated before 2026-10-03 19:34:21 (the box's first import) and to the
box item otherwise. Verified: the laptop's 67 events and 20 runs on these ids
are all dated ≤ 2026-10-01; the box's 16 rows are the created events at
import time; no row shared by both copies references these ids.
Handling
- The laptop items keep their ids: created first, with runs, PRs, children and dependents inside Mainark.
- The box items get new ids in creation order, then old-id order, after the highest FUL number in any copy at the final snapshot. Today that is FUL-1325, giving FUL-1326…1341. If the box has allocated ids ≥ 2000 by then (interim partition), they are imported as they are and the 16 follow the highest of them.
- Their 17 edges among themselves are rewritten to the new ids; their 16
createdevents and the 2inbox_importsrows move with them;plan_inboxrows 1 and 2 get the new ids in the same transaction. work_item_aliasesrecords each former id (reasonrenumbered: id collision, box copy 2026-10-03), so a search for "FUL-1066" finds both items; the import writes arenumberedevent on each.- Not inferred: six PRs for the box plans were merged on 2026-10-04
(#1498, #1499, #1501, #1502, #1504, #1505) but never linked, because merge
sync links through runs and these items have none. The import keeps those
items
todo; the cutover report lists them for a human to link and close.
| Old id | Laptop item (keeps the id) | Box item | Box item's new id (today) |
|---|---|---|---|
| FUL-1061 | Dashboard: Hiring Signal Performance by Country / Job type | Permanent workspace invite dedupe across seats | FUL-1326 |
| FUL-1062 | Dashboard: Hiring Signal Opening titles rollup | Migration: invite_not_before floor, cross-account stats, owned-by-seat skip | FUL-1327 |
| FUL-1063 | CEO Weekly Outreach Report digest | Stop seat reconcile from undoing cross-account cooldown deferrals | FUL-1328 |
| FUL-1064 | Goal: Recipient Technical Hiring Snapshot (LinkedIn PDF) | Use effective available inventory in capacity, recovery and pace monitor | FUL-1329 |
| FUL-1065 | Operator inbox: preview → confirm → send hiring snapshot PDF | One owner seat per person at enqueue | FUL-1330 |
| FUL-1066 | Interest-gated auto generate → send hiring snapshot | Observability: cross-account deferrals and workspace_overlap_capacity alert | FUL-1331 |
| FUL-1067 | Sequence step: configurable hiring_snapshot dynamic asset | Partition script for existing cross-seat overlaps (dry-run default) | FUL-1332 |
| FUL-1068 | Campaign defaults for hiring snapshot steps | Inbound Analysis respects the dashboard period (from/to) | FUL-1333 |
| FUL-1069 | (Parked) LLM narrative enrichment for hiring snapshot | Share one LinkedIn posts query between Inbound Analysis and Suggested posts | FUL-1334 |
| FUL-1070 | Goal: LinkedIn Provider Governor (Unipile envelopes) | Clarify misleading inbound metrics: daily chart and Booked calls | FUL-1335 |
| FUL-1071 | Spec + correct Unipile provider-limits mental model | Post → lead attribution domain service via engager matching | FUL-1336 |
| FUL-1072 | Profile retrieve: P0–P3 scarcity watermarks | Expose post attribution via metrics API and web client | FUL-1337 |
| FUL-1073 | Adaptive account-level magnet comment poll scheduler | Post performance table: Impressions, Engagers, Leads, Conversations, Calls | FUL-1338 |
| FUL-1074 | Typed action buckets + wire writes | Engaged-but-not-connected domain service and endpoint | FUL-1339 |
| FUL-1075 | Governed Unipile wrappers + enforceable call-site boundary | Engaged-but-not-connected panel with add-to-campaign action | FUL-1340 |
| FUL-1076 | Goal: Warm engager precedence | Pipeline-first inbound headline | FUL-1341 |
Canonical dataset
Computed by applying these rules to the 2026-10-10 snapshots (a dry run in memory); the T-1 rehearsal recomputes it.
| FUL | MAI | |
|---|---|---|
| Work items | 868 = 751 identical − 5 test rows + 50 laptop-newer + 1 box-newer + 39 laptop-only + 16 laptop colliding + 16 box renumbered | 48 (box) |
Goals / Tasks (kind recomputed: children ⇒ Goal) | 87 / 781; equals the laptop's 013 kind column | 4 / 44 |
work_status | done 337, paused 278, todo 212, in_progress 33, cancelled 8 | todo 27, done 10, cancelled 5, in_progress 5, paused 1 |
Dependency edges (created_by = 'migration') | 452 active; 3 references not imported | 34 |
| Aliases | 16 | 0 |
| Runs | 129 (run_number 1–129) | 68 (1–68) |
| Item events | 2,038 + 16 renumbered written by the import | 161 |
events | 457 | 0 |
review_cycles / inbox_imports | 1 / 2 | 0 / 0 |
id_sequences | item 1342 (today), run 130 | item 49, run 69 |
migration_ledger | 5 test rows, event 525, 3 dependency references, 16 renumberings, 2 plan_inbox rewrites, source schema levels | source schema level |
Checks on the data: no run and no PR is attached to a Goal (I5, I6 hold);
27 FUL Goals are done or in_progress, so I8's CHECK stays deferred to
ADR-004 phase 3 as planned.
Treatment of each discrepancy
| Discrepancy | Treatment | Recorded in |
|---|---|---|
| 16 colliding FUL ids | Laptop keeps ids; box items renumbered as above | aliases, ledger, renumbered events |
| Rows on both sides with different content | Newer updated_at wins the whole row (51 FUL, 1 MAI). A three-way comparison against the common ancestor (/home/abdullah/fulreach-tasks.db, last write 2026-09-15 10:43) shows no true conflict: for the 50 laptop-newer items the box never changed the row; for FUL-1325 the laptop never did; FUL-999 and FUL-1001 changed on both sides (laptop 09-27, box merge sync 09-29) and ended identical. The T-1 inventory repeats this check; a true conflict stops the cutover | ledger (source chosen per row) |
| Runs 4 and 5 metadata | Laptop row (superset of the box's keys) | — |
| Rows on one side only | Imported (union) | — |
| Orphan item event 525 (FUL-759) | Not imported into work_item_events, whose item FK is not null; kept verbatim in the ledger. Its item exists in no copy, and the title "Main" suggests a test create | ledger |
Orphan events row 36 (FUL-207) | Imported: work_item_ref = 'FUL-207', work_item_id null | the row |
| FUL-307 → FUL-206, FUL-207 | Not imported as edges (FUL-307 is done; targets exist nowhere) | ledger |
| FUL-696 → FUL-692 | Decided at the T-1 inventory; the importer refuses to run without a recorded decision. Options: drop the reference (FUL-696 stays blocked by FUL-691 and FUL-693), or re-point it to an existing item | ledger + decision file |
Test rows PG-1..3, SP-1, NEW-1 | Not imported | ledger |
| Counters | Both SQLite values discarded (laptop 1116, box 1077/2000). Item counter = highest number + 1 per Product after renumbering; run counter = highest run_number + 1; the stray FUL row in the MAI files is dropped; the incident family starts with ADR-003 | id_sequences |
| Schema levels (FUL laptop 013, box 012; MAI laptop 011, box 012) | The importer reads each source at its own level: requires ≥ 011_agent_run_retry; a missing auto_dispatch reads as 0 and a missing inbox_imports as empty; the laptop's kind column and phase 2 triggers are ignored, then kind is recomputed and compared with that column (must match) | ledger |
_migrations rows | Not imported | ledger (levels) |
issue_changes | Not imported; feed restarts above the highest source id | — |
autonomy_json (FUL-1026) | Carried as autonomy | — |
| Older copies | Contain nothing absent from the sources: archived off-site at window close | — |
Interim measures until cutover
Taken 2026-10-10, independent of this ADR's acceptance.
- FUL id partition (temporary). The box's FUL counter is raised from 1077
to 2000 (guarded
UPDATE … WHERE next_value = 1077, run asmainarkwith the coordinator running; rollback snapshot/opt/mainark/backups/fulreach-tasks-pre-counter-*.db). Until cutover the laptop allocates from 1116…1999 (skipping FUL-1325, which exists) and the box from 2000 up, so the two SQLite copies cannot allocate the same id again. The laptop counter is unchanged. This is a namespace partition for the SQLite coexistence period only, not part of the Postgres identifier design: at cutover the partition ends, the FULid_sequencesvalue is set by the rule above (highest FUL number + 1 after renumbering), and the unused gap below 2000 is not back-filled. Applied 2026-10-10 17:01 UTC after the backup below passed verification (1 row changed; verifiedFUL|2000; rollback snapshotfulreach-tasks-pre-counter-20261010T170120Z.db). - Box backup covers both databases.
mainark-backup.shnow backs upfulreach-tasks.dbas well asmainark-tasks.db(previous script kept asmainark-backup.sh.bak-20261010). Verified 2026-10-10 by a manual run and a restore into/tmp:integrity_checkok; row-level diff against a fresh snapshot of the live file shows no differences; the box's repository code reads the restored copy with results identical to the live copy.foreign_key_checkreports one row, orphan item event 525 (FUL-759created), which is pre-existing in the live file and handled by the rule above. Same-disk limitation: see Backups and recovery.
MAI authority handover (2026-10-10)
MAI had two SQLite copies with separate counters, like FUL. Instead of a partition, MAI moved to a single writer: the laptop. Done 2026-10-10, 17:21–17:23 UTC, before any new MAI item was created:
- Pre-check. No process held the box's
/opt/mainark/data/mainark-tasks.db(fuser,lsof; WAL empty, last write 2026-09-28). State as expected: 48 items, highest MAI-48, MAI counter 49. - Handover snapshot.
VACUUM INTO/opt/mainark/backups/mainark-tasks-handover-20261010T172121Z.db(asmainark):integrity_checkok,foreign_key_checkempty, sha256f1af9737…a70d435f9(identical to the reconciliation snapshot). Row count and per-table checksum (sha256 of all rows in rowid order) equal the live file for all 10 tables. - Transfer. Copied to the laptop; sha256, all 10 table checksums and the
MAI counter equal the box source. The stale laptop file (46 items, counter
47, sha256
c0e03841…c9e4a7fd6) was moved todata/archive/mainark-tasks.laptop-stale-20261010.dband the handover copy installed asdata/mainark-tasks.db. - Box file frozen in place (not renamed:
sqlite3in the backup would create an empty file at the old path).wal_checkpoint(TRUNCATE),journal_mode = DELETE, 30freeze_<table>_<op>triggers (BEFORE INSERT / UPDATE / DELETEon all 10 tables) raisingMAI authority moved to laptop 2026-10-10,chmod a-w. Frozen sha2564e0ec6b4…d9e157ef8; data checksums unchanged. Verified: writes asmainarkfail (attempt to write a readonly database); writes as root (which bypasses file modes) fail on the triggers; the nightly backup still reads it (48 items, 68 runs); the janitor's two read queries still run against it. - Migration 013 on the laptop copy via the repository's
migrate()(only 013 was pending; the box's011_autonomyrow came with the file). Result:kind= 4 Goals (MAI-1, 12, 24, 25) / 44 Tasks, as in the canonical dataset; every Goal has children and no Task does; 8 triggers andidx_issues_kindpresent. Apart from the new column, items are unchanged;issue_changesgained 4 backfill rows (the 4 Goals). - Pre-seed snapshot, then seed. A snapshot of the laptop file was taken
and restore-verified (
integrity_checkok, no FK rows, all 10 table checksums equal the live file, repository code reads it);scripts/seed-mainark-postgres-control-plane.tsran as a dry run against that exact file (MAI-49…100 planned, counter 101, source sha256 and table checksums unchanged) and was then applied. - Result. 100 items (12 Goals, 88 Tasks), MAI counter 101. Root Goal
MAI-49 "Control plane on Postgres (ADR-005)" with Goals MAI-50 (A, 11
Tasks), MAI-62 (B, 10), MAI-73 (C, 7), MAI-81 (recovery gate and
prerequisites, 5), MAI-87 (D, 3), MAI-91 (E, 3), MAI-95 (F, 5); 74
dependency edges, none dangling, no cycle; 9 Human Tasks marked "not for
agent dispatch". MAI-1…48 and their item events are identical to the
handover snapshot.
integrity_checkok,foreign_key_checkempty.
Consequences for the plan:
Transitional state (until the Postgres cutover, which makes Postgres the single authority for MAI and FUL):
- Sole MAI SQLite authority: the laptop's
~/mainark-data/mainark-tasks.db(schema 013), the MAI source for reconcile and import..env.mainark-selfsetsSQLITE_PATHto it directly. The file was moved there after the seed (table checksums equal before and after the move). - Compatibility symlink:
data/mainark-tasks.dbpoints to it, so scripts that hard-code the repository path open the authoritative file instead of creating a second, empty MAI database with its own counter. - Frozen box copy: the box MAI file is a frozen archive, not a source; its MAI-1…48 must equal the laptop's except for later laptop-side changes.
- Preserved, read-only, in
data/archive/(eachintegrity_checkok,foreign_key_checkempty):mainark-tasks-handover-20261010T172121Z.db: the handover snapshot, sha256f1af9737…a70d435f9, tables equal to the box source (the original stays in/opt/mainark/backups/);mainark-tasks.pre-seed-20261010T172302Z.db: the pre-seed snapshot, 48 items, MAI counter 49, tables equal to the live file just before the seed;mainark-tasks.laptop-stale-20261010.db: the stale laptop copy.
- No scheduled laptop backup. A nightly LaunchAgent was set up on 2026-10-10 and removed the same day: no operational infrastructure is built around the transitional SQLite file. Recovery requirements apply to Postgres through the recovery gate before SQLite is retired.
- The MAI figures in Canonical dataset and step 13 are the pre-handover baseline (48 items, 34 edges); the T-1 rehearsal recomputes them (today 100 items, 108 edges).
- No undo. The handover could only be reversed before the first new laptop MAI write; the seed was that write, so the laptop stays the MAI authority until cutover.
Cutover
Migration sequence
Each step says who runs it, what proves it, and what stops the cutover. The
sources are the four files in Copies; writers are the box's
mainark-coordinate (the only process holding fulreach-tasks.db open on
2026-10-10) and the laptop's CLI and dev daemon (the only MAI writer since
the handover). mainark-worker-janitor only reads: its unit sets
MAINARK_JANITOR_DB=/opt/mainark/data/fulreach-tasks.db (the script's
default is mainark-tasks.db); mainark-daily.service has no enabled timer.
The frozen box MAI file is not a source; T-1 checks it is unchanged.
Prerequisites (before T0 is scheduled)
- P1. Recovery gate passed (Recovery gate): a recovery strategy explicitly approved and tested. A Pro upgrade is one option, not a requirement. P1 blocks only step 15 (Postgres becoming the production authority); it does not block implementation or rehearsals, which run on the Free project.
- P2.
mainark db migratehas created schemamainark(empty tables) and rolesmainark_owner/mainark_app;mainarkis not in the Data API's exposed schemas; the security advisor is clean. - P3. Hourly
pg_dumprunning with an off-site copy; one restore drill passed. - P4. Phases A–C merged; box and laptop run release R0 (the last SQLite
release,
TASK_STORE=sqlite), which containsmainark db reconcile,import,export-sqliteand the fence commands.
T-1 day: inventory and rehearsal (operator; read-only on the sources)
- Snapshot the four sources with
VACUUM INTO(box files asmainark) into a dated directory;integrity_checkeach. mainark db reconcile --baseline <2026-10-10 baseline>: every change since the baseline must fall into an existing class (new item on one side → union; item advanced on one side only → that row; new runs/events → union). Stop on: a new id collision; a true conflict (three-way: both sides changed one item differently); a new orphan or missing dependency target; a box item < FUL-2000 or a laptop item ≥ FUL-2000 created after 2026-10-10 (partition broken); the frozen box MAI file no longer at sha2564e0ec6b4…d9e157ef8, or missing itsfreeze_*triggers (MAI authority broken).- Record decisions in the decision file: FUL-696 → FUL-692, and anything new from step 2. The importer refuses to run with an undecided item.
- Rehearse: full import of the snapshots into a local Postgres and into a
scratch schema on Supabase; every T0 validation check (step 13) passes;
export-sqliteround-trips; duration recorded. - Publish the provisional renumbering map and cutover report (renumbered items, the six PRs to link, excluded rows, FUL-696 decision); announce the freeze window.
T0: freeze and fence
- Stop writers. Box:
systemctl stop mainark-coordinate. Laptop: stop the dev daemon and every CLI session; no agent may run Mainark commands. - Prove it:
fuser /opt/mainark/data/*.dbandlsof data/*.db ~/mainark-data/*.dbon the laptop list no process. Per Product, no run isqueued/running, or each is recorded for re-attachment byexternal_run_id. - Fence each source:
PRAGMA wal_checkpoint(TRUNCATE),PRAGMA journal_mode = DELETE; record its sha256; createBEFORE INSERT / UPDATE / DELETEtriggers on every table raisingmoved to postgres;chmod a-w. A test write must fail. - Copy the fenced files as the final snapshots;
mainark db reconcileagainst the T-1 snapshots: differences are only writes made between T-1 and step 6, each in a known class (else stop → R1). - Compute the final renumbering map and counters from the final snapshots.
T0: import
- Check the
mainarktables are empty and the schema is at the release's version. mainark db import --sources <final snapshots> --decisions <file>in one transaction: Products and policies; work items (rules above, renumbering, recomputedkind); aliases; dependency edges; runs; item events (+renumberedevents);events;review_cycles;inbox_imports;id_sequences;migration_ledger; and the update ofpublic.plan_inboxrows 1 and 2 to the new ids.
T0: validation (inside the transaction before commit; any failure ⇒ rollback ⇒ R1)
-
- Per Product and table: source rows = imported + ledger exclusions
(today: FUL 868 items, 129 runs, 2,038 + 16 item events, 457
events, 452 edges; MAI 48, 68, 161, 0, 34). - Per-row checksums (canonical JSON, normalised timestamps) equal the row the rules chose.
- Every renumbering, alias, exclusion and dropped reference is in the
ledger; the 16
renumberedevents exist. - Recomputed
kindequals the laptop's 013kindcolumn for every laptop item (today 87 Goals). id_sequences> highest number per Product and family; thework_item_changesidentity starts above every sourceissue_changesid.- Constraints hold (enforced on insert).
- Golden diff per Product:
queue,next,listAutoDispatchReady, Goal views and run listings from R0 code on the merged SQLite export equal the new code on Postgres.
- Per Product and table: source rows = imported + ledger exclusions
(today: FUL 868 items, 129 runs, 2,038 + 16 item events, 457
- Commit.
T0: switch
- Deploy the Postgres release with
MAINARK_DATABASE_URL,MAINARK_PRODUCT,TASK_STORE=postgreson box and laptop; startmainark-coordinate. - Smoke test: one tick per Product,
mainark nextper Product, one test plan through the inbox in a scratch Product. The first production write to Postgres is go-live; record its time. - Send the cutover report (renumbered items, the six PRs to link and close, FUL-696 decision, excluded rows).
Window and close
- Rollback window (2 weeks): nightly
export-sqliteto the box; watch latency and query counts. - Close the window: delete the SQLite adapter,
TASK_STORE,SQLITE_PATH; move the janitor's checks to Postgres; archive the fenced files, snapshots and older copies off-site; move per-Product knobs intoproduct_policies.
Nothing is lost during the freeze: merged PRs are found by syncMergedIssues,
Cursor runs by polling external_run_id, plans wait in plan_inbox. Step 7
also confirms no webhook-driven writer is active (the events history shows
Linear webhooks were used before; none must be live at cutover).
Rollback sequence
| When | Steps |
|---|---|
| R1 — before go-live (steps 6–16) | 1. Roll back the import transaction; if it was committed, truncate the mainark tables and restore plan_inbox rows 1 and 2 from the ledger. 2. On each source: drop the fence triggers, chmod u+w; mainark db reconcile against the final snapshot shows zero differences (the sha256 from step 8 identifies the pre-fence file; R0 sets WAL mode again on open). 3. Start mainark-coordinate and the laptop on R0 with TASK_STORE=sqlite. The interim FUL partition still holds. |
| R2 — during the window | 1. Stop writers on both machines. 2. mainark db export-sqlite --all-products writes one SQLite file per Product in R0's schema, with merged data and new ids, counters equal to Postgres's, and a list of data dropped on rollback. 3. Validate: checksum round trip and golden diff. 4. One writer per Product after rollback: the exports go to the box, which becomes the only writer; the laptop gets read-only copies. If the laptop must create items, a counter partition like the interim one is set before it does. 5. TASK_STORE=sqlite; restart. 6. Postgres stays intact behind a read-only role for forensics. |
| R3 — after the window | No SQLite rollback. Recovery = hourly dump, Supabase daily backup, or PITR if enabled. |
Rollback compatibility window (2 weeks)
Development continues during the window; schema changes are limited to what the reverse export can carry to the last SQLite release (R0).
| Allowed | Condition |
|---|---|
| New nullable columns, new tables, new indexes, new views/functions | The exporter ignores them; the migration header declares "dropped on rollback: …", and losing that data on rollback is acceptable |
| New constraints | Existing data satisfies them, and they only reject writes R0 would also reject or never makes |
| New rows in existing tables | Representable in R0's schema and meaning (e.g. new Tasks, runs, events) |
| Feature code (e.g. phase 3 queue/orchestrator logic) | Its data obeys the rows above |
| Not allowed in the window | Why |
|---|---|
| Dropping or renaming anything the exporter reads | Breaks R2 |
| Changing the type or meaning of existing columns | Silent corruption after rollback |
New values in existing enumerations (e.g. work_status = 'merged', ADR-004 phase 4) | R0 cannot represent them losslessly |
Moving per-Product config from env to product_policies | Rollback would lose the change silently |
Every PR touching migrations during the window runs the round-trip test
(Postgres → export-sqlite → R0's validation and golden diff) in CI.
Implementation order
| A. Postgres first, then ADR-004 phases 3–5 | B. Phases 3–5 on SQLite, then Postgres | |
|---|---|---|
| Phase 3 | Built once | Mostly logic, little throwaway, but more sync call sites to convert |
Phase 4 (merged) | One CHECK change | Table rebuild with all triggers, then ported |
| Phase 5 | Built once; products/product_id in the port | Built per file, merged, ported |
| ADR-003 incidents | Once | Twice |
| Divergence between machines | Ends at cutover | Continues; collisions recur |
| Migration risk | Smaller schema and data | Larger |
| Time to next feature | Delayed by phases A–D | Immediate |
Recommendation: A. Phase 3 logic work may proceed in parallel if it adds no SQLite schema.
Phases
- A — Async repository boundary on SQLite.
Dbinterface; async repositories and callers; floating-promise lint; stray SQL moved into repositories. No behaviour change. - B — Postgres schema and adapters. Migrations (Products, policies, translated tables, dependencies, FK invariants, triggers); postgres.js and PGlite adapters; three test tiers; suite runs on SQLite and PGlite in CI.
- C — Importer, exporter, rehearsals.
mainark db reconcile(with the 2026-10-10 baseline), reconciliation rules and decision file as code, preflight, import, post-validation, golden diff, reverse export; Supabase roles, schema exposure, hourly dumps, restore drill; rehearsals on the Free project. - D — Cutover (sequence above).
- E — Rollback compatibility window (2 weeks).
- F — Remove SQLite; move knobs into
product_policies; then ADR-004 phases 3, 4, 5 on Postgres.
What this supersedes and amends
| Document | Change |
|---|---|
| ADR-001 | Storage decision superseded; offline hard requirement removed. TaskProvider boundary, Linear exit, parity checklist stand. |
| ADR-001 backend spike | "SQLite prior stands" superseded. Beads remains rejected. |
| ADR-004 | "One database per Product" → one database with product_id. "Storage names" → Postgres uses work_items (persistence name only). I2/I3/I5/I9 by composite FKs; I4/I10 by locked triggers; I8 by row CHECK; I12 by the dependency table. Phase 5's products/product_id and the dependency table move into this migration; phases 3–5 start after phase E. |
| ADR-003 | Postgres DDL; incident ids from id_sequences family incident. |
plan-inbox.md | Import is one transaction in the same database; service key retired; plans name their Product. |
dual-task-graphs.md | Retired. |
task-provider.md, http-api.md, local-cli.md, pipeline-state-mapper.md | SQLite setup and wording replaced by the Postgres store and MAINARK_PRODUCT. |
.env.mainark-self(.example), seed scripts | Replaced by Product rows and MAINARK_PRODUCT. |
Box: mainark-backup.sh (both SQLite files since 2026-10-10) | Replaced by hourly pg_dump; kept for the SQLite files until the rollback window closes. |
| Interim FUL counter partition (box from 2000) | Ends at cutover; Postgres id_sequences takes over. |
Box: mainark-worker-janitor.sh (reads fulreach-tasks.db via its unit, default mainark-tasks.db, with sqlite3) | Reads Postgres, or is retired, by window close. |
| MAI authority handover (laptop sole writer; box MAI file frozen) | Ends at cutover: the laptop MAI file is fenced at step 8 like the others; the frozen box file is archived off-site at window close. |
data/mainark-tasks.db symlink to ~/mainark-data/mainark-tasks.db | Removed with SQLITE_PATH at window close; the file is archived with the other fenced files. |
Consequences
- One truth for every machine and Product; id collisions and stale copies end.
- Mainark no longer works offline. Every command that touches work needs the database; a Supabase outage stops dispatch and the CLI.
- The Goal/Task model is enforced by the schema under concurrency.
- A large mechanical async conversion lands before new features.
- ADR-004 phases 3–5 and ADR-003 pause for phases A–E, then proceed on a schema that already has Products and normalised dependencies.
- Running cost: none while on Free; whatever the recovery gate chooses (possibly Supabase Pro); PITR later, if ever.
- The migration carries one-time data repairs that are recorded, not silent: 16 renumbered items, 5 excluded test rows, 1 orphan item event moved to the ledger, 3 dependency references not imported (FUL-696's decided at T-1), and a change feed that restarts at go-live.