Agent supervision

Every agent CLI call the loop makes goes through cli/lib/agent-supervise.sh.

Why

The review loop's most common failure was never a wrong verdict. It was claude --print not coming back, or coming back with something the parser could not read. Both ended the run:

claude --print ... -p - <prompt.txt     # no deadline: a hang hangs forever
CLAUDE_EXIT=$?
if [[ $CLAUDE_EXIT -ne 0 ]]; then exit 1; fi   # no retry, no fallback
node parse-claude-verdict.mjs ... || exit 1     # prose instead of JSON = dead

A hang had no way out but Ctrl-C, which is not available to an unattended coordinator at 3am. This is the "the review loop doesn't always complete" symptom.

What it does

mainark_supervise wraps one invocation with:

A wall-clock deadline. Enforced without coreutils timeout, which macOS does not ship. At the deadline the process tree gets SIGTERM, then SIGKILL after a five second grace.
Outcome classification rather than a bare exit code.
Bounded retries with backoff, for the outcomes worth retrying.
A stricter re-prompt when the answer parsed badly — asking again identically tends to produce the same bad shape.
An attempt journal, written beside the verdict as .reviews/<key>.attempts.jsonl.

Outcomes

Outcome	Meaning	Retried
`ok`	exited zero and validated	—
`timeout`	still running at the deadline	yes
`transient`	rate limit, 5xx, reset connection	yes
`malformed`	exited zero, output failed `--validate`	yes, with a stricter prompt
`auth`	not logged in, 401, expired token	no
`budget`	hit `--max-budget-usd`	no
`error`	anything else non-zero	no

auth and budget are deliberately not retried. Both are deterministic: retrying spends money to be told the same thing. Budget is checked before the exit code, because it can arrive with a zero exit inside a well-formed envelope.

Killing the process tree

claude leaves children behind, so killing only the direct child leaves them holding the API session. The walk uses ps -A, and that flag is load-bearing: a bare ps lists only processes with a controlling terminal, so under systemd or a test runner it finds nothing and every child survives. There is a test for exactly this.

Configuration

Variable	Default	Meaning
`MAINARK_AGENT_TIMEOUT_S`	`900`	per-attempt deadline
`MAINARK_AGENT_MAX_ATTEMPTS`	`3`	attempts including the first
`MAINARK_AGENT_BACKOFF_S`	`5`	multiplied by attempt number; `0` disables

Fallback

When Claude cannot answer at all, claude-review-pr.sh falls back to Codex (mainark_codex_fallback review) so the loop keeps moving. Budget exhaustion is excluded: that is a spend decision for a human, not a reason to pay twice. Codex writes the verdict file itself, so the Claude output is not re-parsed over it.

Tests

cli/lib/agent-supervise.test.mjs drives the library with stub binaries that hang, spawn surviving grandchildren, fail transiently, emit prose instead of JSON, and report auth or budget errors. They run under both node --test and bun test.