A run has one live state, owned by the server: a closed set from admitted to ended, one event the moment it changes, one projection and one wording function every surface reads

The ask. Decide (the maintainer, 2026-09-20, after issue #2101 — a coding child sat 18 minutes in "attaching the workspace" while it was only waiting for a deploy's drain): "it should be a lot more clear in the pipeline agents/runs when looking at the UI that it's waiting for deploy to drain; this is not a stall; refactor all the possible states and ensure there is a single way this is managed throughout the app and is driven by the server data." Written for an engineer who knows the dispatcher, the run registry and the trace. Success criteria:

1. One closed set of live states the server assigns: admitted → waiting for the deploy (detail: the drain's until) → waiting for the repository's container (detail: the refusal's words) → falling back to a sandbox (seeded or fresh) → preparing the workspace → working (the model turn or the tool call, with its bound) → waiting for the model provider (a held retry, bounded by the run lease's existing absolute expiry) → wrapping up → ended — every run passes through a subset, and the ended state carries record 0064's endings and causes, extended where a cause is missing, never a rival set.
2. The state is recorded as an event in the run's log the moment it changes, and projected onto RunSummary as one field: {state, since, bound, detail} — since when the state began, bound the moment the state itself promises to be over by (the drain's until, the wake wait's maximum, the run lease's absolute expiry for a provider wait, the tool call's declared bound), detail the words of the fact that put the run there. The assignment commits the event and the materialized projection to the durable live row before it becomes visible; a restart rebuilds from that row and the event sequence, never process memory.
3. Every surface — the Slack and web card, the run page's header and timeline, the runs index, the plane table, the unit page, runs get and runs list — renders the run's live condition from that one field, through one wording function beside endingCauseWords. The seven rival sources are deleted as sources of the live condition.
4. The state words join the vocabulary; internal words (drain, preflight, image gate, attach) never print; the drift check holds the line.
5. The judgements read the state, never the spans: runs friction's drain_wait and infra_failure categories are the waiting states' intervals, and the plane's stuck and steer watches compare now to the state's own bound — a run inside its bound is never "stalled".
6. No new waits, polls or timers are introduced to discover a state: every transition is an event some component already emits or can emit at the moment the fact arrives (the executor's refusal, the fallback's decision, the provider hold, the loop's turn boundary), on the event buses that exist.

TL;DR

On 2026-09-20, during the 1.258.0 deploy, Boris's coding child (run 55c2a001) opened dispatch.workspace.attach at 17:19:58Z and closed it at 17:37:59Z — 18 minutes 1 second, zero events in between — because the fleet was drained and then held by the image gate; the run then did its work in a seeded sandbox without complaint. For those 18 minutes every surface lied by omission: the card read ◑ coding · Nm (a timer, no cause), the plane's health column was empty, and runs friction afterwards counted drain_wait: 0 because the category is computed from events the wait never emitted. The run was never stalled — it was waiting for a deploy to drain, a state the system knew (the resident client was polling the drain's own until every 30 seconds) and printed nowhere. The bet: one closed set of live states, owned by the server, written to the run's log as an event the moment the state changes, durably projected onto RunSummary as {state, since, bound, detail}, and worded by one function beside endingCauseWords — every surface reads the one field, and the seven rival derivations of "what is this run doing" are deleted. Doing nothing keeps seven part-truths that agree only when nothing interesting is happening.

Today at c584a0ce

A run's live condition is printed from seven rival sources with no shared model:

Source	What it says	Why it fails the drain wait
Span display names (src/core/trace/displayNames.ts)	"dispatch.workspace.attach": "attaching the workspace" — the open span's noun becomes the activity line	A span is not a state: the drain wait, the container wait and the wake wait all hide inside one 18-minute dispatch.workspace.attach span with zero child events (run 55c2a001, seq 16 → 17, durationMs: 1081173)
The card's setup note (src/core/statusCardFrame.ts setSetupLabel; fed by onSetupNote in src/core/dispatch/provision.ts)	Free text pushed at the card while provisioning	Slack/web card only — no event, no record; the run page, the plane and runs get never see it; whether the drain note rendered on #2101's card is unverifiable because nothing was recorded
RunSummary.activity / inFlight (src/core/runRegistry/projections.ts)	The last tool call and the call without a result yet, with its declared bound	The last tool, not the wait: before the loop starts there is no tool, so the whole provisioning phase projects nothing
The shape buckets (src/core/runShape.ts)	Post-hoc partition: getting ready · thinking · in tools · finishing up	"getting ready" swallows every wait — a 6-minute drain wait, an 11-minute image gate and a 2-minute clone are one bucket, and only after the run ends
run_note kinds (src/core/runEvents.ts, RunNoteKind — about twenty)	Facts appended to the log (drain_wait, fleet_busy, sandbox_restarted, …)	Facts, no machine: each note is written after its episode by whoever noticed, nothing says what the run is in now, and #2101's wait wrote none
The plane's health words + PlaneEndingCause / endingCauseWords (src/core/plane/table.ts RunHealth, src/core/plane/decide.ts)	stalled, bound-exceeded, no-signal, … over live rows; one wording function over endings	The right shape — a closed union, one wording function — but endings and alarms only: there is no vocabulary for a healthy wait, so #2101 showed health: [] while the person saw a stall
Friction categories (src/core/runFriction.ts)	drain_wait, infra_failure, … computed from the run's events after it ends	Blind to a wait that emits no event: drain_wait counted 0 on #2101 — the category exists, the event never did

Each source was added to answer one surface's question; none owns the answer, and the one component that knew the truth minute by minute — the resident client polling awaitDrainEnd (src/execution/resident.ts) and the wake wait (src/execution/residentWake.ts, WAKE_WAIT_MAX_MS) — reports to none of them.

The shape

The set. One closed union of live states, in the order a run can pass through them (every run passes through a subset; only ended is mandatory):

State	Enters when	bound	detail
admitted	the plane admits the run, before the dispatcher reaches a workspace	the lease	the queue position when it had one
waiting for the deploy	the resident's attach is refused by a drain (resident-repos item 69)	the drain's own until, capped by the fallback rule below	the refusal's words
waiting for the repository's container	the attach is refused because the repository's container is restoring or restarting (the image gate, the wake wait)	the wake wait's maximum	the refusal's words (image-stale: …, attach-failed: …)
falling back to a sandbox	the executor gives up on the resident and provisions cold	the provision's budget	seeded (from the resident snapshot) or fresh (no seed found)
preparing the workspace	the workspace exists and setup runs (checkout, branch, deps)	the setup budget	the step
working	the loop runs	the in-flight tool call's declared bound, or the turn's	the model turn or the tool call
waiting for the model provider	a model transport failure parks the turn for retry	the run lease's existing absolute expiry (lease.endsAt); the park carries no deadline of its own	the provider's sanitized condition, never its gateway body
wrapping up	the loop ended, the record has not closed (the wind-down, the description, the handoff)	the wrap-up window	—
ended	the record closes	—	record 0064's ending and cause, extended where a cause is missing — never a rival set

Who assigns it. The server owns the state: the dispatcher and the run registry assign every transition they can see (admitted, falling back, preparing, working, the provider hold, wrapping up, ended). The two resident wait states are visible only inside the executor seam — the resident client is the component holding the drain's refusal and the container's refusal in its hands — so the resident client requests only those two transitions through a typed callback; the server validates, stamps and records them. The callback is awaited, and every request and server-originated assignment goes through one per-run serializer carrying an expected sequence: provisioning cannot advance until the wait assignment is durable, and an observation from an abandoned attach cannot overwrite a later fallback or preparation assignment. The client never renders anything, stamps since or owns the set. A client that dies mid-wait leaves the last state standing with its since and bound, which is exactly what the plane's watches need to judge it.

The event. Each transition is one event in the run's log, written the moment the fact arrives — the refusal answered, the fallback decided, the provider parked, the turn opened — never discovered by a new poll or timer. One server operation commits the event and the materialized {state, since, bound, detail} projection to the durable live row before it updates the in-process registry or returns to the source. The event sequence is the ordering fence; replay of that sequence is authoritative if a cached projection disagrees. The log stays the run's one history, so the run page's timeline is the state history for free.

The projection. RunSummary gains the one field {state, since, bound, detail}, and the same projection is materialized on the durable live row and final record so a cross-generation list and a cold record read do not need process memory. State events change state and since. Inside working, the already-durable tool_call and tool_result events refresh detail and bound without a second state event per tool call: a call uses its stamped time plus declared bound, and its result restores the turn's projection. The ledger append path updates the materialized projection in the same transaction as those events. A restart therefore preserves all four values, including the current tool; it never reconstructs them from the registry's volatile inFlight field.

The wording. One function beside endingCauseWords — the same module discipline record 0064 set for endings — turns {state, since, bound, detail} into the user's words: "waiting for the deploy to finish · until 18:21Z at the latest", "waiting for the repository's container · restarting", "falling back to a sandbox · from the repository's snapshot", "working · running tests · up to 10 min", "waiting for the model provider · run ends by 18:24Z". The nine state words join the vocabulary page as values of the run noun's live condition; internal words (drain, preflight, attach, image gate, parked) never print, and the vocabulary drift check holds it.

The deletions. The rivals are deleted as sources of the live condition: the card's onSetupNote free-text channel goes (the card renders the state field); the activity line stops being the raw span noun (the span names remain for the timeline's rows, where a span is the right unit); runShape's buckets remain the finished run's retrospective shape line and are never printed as a live condition; run_note facts remain facts and stop being any surface's guess at "now"; the plane's health words become judgements over the state and its bound rather than a parallel vocabulary; friction reads the state intervals from the log instead of hunting for episode notes.

One trace: run 55c2a001, 2026-09-20, minute by minute as it would render

Boris's coding child for the target repository, admitted during the 1.258.0 deploy (issue #2101's timeline). Left column the fact, right column what every surface — card, run page header, plane table, runs get — would say from the one field. What it actually said, the whole 18 minutes: ◑ coding · Nm.

Z	The fact	The state event and its rendering
17:19:58	dispatch reaches the resident; the attach is refused: the fleet drained at 17:16:59, refusing new runs until 18:21:58Z	waiting for the deploy · since 17:19 · until 18:21Z at the latest — the card says it, the plane's row says it with no health flag (inside its bound is healthy), the run page timeline gains the event
17:26:16	the swap lands; the fleet stays closed on the image gate; the retried attach is refused with the container's words	waiting for the repository's container · since 17:26 · "the container predates the current pool and is restarting" — the detail is the refusal's own sentence, worded for the user
17:34:30	the container's restore completes; the report is still pending; the attach still refuses	same state; since unchanged (17:26) — the person sees one wait of 8 minutes, not a fresh state per retry
17:37:59	the wake wait's bound expires; the executor gives up on the resident and seeds a sandbox from the repository's snapshot	falling back to a sandbox · seeded — a state, an event, ten seconds of it
17:38:0x	the sandbox is up; checkout, branch, deps	preparing the workspace · since 17:38
~17:40	the loop starts	working · the model turn, then each tool call with its bound as the detail — the projection's inFlight refresh, no log event per call
later	the loop ends; description and handoff land; the record closes	wrapping up, then ended — record 0064's ending and cause, worded by endingCauseWords as today

And the judgements, on the same field: runs friction reads drain_wait: 6m18s (17:19:58–17:26:16) and infra_failure: 11m43s (the container wait) from the state intervals instead of drain_wait: 0 from absent notes; the plane's stuck watch stays quiet the whole wait because now never passed the state's bound, and would have flagged bound-exceeded at 18:21:58Z had the drain overrun — the flag the person can trust because it is the exception, not the silence.

The difficulty map

1. The two resident wait states cross the executor seam (most consequential): only the resident client sees the drain's refusal and the container's refusal, so the transitions must travel from inside the seam to the server's log without giving the client ownership of the set or a side channel past the registry. The client requests, the server validates, records and assigns; a client that stops requesting leaves a state whose bound the watches judge. Getting this seam right is unit one's hard half.
2. working without a second event flood: the run already records every tool_call and tool_result. The projection folds those durable events over the last state boundary and materializes the result on the live row; adding a parallel state event for every call would double the stream without adding a fact. The hard part is transactional ordering: a call cannot start before its event and refreshed projection are durable, and replay must restore the same {state, since, bound, detail}.
3. Deleting seven sources without losing their surfaces: each rival feeds rendering paths with their own tests and spec rows (the card frame, the runs index, the plane table, friction). Unit two is mostly deletion, and deletion is where a surface silently goes blank — the cold-reader gate below exists for exactly this.
4. ended must extend, never rival, record 0064: the plane's PlaneEndingCause and endingCauseWords are the accepted ending model; the live-state union ends by pointing into it. A second ending vocabulary — even an innocent-looking one on the new field — recreates the disease this record cures.
5. Bounds without timers: a waiting state's bound comes from the fact that created it, except the provider park — which carries no deadline — reuses the run lease's already-recorded absolute expiry. The watches compare now to that bound on the events they already receive. No new timer, no poll — the constraint in force — which means any other creating fact with no bound (a refusal with no until) needs a stated default from the budgets module, not an invented clock.
6. The vocabulary cost: nine state words become user-facing values of the run noun, each needing a wording row, a vocabulary entry and the drift check's blessing; the refusals' raw words (image-stale, attach-failed) and the provider's gateway body are internal and must be translated at the boundary, never printed.

The hard parts

The witness is not the owner. The deepest asymmetry: the server owns the state machine, but for the two resident wait states the only witness is the resident client inside the executor. The design accepts that split rather than hiding it — the client requests the wait transitions it alone can see, and the server validates, records and projects them — because the alternative (the server inferring waits from the absence of progress) is a timer and a guess, both banned. The cost is a protocol: the awaited, sequence-fenced request path must survive the client's death (last state stands, bound judges it) and the deploy window itself (the state events of a run admitted mid-deploy travel the same seam the deploy is draining).

One field that is honest after a restart. The web derives from RunRecord; the card can be rebuilt from channel history; the registry dies with the process. The state field is therefore a fold over durable state boundaries plus the existing durable tool events, materialized on the live row and final record in the same transaction as the event that changes it. A rehost and a cold record read recover all four values; volatile inFlight may optimize a local update but is never authority.

The wording function's temptation. With one function rendering nine states on seven surfaces, every surface will want its own flourish ("the card needs it shorter", "the table needs it without the detail"). The line to hold, from record 0064's endings: the words are one function's; a surface may truncate or omit a part, never re-word it. The moment two wordings of one state exist, the drift check is the only thing standing between this record and the seven-source swamp it replaces.

Why not X

Why not fix the spans — give the drain wait its own child span? Issue #2101's live fixes do add the spans, and the timeline needs them. But a span is a duration with a name, not a state: it cannot say what the run is in now without a reader walking the open spans and guessing, which is exactly today's activity line. The span answers "what happened"; the state answers "what is happening"; both exist, one renders the live condition.

Why not push richer setup notes at the card? onSetupNote is the disease in miniature: free text, one surface, no event, no record — #2101 cannot even establish whether the drain note rendered. A state event is the note with a type, a bound and a history.

Why not extend the plane's health words to cover waits? The health words are judgements (something is wrong); a wait is a condition (nothing is wrong yet). Making stalled's siblings cover healthy waits would put the plane in the business of narrating every run's normal life, and the plane's table already carries the run rows — it should read the state like every other surface, and keep its words for exceptions.

Why not a per-surface mapping table instead of deleting the rivals? A mapping keeps seven sources and adds an eighth artifact that claims they agree. The incident's lesson is that part-truths do not compose; the fix is ownership, not reconciliation.

Boundaries

Out of scope, deliberately: issue #2101's two live units — bounding the drain wait by the fallback's own cost instead of the deploy's, and making the new-image report the fresh container's own start so an attach never restarts the container it cannot verify. Those change how long the waits last; this record changes whether anyone can see them. Also unchanged: record 0064's ending model and the plane's admission (this record's ended points into them), the trace and its timeline (spans remain the timeline's rows), the finished run's shape line (retrospective, never live), and the card's transport. A plane queue row is deliberately outside this set: it is a request waiting to be admitted, not a live run. Its queue position and conditions remain the admission answer until admitted, when the live-state history begins.

Rollout

Three units, in dependency order, one plan in the same pull request series as this record's acceptance:

1. Unit one — the server: the closed set, the state event in the run's log, the durable {state, since, bound, detail} projection on summaries, live rows and records, the wording function beside endingCauseWords, and the vocabulary rows; the resident client requests the two wait states only it can see through the executor seam, while the provider hold is assigned by the server.
2. Unit two — the surfaces: the Slack and web card, the run page header and timeline, the runs index, the plane table, the unit page, runs get/runs list all read the one field through the one function; the rival sources are deleted as sources of the live condition (onSetupNote, the activity line's raw span noun, the live use of the shape buckets).
3. Unit three — the judgements: friction's drain_wait and infra_failure read the state intervals; the plane's stuck and steer watches read the state and its bound, never spans.

The gate before acceptance is the cold-reader test: a fresh reader who sees only this document must restate the bet, name the hardest part, and find the first objection already answered. The implementation's later live gate is a staged drain: a person who knows nothing of the system sees each surface and answers "waiting for a deploy, until about N"; it fails on the first surface that needs private vocabulary. Run 55c2a001's timeline, replayed against the staged drain, is the fixture.

Sources

• Issue #2101 — the incident this record answers: the 18-minute silent attach, the timeline, the three defects (two of them the live units out of scope here).
• Run 55c2a001 (2026-09-20 17:19:58Z–17:37:59Z) — the hard-case trace's facts: one span, zero events, drain_wait: 0, health: [].
• Record 0064 — the ending causes and their one wording function, the pattern this record extends to the live side.
• Record 0066 — the vocabulary as the prime reference and the drift check the state words join.
• src/core/trace/displayNames.ts, src/core/statusCardFrame.ts, src/core/dispatch/provision.ts, src/core/runRegistry/projections.ts, src/core/runShape.ts, src/core/runEvents.ts, src/core/plane/table.ts, src/core/plane/decide.ts, src/core/runFriction.ts — the seven rival sources at c584a0ce.
• src/execution/resident.ts (awaitDrainEnd), src/execution/residentWake.ts (WAKE_WAIT_MAX_MS) — the waits' witnesses, reporting to no surface today.

Amended 2026-09-21 — the correctness review closes restart and ordering holes

Re-evaluation. The adversarial correctness gate was run against the document and the current tree at 1bec076e. Cross-model review was unavailable in the isolated coding workspace, and the attempted cold-reader pass came from the same run; that is not independent evidence from a genuinely fresh reader who saw only this record, so it does not close the acceptance gate.

Cold-reader gate — passed. The independent receipt was posted as comment 5766141180 on pull request 2140 at 2026-09-21 19:20Z after a fresh agent session was handed only this document at 9a904c88, with no repository, tools or other context. Its answer, verbatim:

1. Restate the bet. This record decides that a run has exactly one live condition at any moment, drawn from a closed, server-owned set of nine states running admitted → waiting for the deploy → waiting for the repository's container → falling back to a sandbox → preparing the workspace → working → waiting for the model provider → wrapping up → ended. Every transition is written as an event in the run's log the instant the causing fact arrives, committed durably together with a materialized {state, since, bound, detail} projection on RunSummary, and turned into user-facing words by a single function sitting next to the existing endingCauseWords. It replaces seven rival derivations of "what is this run doing" — open span names, the card's free-text setup note, RunSummary.activity/inFlight, the runShape buckets, run_note kinds, the plane's health words, and the post-hoc friction categories — by deleting each of them as a source of the live condition, while letting some survive in their honest roles (spans as timeline rows, shape as the finished run's retrospective line, run_notes as facts). The motivating failure is the 18-minute drain wait on run 55c2a001, where every surface either showed a bare timer or nothing at all, and friction later reported drain_wait: 0 because the wait emitted no event.

2. The hardest part. The hardest thing to get right is the executor seam for the two resident wait states, where the only component that can witness the fact — the resident client holding the drain's refusal and the container's refusal — is explicitly not the owner of the state machine. Everything else in the design flows one way (server sees fact, server records event), but here a request must cross a process boundary, be validated, sequence-fenced and durably committed before provisioning may advance, and it must behave correctly when the client dies mid-wait or when the very deploy being waited on is draining the seam the request travels. That combination — an awaited cross-boundary callback, a per-run serializer with an expected sequence, durability before visibility, and no timer allowed to paper over a missing message — is where a subtle ordering bug would reintroduce exactly the silence the record exists to remove. The document agrees, naming it "unit one's hard half" and "the witness is not the owner".

3. First objection. My first objection was that a state whose only witness is a remote client becomes a lie the moment that client dies — the run would sit forever displaying "waiting for the deploy" with nobody to move it on, which is a worse failure than today's empty timer because it looks authoritative. The document answers this directly under "Who assigns it": "A client that dies mid-wait leaves the last state standing with its since and bound, which is exactly what the plane's watches need to judge it." Success criterion 5 and the trace table close the loop by making the watches compare now against the state's own bound, so the stuck state is flagged as bound-exceeded at the moment its promise expires (18:21:58Z in the worked trace) rather than being trusted indefinitely. The answer depends on the watches seeing events they already receive rather than a new timer, which is stated as a deliberate constraint, not an oversight.

Verdict: passes

Adversarial correctness review. The review found three substantive holes and one boundary that needed to be explicit.

1. The closed set did not name the provider hold even though the plane already parks a live model turn, and its “working” bound could otherwise trigger a false stuck judgement. The set and wording table now include waiting for the model provider. The existing park carries only the run, provider and observation time and waits for provider_up or the run lease to end, so this state's bound is the current durable lease event's absolute endsAt; it does not invent a park-local retry deadline. The waits printed by the seven rival sources at head are now total: the deploy drain → waiting for the deploy; resident restore/wake/image reconciliation → waiting for the repository's container; sandbox capacity/provision → falling back to a sandbox; workspace mutex, clone, install and compose → preparing the workspace; model turn, tool call and diff join → working; provider park → waiting for the model provider; PR-description join and wind-down → wrapping up. The shape captions and friction findings are retrospective, not live states. The plane queue remains outside the set by an explicit boundary because its row is a request before admission, not a live run.
2. The old working asymmetry lost detail and bound on restart by treating the registry's volatile inFlight field as authority. The projection is now a fold over durable state events plus the existing durable tool_call/tool_result events, materialized on the durable live row and final record in the same transaction as the changing event. A restart therefore recovers {state, since, bound, detail} whole.
3. A fire-and-forget resident callback could race a server assignment after attach or fallback. The resident now requests only its two observable wait variants through an awaited callback; one per-run server serializer validates an expected sequence, durably commits the transition, then returns. A stale observation from an abandoned attach cannot overwrite a later server assignment.

With those corrections, the adversarial review found no uncovered current wait, process-local projection value or unordered writer left in the decision. The independent cold-reader receipt above closes the acceptance gate; the three-unit plan carries the tests that must disprove each claim before implementation stands.