A deploy drains the resident fleet — admission closes to new runs, the runs in flight end, the swap lands, admission reopens
The ask. Decide (the maintainer, before the next release lands under working-hours traffic): adopt the fleet drain as the resident step's mechanism, and give the deploy workflow a drain-only bearer as its RESIDENT_DRAIN_TOKEN secret. Success: a release whose deploy starts with runs in flight on the resident lands within the longest of those runs' remaining leases plus a margin, kills none of them, and a run asked during the drain starts after the deploy with the wait named on its card.
TL;DR
Three consecutive releases of one evening deployed the bot and the memory Worker and then refused at the resident for their whole 30-minute wait — five waits, one hundred and fifty minutes — because the resident preflight waits for a minute with no run in flight, and steady traffic, with two to four runs in flight at every look and a new one landing every ten to twenty minutes, never has one. The bet is to make the wait end by construction: the deploy closes the fleet to new runs first (a drain, one record with an end in the registry Durable Object, read by the attach path), the runs already in flight finish and release, the preflight reads zero, the deploy lands, the fleet reopens. It costs a new write on the deploy path — a bearer in CI that can drain and undrain and nothing else — and, for a run asked during a drain, a wait of up to the longest in-flight lease at its attach instead of a start. Decided: the record, the routes and their scope, the gate and its exemption for runs in flight, the client's wait and the runner's drain; open: a card line while a run waits, and what the dispatcher does with a run whose lease cannot hold the wait. Doing nothing leaves every release split until a person waits for a lull or forces the swap over live runs.
Today at dc064a18
The delta from what a reader of the deploy specs expects, each with its proof.
- The resident preflight refuses on any run in flight and the runner waits for silence.
deploy/cloudflare-resident/preflight.mjsdeciderefuses while any resident'srunsInFlightis above zero;src/deploy/run.tsretries the wholenpm run deployevery minute forRESIDENT_WAIT_MAX_MS(30 min,src/deploy/plan.ts) and then fails by name (release-and-deploy item 13). Nothing stops new runs from landing during the wait. - The bot already drains itself; the resident cannot. On SIGTERM the bot closes its Slack socket and holds until in-flight runs finish or hand off (
src/index.ts,src/core/drain.tsDRAIN_DEADLINE_MS15 min). A resident Worker deploy swaps every ResidentDO isolate and invalidates the Sandbox SDK process handles of every run inside (resident-repos item 44): there is nothing to hand a running command to. - Every attach passes one registry, and every run in flight holds a registration.
ResidentRegistryDO(deploy/cloudflare-resident/worker.ts) is a singleton namedregistry; the attach handler resolves the thread's route through it, andGET /residentslists from it. Inside the resident,registerRunwrites arunReg:<threadKey>row at a successful attach andevictBindingclears it at the release, so the row exists exactly while a run is in flight (item 44's revision); a re-attach refreshes it. - The client already waits on a refusal it can read.
ResidentExecutor.attach(src/execution/resident.ts) waits through a refusal the Worker typed transient and through a rolling container (item 65), re-attaching on a budget and returning the wait aswokeAfterMs, which the factory prints on the card asafter waiting Ns for the resident. The run's remaining lease is read fromopts.remainingMs()at each attach it opens. - CI holds the read bearer only, and the Worker knows three scopes.
deploy-production.ymlpassesRESIDENT_READ_TOKEN;tokenScopeanswersadmin,operatororread, andhasScopepasses admin everywhere and every other scope only on its own routes. Admin can onboard, offboard, reconfigure and rebuild a resident.
The shape
The resident step becomes drain, wait, swap, reopen. Before its first attempt the runner posts /drain with the drain bearer: the registry stores { since, until, by, reason }, until sixty-five minutes out. From then the attach path answers a real 503 carrying that record to every attach for a thread with no run registration — a new run — while an attach for a thread whose run is registered — a run in flight re-attaching after a rolled container, an evicted worktree or a bot restart — passes as before; /exec, /read, /write, /op and /detach are untouched. So the runs in flight finish and release, the runner's preflight reads zero within a minute, the deploy uploads, and in a finally the runner posts /undrain. The bot's client, meeting the 503, re-attaches every 30 s under the run's own lease less a ten-minute reserve and returns the wait on the binding, so the card's opening note names it. The closest known shape is a load balancer draining a node before a rolling restart: no new connections, the open ones finish, restart, route again. The difference is who asks and who refuses: the deployer asks a registry the callers already consult, the registry refuses at the door, and each caller waits by itself under its own clock, because the deployer is outside every caller and cannot hold their threads.
deploy all registry DO ResidentDO (per repo) bot
│ POST /drain ───────►│ drain = {until}
│ │ new run's attach (no runReg row) ◄──────┤ POST /attach
│ │◄─ getDrain ──────┤ │
│ ├─ record ────────►├─ 503 draining {until} ────────────────►│ wait 30 s, retry…
│ │ in-flight run's re-attach (runReg row) ◄┤ POST /attach (container rolled)
│ │ ├─ 200 binding (registered: passes) ────►│ run continues
│ GET /residents (1 min) ──► runsInFlight: 2 … 1 … 0 (in-flight runs finish, /detach clears their rows)
│ wrangler deploy ─────────────────────────► isolates swapped, nothing inside
│ POST /undrain ──────►│ drain cleared
│ │ ◄──────────────────────────────────────┤ POST /attach (the waiting run)
│ │ ├─ 200 binding (wokeAfterMs = the wait) ►│ run starts; card notes the waitOne trace: the third release of that evening, replayed with the drain
At 04:44 the resident step begins with two runs in flight on the switchboard resident: a coding child under a 45-minute lease that started at 04:20 (25 min left) and a review that started at 04:41 (about 8 min to go). The runner posts /drain for 65 minutes; the registry stores until 05:49. Two runs arrive during the drain. At 04:46 a person asks for a coding change in a fresh thread: the attach finds no runReg row, meets the 503, and the client re-attaches every 30 s under its lease — a 45-minute ask less the 10-minute reserve, 35 minutes of wait. At 04:50 the coding child's container rolls under it (the platform's own restart); its wake wait re-attaches, finds its runReg row, and is admitted through the drain: the run that holds the fleet open is never refused by the drain that waits for it. The review in flight finishes at 04:49 and releases; the preflight reads one. The coding child pushes, writes up and releases at 05:07; at 05:08 the preflight reads zero and the deploy uploads; the swap lands on empty isolates; the runner posts /undrain at 05:09 and logs fleet reopened. At 05:09:30 the waiting coding run's re-attach is admitted; its binding carries wokeAfterMs of about 23 minutes and its card opens with after waiting 1410s for the resident. Nothing was killed. Without the drain, the same step refused thirty-one times until 05:30 and deployed nothing, because a third run had started at 04:52.
The second arrival is the one that shows the open question. At 04:52 a person asks for a review under a 25-minute ask: 15 minutes of wait in its lease, and the fleet reopens 17 minutes later. At 05:07 that run ends refused with ResidentDrainingError naming the 900 s it could wait and the drain's stated end, and the person re-asks after 05:09. The wait is correct; what the dispatcher does next is not yet designed (Open questions).
The difficulty map
- New run or run in flight — the drain must refuse the first and admit the second through one door, or it refuses the run it waits for. → The gate.
- The record outliving the deploy — Durable Object storage survives the swap; a dead runner would close the fleet for good. → The record's own end.
- A run asked during the drain — it must wait legibly under its own clock, never fail at once and never start on the old generation. → The client's wait.
- The arithmetic — the drained wait must outlast the longest lease a run in flight can hold, or the drain still loses. → The numbers.
- The wiring — five files across three packages, a fourth bearer; the most work, the least risk. → Validation.
The gate
The constraint: every attach comes through one route, and the ones the drain must let through are exactly the ones that matter most — a run whose container rolled mid-drain re-attaches to continue (item 65), a run whose worktree was evicted re-attaches to recover, a run resumed after a bot restart re-attaches with reuse. Refusing any of them ends a run the drain exists to protect, and holds the fleet closed on it. The attach body cannot say which kind it is: a resumed run says reuse, a rolled one says nothing.
The design: the resident already knows. registerRun writes runReg:<threadKey> at a successful attach and the release clears it, so a thread whose run is in flight has the row and a thread starting a run does not. The gate therefore lives in the Durable Object's attachThreadTraced, after hydration and before the image reconcile: read the drain from the registry; when it is in force and the thread has no runReg row, answer the streamed refusal { error: "draining: …", status: 503, draining: <record> }; otherwise proceed exactly as before. The Worker-level handler gates nothing, because the decision needs the thread's row. The client reads the drain by the draining record on a 503, never by the words, so it is a third kind of answer beside the platform's transient (item 27) and a deterministic refusal, and it is never mistaken for either.
Invariants: no attach for a thread with a runReg row is refused by the drain; every attach for a thread without one is, while the record is in force; the row is written only by a successful attach and cleared only by a release. Failure modes: a registration whose release never came (the clean-idle sweep's case) lets one new run of that thread through the drain — thread-scoped, rare, and the sweep clears it; the drain lifts while a re-attach is in flight — the re-attach is admitted, nothing is lost; the registry cannot be read at the attach — that is no drain, said in the log, because a run must never fail on a flag it could not read, and the deploy's own preflight fails closed on its side (an unreadable fleet refuses the deploy), so the failure lands on the deploy and never on the run. The alternative rejected: a flag on the attach body naming a re-attach. The client would have to know at every attach site whether it is continuing a run, the sites are five, and a lie (a fresh run claiming to continue) would pass the drain; the row is a fact the resident holds and the caller cannot forge.
The record's own end
The constraint: wrangler deploy swaps the isolates but not the storage, so whatever the registry holds at the swap it holds after; and the process that asked for the drain — a GitHub Actions job — can die at any line, the finally included (a cancelled job, a runner lost). A flag would then close the fleet until a person noticed that every run's card said attaching the workspace… forever.
The design: the drain is never a flag. parseDrainRequest refuses a request without a finite minutes in [1, 90] and writes until; liveDrain(stored, now) reads a record whose until has passed as no drain, and so does every route, so an expired record needs no sweeper and no clock in the registry. The runner still posts /undrain in a finally because a deploy that is over should not cost a waiting run the five-minute margin — and it posts it whenever a /drain was asked for, not only when the answer said the record landed: an answer lost after the registry's write would otherwise close the fleet for the record's whole life. /undrain is idempotent. The cap of 90 minutes is the longest a forgotten drain can cost anyone.
Invariants: no attach is ever refused by a record whose until is in the past; a record this build cannot read is no drain; a second /drain replaces the first (two deploys overlapping extend, never stack). Failure modes: the runner dies after /drain — the fleet reopens at until, at most 65 minutes after the drain began, and every waiting run either is admitted then or has already ended refused naming that time; the clocks of the Worker and the runner disagree — until is written by the Worker's clock and read by the Worker's clock, the runner only reads it back for its lines. The alternative rejected: a flag cleared by the new isolate on its first request. It would reopen the fleet the instant the swap landed, which is right only when the deploy succeeded; a refused or failed deploy would leave the flag standing, and the failing case is the one that needs the safety.
A run asked during the drain
The constraint: the bot has no queue for a thread — a run is provisioned on its request, and the attach is the first thing the resident does for it. The deployer is outside the bot and cannot hold the thread for it. The wake wait (item 65) exists for a container that is coming back within seconds, on a three-minute budget; a drain lasts as long as the longest run in flight.
The design: the draining answer is waited for under its own budget: the run's remaining lease (opts.remainingMs(), never negative, clipped to zero) less a ten-minute reserve — the attach itself may clone and install, and a run that would start with less has nothing to start for — and sixty minutes with no lease (the CLI, staging). One re-attach every thirty seconds; the run's stop ends the pause and the request alike. An answer that is no longer the drain is judged exactly as a first answer would be. Admitted, the run starts. The platform's transient, the wake wait takes over with what the budget left. Anything else, the attach's own refusal, at once. The wait rides the binding as wokeAfterMs — the same field a wake wait uses, so a drain followed by a wake on the same attach is one total — and the factory's existing note names it without a new field. When a drain answer is met inside a wake wait's own re-attach, the wake wait keeps waiting rather than ending.
Invariants: a waiting run never starts on the old generation — its attach is refused until the registry says the drain is over, and the drain is over only after the runner lifted it or its end passed; the wait never exceeds the run's lease less the reserve; the stop wins within one poll. Failure modes: the lease cannot hold the wait — the typed ResidentDrainingError (refused) names the wait and the drain's end, and the run ends refused (the open question below). The alternative rejected: the bot reads /residents before provisioning and holds the request itself. It would let the card say why at once, but it puts the drain's semantics in every caller instead of at the one door, and the CLI, the load harness and a second bot generation mid-roll would each need the same code.
The numbers
DRAIN.deployWaitMaxMs is 60 minutes: the longest lease any resident run can hold is a coding child's 45-minute ask — the write-up reserve is inside it, not added, and a ship child is a coding run whose lease the ship parent carves from its own remainder and never renews past it (record 0046), so no run in flight outlives the drain's start by more than 45 minutes — plus fifteen minutes of slack for the preflight's one-minute poll, the swap itself and a run that ended late. The drain itself is that wait plus a five-minute margin (marginMinutes), so the fleet stays closed through the deploy and reopens by itself soon after if the runner died; the resident's wrangler deploy has not been timed under a drain, and the first drained release measures it against the margin. The registry caps a drain at 90 minutes (maxMinutes) so a forgotten one is an hour and a half, not a day. The client polls every 30 seconds (pollMs) — six times the wake poll — because the thing it waits for is minutes long and each poll is a request the resident answers before any work. The arrival rate the wait is sized against is the evening's: two to four runs in flight at every look and a new attach every ten to twenty minutes on the busiest resident, so a 25-minute drain meets one to three new runs; a coding ask (45 min, 35 of wait) rides it out, and a review ask (25 min, 15 of wait) ends refused whenever the drain runs past a quarter of an hour — the count to watch in run history. Undrained, the step keeps its 30 minutes: waiting longer for a quiet minute buys nothing, which the third attempt of the first release showed at 90 minutes of retry budget. Every one of these numbers lives in src/core/budgets.ts DRAIN, the one clock table (record 0046).
Why not
- Wait longer than a lease (the sibling issue's first shape). The third attempt of the first release waited while three to four runs were in flight the whole time and new children started as old ones ended; a wait for silence has no length that wins under steady traffic.
- Roll one Durable Object at a time. A Worker deploy is atomic across every isolate.
- Deploy off-hours. Releases are cut by merges all day, several a day; a person timing them to a lull is the manual step this record removes.
- Make the swap safe and resume the runs. The run ledger's handoff is a recovery a harness can refuse — the deploy-holds change exists because a release lost two children to exactly that — not a guarantee a deploy may lean on.
- Drain from the bot. The bot is one caller of the fleet, and the deploy runs outside every bot; the registry is the one place every attach already passes.
- Queue the request in the bot. There is no queue for a thread today; building one for this case alone is a larger design than the wait, and the wait already keeps the thread's order.
- Give CI the admin bearer. It would save minting one token, and hand a leaked CI secret the power to offboard and rebuild residents; the drain scope passes two routes and nothing else, so a leaked one can close a fleet — for 90 minutes per post, and for as long as its holder keeps posting until the token is rotated — and can never offboard, rebuild or read one.
What would change our mind
If drained waits still reach 60 minutes often, the lease arithmetic is wrong for the runs that actually land (measure: the waited Nm of 60m heartbeats in deploy logs across ten releases). If runs ending refused with ResidentDrainingError are common, the dispatcher needs the re-issue below before the drain is worth its cost (measure: count that error class in run history over a week). Both are reversible: without the secret the step behaves exactly as before, and the record expires by itself.
Rollout
Two human steps, both secrets: mint a drain bearer and put it on the resident Worker as RESIDENT_DRAIN_TOKEN (the secrets manifest lists it; optional, so a Worker without it simply has no drain scope), and give the deploy workflow the same value as its optional RESIDENT_DRAIN_TOKEN secret. Until then every release logs RESIDENT_DRAIN_TOKEN is not set — waiting without a drain and behaves as today. --force still deploys over live runs with its warning and neither drains nor waits; the CLI's deploy all on a laptop drains too when the bearer is in its environment. The first release after the secrets is the receipt: the resident lines of its deploy job and the resident's /healthz at the end.
Open questions
- A card line while the run waits — today the card shows
attaching the workspace…for the whole wait and names it only once the run starts. Owner: the next unit on the resident client; resolves when the factory can note a wait in progress. - A run whose lease cannot hold the wait — it ends
refusednaming the drain's end. Whether the dispatcher re-issues the request when the fleet reopens (as it does after an interruption) is a decision for the dispatch specs. Owner: the maintainer; resolves with the first live occurrence.
Validation criteria
Bound to their proofs in the living specs: resident-repos item 69's rows (the record, the refusal, the wiring with the gate's exemption and the drain scope, the client's wait) and release-and-deploy item 31's rows (the runner's drain, the plan and the workflow, the live receipt).
Amended 2026-09-18 — a pending bot deploy drains the bot too: admission closes, a new ask is queued with its wait named, the runner's spawn idles
Re-evaluation, written while the record is still proposed, because tonight's release replayed the failure this record fixes, one level up. The record was written for the resident step: its preflight waits for a minute with no run in flight, steady traffic never has one, so the drain makes the wait end by construction — new runs are refused at the fleet's one door, the runs in flight finish, the count reaches zero. The bot step has the same wait with no door at all: deploy/cloudflare/preflight.mjs refuses while the bot's /healthz reads inFlight > 0, src/deploy/run.ts retries the step every minute up to the plan's 30-minute budget, and nothing stops a new run from landing on the bot while it waits. Under a plan runner the arrival rate is not even a person's: the runner spawns the next unit's child the moment the previous one ends. This amendment extends the record's bet to the bot itself: a release deploy waiting on the bot preflight closes the bot's admission first — a new ask is answered "deploy pending, N minutes" and queued, and the plan runner's next-unit spawn idles instead of starting — the runs in flight finish, the roll lands, admission reopens, the queue and the idled spawn proceed.
The trace (tonight's release). The bot step began with one run in flight: a plan unit's review. The review ended, and the next preflight poll would have read zero — but the plan runner, holding the merge and the rest of the plan, spawned the next runnable unit's coding child (unit five's review ended, unit seven's child started) within seconds of the review's release, before the preflight's next one-minute poll. Every later poll met one or two runs in flight for the same reason: the quiet gap the preflight needs is a minute, the gap the runner leaves is seconds. The preflight never read zero, the step gave up at its budget's end by name, and the release landed split — the memory Worker on the new version, the bot on the old — until a person re-ran it in a lull. The resident step of the same release, drained under this record, landed.
The seam, three doors. The pattern is unchanged — drain, wait, swap, reopen — but the bot's callers are not the resident's, so the drain lands on three seams:
- The bot preflight's record, written by the deploy runner. Before its first bot preflight the runner posts a deploy-pending record to the bot —
{ since, until, by, reason }, the same shape and the same self-expiring end as the fleet drain's (DRAIN.maxMinutescaps it, an expired record is no drain, a second post replaces the first) — under a drain-scoped bearer, and lifts it in the samefinallythat undrains the fleet./healthzcarries the record, so the preflight, the dispatcher and a person read one truth. A record the bot cannot read is no drain for an ask and a refused deploy on the runner's side — the failure lands on the deploy, never on the run, exactly as the fleet gate chose. The record does not survive the roll it exists for, and should not: the new generation starts with admission open, and the runner's lift plus the record's own end cover a runner that died. - The dispatcher's admission. The admission stage of the dispatch pipeline is where one-live-run-per-thread is already decided; it gains one earlier question: is a deploy pending? A new ask while the record is in force is answered at once — "deploy pending, N minutes", N read off the record's
until— and queued durably on the ledger's inbox; when admission reopens, the queued asks dispatch in arrival order as if they had just arrived. A message for a run in flight passes as before — a steer, a stop, a follow-up into the run are the run in flight, exactly as a re-attach passes the fleet drain. Unlike the resident's attach, which can only refuse with a 503 the client waits on, the bot holds the person's channel: the wait is said the moment it starts. That legibility is why admission is the seam and not the executor below it. - The plan runner's spawn refusal. The runner is the caller that defeats the preflight — it spawns within seconds of an ending — and the one caller that can wait without losing anything: the next unit has not started, so there is nothing to kill and nothing worth queueing. Its next-unit spawn reads the same record and idles until admission reopens, the wait named on the unit's card, bounded by the record's
untilplus the drain margin. Refusing at the spawn rather than letting the queue hold it keeps the runner's clock honest: an idling runner spends no child lease on waiting, and the preflight sees the quiet minute the moment the runs in flight end.
The arithmetic holds one level up. The bot's runs hold the same leases the fleet's do — the longest is the coding child's — so the drained wait is the same DRAIN table: the wait outlasts the longest lease a run in flight can hold, the record outlasts the wait by the margin, and the cap bounds a forgotten record at 90 minutes. The bot step's drained wait extends past the plan's 30 minutes exactly as the resident's does.
Against the record's reasoning. (1) The gate's invariant translates whole: no run in flight is touched — its follow-ups steer, its turns run — and everything new waits; the fact that tells them apart is the bot's own thread admission, which it already holds, so nothing is forged. (2) The record's own end holds unchanged: never a flag, until on the record, the lift in a finally, the lift idempotent. (3) "A run asked during the drain waits legibly under its own clock" improves: the bot answers the person at once, which the resident's attach path could not. (4) The numbers move nowhere and no new constant leaves src/core/budgets.ts. (5) The why-nots hold; the one this amendment revisits is "queue the request in the bot" — rejected there because a queue for one case was a larger design than the wait. Here the queue is not built for the case: the ledger's durable inbox already carries a thread's messages across a generation, and the queued ask rides it.
The units. Four, in the plan: the record and its route on the bot with /healthz carrying it; the admission answer and the durable queue with its reopen dispatch; the runner's idling spawn with its card line; the deploy runner's wiring — post before the first bot preflight, the extended wait, the lift in finally — with the first drained bot roll as the live receipt.
Open. What happens to a queued ask when the record's until passes with no roll (a deploy that failed and will be retried): re-answered with the new wait, or dispatched into the window. And whether deploy restart — the same preflight, no version change — closes admission too or keeps today's wait; its first collision decides.
Amended 2026-09-20 — the fleet reopens after the containers reconcile, never at preflight-ok, and the reconcile never restarts a container under a registered run
Re-evaluation after the record's first releases: the drain held the swap for the runs in flight, but the lift was posted seconds after preflight-ok — before any container had been reconciled onto the new image — and a run admitted in that window was killed when the shared container restarted under it; the reconcile that followed restarted the container under another thread's live run. Three corrections, each its own seam: (1) the deploy runner posts POST /reconcile (drain scope) after a deployed step and before its /undrain, so every resident's Durable Object reconciles its container onto the new image inside the drain window — the order is drain, wait, swap, reconcile, reopen (release-and-deploy item 31, resident-repos item 69); a failed reconcile never holds the lift, and a failed or refused step posts none. (2) The image reconcile itself reads the run registrations beyond the op counters and defers its restart while any is live, re-checked on every later attach and refresh until the resident is quiet (resident-repos items 16b and 44) — isIdle keeps the in-memory count alone, so a stale registration defers only the image restart, never the sleep, and only until the clean-idle sweep drains it. (3) A standing transport failure on a resident-backed run — registered from its attach to its release — is the harness's replaced verdict tagged transport, entering the relaunch's resume path (the re-attach through the resident's restore) instead of ending the run failed (harness-pi item 16).
Amended 2026-09-20 — the hold owns the cycle it asked for: RCA of the 34-minute fleet closure, and the reopen gate gains liveness
RCA, written after the reopen gate's first live deploy (1.256.0, the resident step of 2026-09-20) closed the whole fleet for 34 minutes until a person reopened it by hand (issue 2044). The gate shipped by the previous amendment held the lift while any container still reported the pre-deploy image, reopening on each container's new-image report — a fact, never a timer. The deploy logged fleet drained — /attach refuses new runs until 05:53:27Z, then fleet stays closed — the switchboard resident still reports the pre-deploy image; it reopens by itself on the last container's new-image report (backstop 05:53Z). The report never came: the platform never cycled that container (twenty minutes after the deploy repo list still showed it on its pre-deploy start), so the fresh post-deploy start the hold's marker waits for never existed to be probed. A repo rebuild made the resident warm on a fresh container — necessarily on the new image — but the provision path never reported through the reconcile's marker, so the hold stood on a fact that had already happened. Meanwhile every attach on every repo — six pipelines, a typed repo test, two reviews — waited at the drain behind a card that counted attaching the workspace…, a repo list that said warm, and a friction verdict of none. The maintainer reopened the fleet with POST /drain minutes=1 (replacing the held record) then POST /undrain.
The root cause is a hold without liveness. The gate asked every container to cycle (the reconcile's forced restart) and then waited on the cycle's report with no bound but the drain's own until — 65 minutes out. The ask is the gate's, so the outcome is the gate's: a cycle that has not landed within the cycle's measured bound (the platform replaces container processes three to ten minutes after a deploy — item 65's measurement) is not coming on its own, and waiting past it protects nothing while refusing everything. Three corrections, one design:
- The hold has liveness. The first hold stamps the record with
holdsUntil— the cycle's measured bound (DRAIN.cycleBoundMinutes, clamped tountil) — andliveDrainreads a record whose holds outlived it as no drain: the fleet reopens by construction, whoever died between the reconcile and the lift, and the registry's alarm names the stale containers in a warning (each restarts on its next quiet attach or refresh). The deploy's held line says so up front — the reopen on each report, the bound, the backstop. A rebuild or fresh provision counts as the report: provisioning reports its own fresh container current, since it runs on the image the deployed Worker pins by construction.untilstays the last resort, for a record from before the bound. - A run admitted onto a drained fleet says so. The client's drain wait paints one line in user words on the card —
waiting for the deploy to finish · N min, refreshed on every poll — and the dispatcher publishes the wait as adrain_waitrun note, whichruns frictionreads as its own category (the fleet drain) instead ofnone, and the plane's table shows as the run's cause. - Readiness tells the truth.
repo listleads with the drain and marks every rowdrained — attach refusedbeside each container's image report (pendingnames a resident a held drain waits on), sowarmis never printed bare for a repository no attach can reach;/residentscarriesimageReportper resident; and a typedrepo teston a drained fleet is answered with the drain's own record at once instead of running or hanging.
What would have changed the incident. The rebuild at minute ten would have reported and reopened the fleet; failing that, the cycle bound would have reopened it at minute ten-to-fifteen with the switchboard container named stale; and every minute of the wait would have been legible on the cards, in repo list and in runs friction. The backstop remains what it was — the record's own end — and the gate's core bet stands: the reopen is a fact reported by the container, with the bound as the gate's own admission that a fact it asked for can fail to happen.