Field guide · Operating substrate · July 2026
The door read open and admitted no one.
On July 30 we opened Groundwork's signup to the public: a 14-day trial, no card, started by adding a remote MCP server and clicking through OAuth. Every instrument we had said the door was live. The discovery document advertised the authorization flow. The health loop was green
Posted July 30, 2026
On July 30 we opened Groundwork's signup to the public: a 14-day trial, no card, started by adding a remote MCP server and clicking through OAuth. Every instrument we had said the door was live. The discovery document advertised the authorization flow. The health loop was green. The test suites were green in both repos. The deploy went out clean and the production tail showed no exceptions.
Then we walked the flow the way a stranger would, starting from nothing, and could not get in. Four distinct defects sat between the front door and the first successful read. Not one was visible from inside the system. Each surfaced only at the exact step of the journey where a real person stood on it, and each had been there as long as the path had existed, because the path had never once been walked end to end.
Here are the four, why every gauge missed them, and the checklist we extracted so the next door gets walked before it gets announced.
Defect one: the internal hop that hangs instead of failing
The authorization endpoint returned a generic failure the moment it should have created a session. The cause was nowhere near the OAuth code. A Cloudflare Worker fetching a public URL that is served by another Worker on the same zone does not error. It hangs. Our platform Worker and our connector Worker share a zone, so every internal session call from the platform died on its five-second abort timer and surfaced as a 503. The connector answers the same call from a laptop in 60 milliseconds. Its logs showed nothing, because the request never arrived at all.
The fix is a service binding: a direct Worker-to-Worker call that skips the public edge entirely. Over the binding, the same call completes in 14 milliseconds. A longer timeout was the tempting fix and the wrong one. The timeout was never the constraint, and widening it would only have made the hang slower to find. A missing binding is now named in the log as its own condition, because that case does not error either; it hangs, and five seconds later the timeout names the symptom instead of the cause.
Defect two: the internal seam demanded a header only the public edge sends
Fixing the transport exposed the next defect in the same request. A service-binding call carries no edge headers, because there is no edge in front of it. The OAuth router required one of those headers before any handler ran, since the per-source abuse budget derives from it. So the moment internal calls started arriving at all, every one of them was refused.
The requirement was right for the public surface and wrong for the internal seam. The seam is now authenticated by a shared secret and metered as one fixed source, which is semantically correct for a caller that is a single Worker.
Defect three: the account resolver did not believe in web trials
With the transport fixed, sign-in completed and the token exchange still refused: no account. The resolver that answers "which account does this person hold" consulted the membership roster and the invite list, and nothing else. A trial started from the web has neither. No roster entry, no invite. Every term missed, so a person holding a live 14-day trial was told they had no account.
Two production trials were in exactly that state when we found it. One belonged to the first stranger invited to try the product.
The fix made trial ownership a term in the resolution, read through the same query that already decides which trial belongs to whom, so the signup path and the customer portal cannot disagree about someone's identity.
Defect four: a gate no real client could satisfy
Past the account link, the first metered read refused as an invalid request. The trial's tool-call path required a metadata block carrying two product-specific fields. Groundwork publishes a plain remote MCP URL, and the clients that connect to one, Cursor, Claude, Codex, send no such metadata. A client that does send metadata sends its own keys. Every real client was therefore refused on its first read. Behind that wall stood a second: a workspace binding that only an internal provisioning path ever created, which a web-started trial never takes.
An enforcement you cannot hold against real traffic protects nothing and blocks everyone. The control now degrades honestly instead of failing closed. The fields are optional, a missing dedup key is minted per request, and the first metered call binds the workspace. A derived binding is recorded as derived, because it distinguishes no workspaces and the audit trail should not claim strength it does not have. An asserted binding that conflicts still fails closed.
The prequel: a suite that certified itself
The same day had already taught the lesson once. The trial storage shipped as several hundred lines of code whose migration created 21 columns while the query layer named 22, plus four tables that existed only in the test suite's in-memory schema. The tests were green because they certified the code against a database only the tests had. In production every authorization threw, a bare catch swallowed the throw, and the same generic 503 came back with nothing logged. The discovery document kept advertising the door. The health loop stayed green. The tail stayed silent. The fix was the migration the code had assumed all along, and the deeper fix was in the error handler: every 503 on that surface now names its cause in the log, including which configuration is missing by name and the name and message of any exception it caught.
Why every gauge stayed green
Four mechanisms, and none of them is specific to this stack.
- A test suite that provisions its own world certifies the code against itself. Green suites of 338, 550, and 394 tests were all telling the truth and none of it was relevant, because the schema, the transport, and the client shape in the tests were not the ones in production.
- A generic error at the boundary erases the cause. One handler returned the same bare 503 for a missing header, an incomplete configuration, and a thrown exception, and logged none of them. A total outage of the signup path was indistinguishable from a healthy deployment with a feature flag off.
- Health checks probe endpoints; journeys die between them. Every endpoint answered its own probe. The failures lived in the hops, the handoffs, and the state each step assumed the previous one had created.
- The defects live precisely on the never-walked path. Every account before this had been provisioned by hand. The first stranger was the first execution of the full sequence, so the full sequence had four defects waiting, one per step nobody had ever stood on.
The checklist
- Walk the flow as a stranger before announcing it. Fresh account, stock client, no shortcuts, in order. Armed and walked are different claims about a system, and only the second one means live.
- Make CI prove the migrations produce the columns the code names. A suite that builds its own schema in memory certifies nothing about production. Point the tests at the same migration files the deploy runs.
- Name every failure at the boundary. Log the cause before returning the generic message. A 503 that means five different things is a device for hiding outages.
- Never gate a public protocol surface on fields standard clients do not send. If the ecosystem's stock clients cannot satisfy a requirement, the requirement is not a control, it is a lock on your own front door. Degrade honestly and record the weaker signal as weaker.
- Give internal seams their own transport and their own authentication. Routing internal calls through the public edge imports the edge's assumptions, and same-zone loops can hang rather than fail. A binding with a shared secret is smaller, faster, and legible in the log.
What live actually means
After the fixes, the machine walks the whole sequence against production: session, account link, code, token exchange, first read. The suites now include tests that fail if any of the four requirements quietly comes back. The smoke test walks the signup path in order instead of probing each endpoint in isolation, because isolation is exactly where these four hid.
A person has still to walk it fresh, and that gate is a person on purpose. A door is not open because the sign says open. Live is a property of a walked path, not of a deployed artifact, and the only instrument that measures it is the first stranger through.