What I built to find out what these systems can actually do.

Judging a technology from the inside.

I have spent twenty years around capital and industry, and one pattern has held throughout: you cannot price a shift you have only read about. Roughly once a decade something arrives that changes what a business can do, and the people who judge it well are the ones who got close enough to see the constraints for themselves — not the promises, the constraints.

By early 2026 that thing was agentic AI, and the honest position was that I did not know what it could do. I had circled it for years from a board seat, which is close, but not the same as having your hands on it. So I set a narrow objective: understand what these systems require, what they produce, and where they fail — well enough to tell a real change in a cost base from a vendor's story about one.

What I built was a multi-agent system that builds software — a demanding version of the question. I had never written a line of code.

A governed build system.

Orch takes a plain-language mission and produces working, verified software. The single working assumption underneath it, learned expensively: a model will write a correct 100–300 line unit of code in isolation, but multi-file builds fail at the seams between units. So the system manages seams rather than generation.

Missions decompose into self-contained tickets

Dependency-ordered, on the Auftragstaktik pattern — each ticket carries its purpose, its downstream consequence and its definition of done, so an isolated builder with no view of the wider build can still make a sound call. That idea is borrowed directly from military mission command.

Three build strategies, chosen deterministically

A sequential builder with cumulative context; a parallel set of workers that between waves re-scans the filesystem for what has actually been built rather than trusting the plan; and an architect-first pipeline for multi-subsystem work. Which one runs is a function of the mission's shape, not a preference.

Nineteen gates before any judgment

Compilation, tests, secret scanning, completeness, route wiring, auth headers, reachability. Deterministic checks run before a model is asked for an opinion, and the builder's own report of its work is never accepted as evidence.

A retrospective layer nearly as large as the pipeline

Twenty-seven modules reviewing every build's outcome, tracking recurring failure modes and feeding them into later work. It ended up nearly as large as the pipeline it serves.

The part that transfers.

The system is not the point; it was the instrument. What I wanted was judgment about a class of technology, and this is what two months of it produced.

The generation is not the hard part

Producing code is close to solved. Producing systems that hold together is not. Almost all real engineering effort moved to the seams — integration, contracts, and the question of whether a component is reachable at all. Anyone assessing an AI-built system should look there rather than at the code.

Context is the binding constraint, and it does not go away

Any system worth building is larger than any model's context window, which means no agent can ever see the whole of it. The obvious response — feed it more — is the wrong one: context bloat degrades the quality of the answer while raising its cost. What works is giving each agent a narrow, sufficient slice and a way to ask for more, rather than carrying everything. Keeping orchestration coherent across a system far larger than the window was the hardest single problem, and it took longer than I expected. It is also a familiar problem in another form — it is what running an organisation too large to hold in one head consists of.

Narrow purposes and grounded facts beat broad capability

This one I learned the expensive way and relatively early. Multi-agent systems produce markedly better and more reliable results when each agent has a deliberately narrow remit and is handed grounded facts, rather than a broad remit and the freedom to work things out. Breadth sounds like capability and behaves like ambiguity. The parallel to an operating company is close enough to be uncomfortable: a role defined by a specific outcome, with the relevant facts supplied, outperforms a capable generalist told to go and figure it out.

The clearest measurement I have: replacing a single sequential builder carrying cumulative context with parallel workers on narrow, self-contained tickets roughly halved build times — and the output got better at the same time. Speed and quality usually trade against each other. Here they did not, because the gain came from removing ambiguity rather than from adding throughput.

Quality has to come from the process, not the builder

A model asked whether its work is correct will tell you it is. The only useful architecture is one where a builder's self-report carries no evidentiary weight and the checks are deterministic, mechanical and run before any opinion is sought. This turns out to be the same discipline as a credit file: the claim is worth nothing without the provenance.

Most of the work is in the correction loop

The learning layer ended up nearly as large as the pipeline it serves. Where a system captures its own failure modes and feeds them forward is where the compounding actually lives. I would expect the relative weight of this to fall as base reliability improves — but not to disappear, because the correction loop is also where domain-specific knowledge accumulates.

Verification beats trust, structurally

The change that made this concrete: parallel workers re-scan the filesystem for what exists rather than trusting their own record of what they had done. Stated generally: never let a component's belief about the world substitute for looking at the world.

The cost that matters is re-doing, not generating

Building this way is cheap — not marginally, but by enough to change which projects are worth attempting at all. That much is not in question. The more useful finding is where the cost actually sits: tokens are cheap and wall-clock time is cheap, while work that has to be thrown away and rebuilt is neither. So the return on getting the specification right, and on narrow roles that produce usable output first time, is far higher than the return on making generation faster. The saving that mattered came from not having to repeat things.

What I would not rely on them for

Three things, consistently. Establishing that a fact is actually true where it matters — a system will assert rather than check, unless checking is forced out into a separate mechanical step. Holding coherence across a boundary the context window imposes — that failure is quiet, and it looks like plausible work. And remembering that a person has to use the result: no amount of sound engineering survives an interface a human cannot navigate, and it is the failure these systems are least equipped to notice on their own.

Capability has an edge, and the edge moves

Model capability is not a single number; it holds up to a certain size and complexity of system and then degrades, and the useful knowledge is roughly where that boundary sits for the current generation. Two months of pushing against it gave me a practical sense of where mid-2026 models start to break as a system grows — which is what lets you say that a given thing is buildable now, that another needs a person, and that a third should wait a release. That boundary will move, quickly. Knowing that it exists, and how to find it, will not.

The question moves from can to should

When building becomes cheap, the binding constraint stops being capability and becomes judgment about what deserves to exist. That is an allocation question, not a technical one — which is the part of all this that belongs to an operator rather than an engineer, and the reason I wanted to understand it directly.

Mostly, I rediscovered what I already knew

The honest summary. For weeks I was reinventing, from first principles, everything I had absorbed about how software actually gets built from years inside a software company — epic to ticket to build to test and validation, with some borrowed nuance from military mission command. Once I noticed that was what I was doing, I used that structure deliberately rather than rediscovering it a piece at a time. Which is the reassuring part rather than the deflating one: almost none of what I found is new. It is old engineering discipline arriving in a new medium — and that is precisely why I expect it to survive the models getting better.

Orch is not a product and never became one. There is nothing here to buy, no signup, no pricing and no waitlist. It was built by one person, over about two months, to answer a question — and once the question was answered I stopped developing it and kept using it as a working instrument, the way a workshop keeps the jigs it made for itself.

The code is not public. The technical descriptions here were checked against the source when this material was first written up, in July 2026, and anything that could not be checked was left out. Numbers of this kind age quickly, which is why this page is dated rather than maintained.

Written by Oscar. The longer story, and the person, are at interstice-advisory.com.