Can I write a unit test that fully specifies the correct output for any given input?
Yes: no model. Stop here. The test refers to the output, not the input source — code that calls a database or a live API is still on the floor.
The 7 GenAI Architectures
A Field Guide to Choosing the Right AI System — Before You Overbuild It
Complexity must be earned by a demonstrated failure of the simpler architecture, and every level up has a measurable price you can compute before you pay it.
Most GenAI systems do not fail because the model was wrong. They fail because nobody decided the architecture.
Somebody in a design review proposed an agent. Nobody in the room could say whether it was warranted, so it got built.
That is not a bad decision. It is the absence of a decision procedure.
This book supplies the procedure.
Eight rungs. You climb one only when the rung below it has demonstrably failed.
Seven of them are architectures. The eighth, Level 0, is the floor: no model at all. Open a rung to see what it adds, what makes it warranted, and the reproducible case that earns the next one.
-
Level 0 Deterministic Code 0 tokens · 0 requests
- What it adds
- Nothing probabilistic; the baseline.
- Warranted when
- The Floor Test passes.
- What forces the next rung
- A payment incident reported in prose rather than by monitoring comes back
UNROUTABLE — needs a human, because: the report is prose, not an alert line. The missing capability, named: language understanding over an input whose correct output no test can specify. There is no rule to write.
Primary risk Brittle if the input space is genuinely open.
-
Level 1 The Prompt Application 1,225 tokens · 1 request · 1.00×
- What it adds
- Language understanding over the given input.
- Warranted when
- An input whose correct output no test can specify.
- What forces the next rung
- Nothing is misclassified. On the one incident where everything is right on the first try, the on-call engineer who receives it still has nothing to do next. The first line of the runbook for that exact symptom is in a markdown file in a repository the model has never read, and no prompt puts it there.
Primary risk Confidently wrong, correctly formatted.
-
Level 2 Retrieval-Augmented Generation 1,882 tokens · 1 request · 1.54×
- What it adds
- Facts outside the request and the model.
- Warranted when
- The answer depends on facts not in the request or the model.
- What forces the next rung
- A duplicate-email incident returns
UNROUTABLE — needs a human, because: no passage cleared the relevance floor. The runbook that resolves it was in the corpus the whole time. Nothing was misretrieved and nothing was misclassified — the retriever simply never returned it.
Primary risk Bad retrieval yields fluent, well-cited errors.
-
Level 3 The LLM Workflow 2,992 tokens · 2 requests · 2.44×
- What it adds
- Several model calls in an order you control.
- Warranted when
- One call cannot do it, and you can draw the steps.
- What forces the next rung
- The failure is not in the output, it is in what the output asks for. Every routed result in the measured run terminates in an instruction to go and read live numbers that exist nowhere in the system. Three for three. The corpus contains a document about those series; it cannot contain their values for the last thirty minutes.
Primary risk Errors compound across steps that cannot check each other.
-
Level 4 The Tool-Using LLM 7,174 tokens · 2 requests · 5.86×
- What it adds
- Live data, and actions with effects.
- Warranted when
- A step needs live data or an action outside the model.
- What forces the next rung
- The incident where checkout is down and nobody knows why routes correctly on 2 runs out of 15. Nothing is hidden and nothing is missing: the source is in the menu, the window is right, and the system searched it on every single run. Five search terms find the cause and one does not, and the model has no way to know which it picked.
Primary risk The model chooses, and side effects are real.
-
Level 5 Multi-Step Reasoning 16,250 tokens · 5.10 requests · 13.27×
- What it adds
- The model picks the next step, within a finite chain.
- Warranted when
- Which step runs next depends on the last result.
- What forces the next rung
- Not an incident — the step count. Five incidents need three different depths, and one of them gives two different answers to itself on the same input. Nothing in the report predicts which: the incident that says I have no idea where to start needs two steps, and the one that says no alert fired needs three.
Primary risk Reasoning drift past step-level assertions.
-
Level 6 The Autonomous Agent 20,158 tokens · 6.33 requests · 16.46×
- What it adds
- The system decides when it is done.
- Warranted when
- You cannot state a maximum call count upfront.
- What forces the next rung
- Authority. The tool menu is a property of the agent, not of the round — a loop is one node run repeatedly with one tool list. So an agent permitted to remediate carries write authority through every investigative round, including round two, on a premise that round five will overturn.
Primary risk Cost and latency have no ceiling you set.
-
Level 7 The Multi-Agent System 23,977 tokens · 8.50 requests · 19.57×
- What it adds
- Specialized roles with handoffs.
- Warranted when
- Measurement shows specialists beat one system.
- What forces the next rung
- There is no rung above it, so this receipt earns a correction rather than an architecture — and the correction is the last third of the book. The coordination multiplier runs the wrong way: overhead is heaviest exactly where the work is lightest, worst on the incident whose evidence belongs five rungs further down.
Primary risk Message passing as a new failure surface.
Level 7's measured row is a floor, because two of its three roles were never run live.
16.46×
Level 1 to Level 6, measured
The gap between the cheapest architecture and an autonomous agent, on the same problem, at an unchanged per-token price. Structure produced all of it.
Which is why the cost chapter teaches estimation in calls and context growth rather than in dollars: prices move on the vendor's schedule, call counts move on yours. The durable claim is the ratio, not the dollar.
The part you can use in an argument.
Run them in this order. The first one that fires settles the question.
Can you state, before the run begins, the maximum number of model calls it will make?
Yes: Level 5. No: Level 6. Level 5 executes a finite chain with a defined end; Level 6 runs an unbounded loop that decides its own termination.
What accuracy must a crew of specialists reach to pay for its own coordination?
Compute R, the crew's coordination cost over what one agent costs. The crew must reach agent accuracy × R, and no crew can break even above an agent accuracy of 1/R.
Eliminate from the top down.
The default outcome is the floor. You land higher only because a question forced you there, and the thing that forced you is written down.
| # | Question | If yes |
|---|---|---|
| 1 | Can a test fully specify the correct output? | Level 0 — stop |
| 2 | Has specialization been measured to beat one system? | Level 7 |
| 3 | Is the maximum model-call count unstateable upfront? | Level 6 |
| 4 | Does the next step depend on the last result? | Level 5 |
| 5 | Does it need live data, or an action with effects? | Level 4 |
| 6 | Are there several calls in an order you can draw? | Level 3 |
| 7 | Does the answer depend on facts outside request and model? | Level 2 |
| 8 | None of the above forced you upward | Level 1 |
Question 2 is answerable only with an A/B result in hand; absent one, the honest answer is no.
The same six columns in all eight chapters, so the tables compare across the book.
- Added p50 / p99 latency
- Tokens per request
- New dependencies
- New failure classes
- Ops burden
- Eval difficulty
Each rung chapter takes the same incident-triage problem one rung higher, shows the code, measures it against those columns, and closes with a Failure Receipt: a reproducible case the rung demonstrably cannot handle, printed as the actual failing output. That receipt is what earns the next chapter. Nothing here is climbed on a hunch.
To climb, you produce a Failure Receipt naming a missing capability, not a missing quality. If you sit down to write one and cannot, the finding is that you do not need to climb.
Who This Is For
You are deciding what to build next.
You have shipped at least one LLM feature. You can read Python, you have opinions about your stack, and you have sat in a design review where somebody proposed an agent and nobody in the room could say whether it was warranted.
You need to argue against a proposal.
The platform or staff engineer who needs something better than instinct. Half the value here is in the receipts and the tests, because those are things you can hand to someone who disagrees with you.
You do not need to have built a retrieval pipeline or an agent. You do need to be comfortable being told that a system you already shipped is a rung too high.
16 chapters, three parts, four appendices.
Part I · Diagnose
- 1 Why GenAI Systems Fail at the Architecture Layer
- 2 The Escalation Ladder
- 3 Level 0 — The Floor
Part II · The Seven
- 4 Architecture 1 — The Prompt Application
- 5 Architecture 2 — Retrieval-Augmented Generation
- 6 Architecture 3 — The LLM Workflow
- 7 Architecture 4 — The Tool-Using LLM
- 8 Architecture 5 — Multi-Step Reasoning
- 9 Architecture 6 — The Autonomous Agent
- 10 Architecture 7 — The Multi-Agent System
Part III · Operate
- 11 Composite Architectures
- 12 Costing an Architecture Before You Build It
- 13 Evaluating Each Level
- 14 Observability Across the Ladder
- 15 Climbing — Migrating Up Without a Rewrite
- 16 Descending — Rightsizing an Overbuilt System
Appendices
- A The Escalation Ladder Reference Card
- B Cost Model Worksheet
- C The Companion Repository
- D Glossary of Named Concepts
The Companion Repository
Every number in this book comes from code that runs.
One incident-triage system built across all sixteen chapters, one tagged increment per chapter. The test suite passes with no API key set — every figure in Parts I and III, and the structural half of every rung chapter, regenerates from committed recordings rather than from live calls.
The simulated environment is deterministic, so an example that worked when the book was written still works years later: six seeded incidents, twelve markdown runbooks, a fake metrics API, a synthetic log store, and the executable ground truth Chapter 13 scores against.
Where a figure could not be measured honestly, the book says so on the page rather than inventing one. There are several such places and they are marked.
github.com/ranjankumar-gh/escalation-laddergit clone https://github.com/ranjankumar-gh/escalation-ladder
cd escalation-ladder
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
pytestThis is not an argument against agents.
The top rung is built here, measured here, and given the fairest test its author could design. What this book argues against is complexity that nobody decided to buy.