Deterministic Rails: How to Build Reliable AI Agents

A multi-agent demo that works in a walkthrough and a multi-agent system that survives a thousand real requests are different products, even when the architecture diagram looks identical. The gap between them is rarely the model. It's how much of the pipeline was handed to the model in the first place, and whether anything checks its work before it reaches a user.

This article was inspired by a production rebuild where the fix wasn't a better model or a longer prompt. It was replacing model calls with code wherever the step had one correct answer. Here's the LinkedIn post that sparked this guide:

Is Your Agent System Actually Unreliable, or Just Unpredictable?

These are different problems with different fixes, and confusing them wastes engineering time on the wrong layer. An unreliable system produces wrong answers; an unpredictable one produces right answers through a path you can't explain or reproduce. Only one of these is a model problem.

Run this check before changing anything: pull ten recent runs of the pipeline and trace each output back to its inputs. For each one, ask whether a human following the same instructions would have arrived at the same number, the same classification, the same formatted result. If a step like data fetching, calculation, or formatting produced a different result on a re-run of identical inputs, that step is doing work a model shouldn't own.

If the variance is in steps with one correct answer, the fix is architectural: move that step to code. If the variance is confined to genuinely judgment-based steps (summarizing an ambiguous request, choosing a tone), the system may be working as designed, and the fix is prompt or eval work, not a rewrite. The rest of this guide addresses the first case.

What Is the Deterministic-Core Pattern?

The deterministic-core pattern is an architecture where plain code handles every step that has a single correct output, and a language model is called only for steps that require language understanding, judgment, or handling ambiguity that code can't resolve. It inverts the default assumption that an "AI system" should route as much as possible through the model.

In practice this means data fetching, validation, arithmetic, and formatting run as ordinary functions with unit tests, not as instructions in a prompt. The model gets three kinds of work instead: reading an unstructured or messy request, producing text a human will read, and making a bounded classification decision (which of N known categories does this belong to).

Why This Matters for Cost and Failure Modes

A model call on a deterministic step is strictly worse on every axis that matters in production. It's slower than a function call, it costs money a function call doesn't, and it introduces a failure mode (an occasionally wrong number) that a function call structurally cannot have. None of this is a tradeoff for accuracy, because the function is also more accurate: it can't drift.

The counterintuitive part is that this constraint doesn't make the system less capable. A pipeline where code handles the parts machines are already good at and the model handles the parts that genuinely need language understanding outperforms a pipeline where the model does everything, on both cost and reliability, because the model is no longer being asked to simulate arithmetic it was never designed to guarantee.

How to Decide Where AI Belongs in an Agent Pipeline

Use this as a per-step audit, not a one-time architecture review. Pipelines drift as features get added, and each new step deserves the same test.

1. Write down the step's expected output for a fixed input. If you can specify it exactly (a total, a formatted date, a validated field), it belongs in code.
2. Check whether the step requires reading unstructured input. Free-text requests, ambiguous phrasing, or content a schema can't capture are model territory.
3. Check whether the step requires producing unstructured output. Writing a summary, a reply, or explanatory text for a human reader is model territory.
4. Check whether the step is a bounded classification with a small, fixed set of outcomes. A model can do this well, but so can a smaller, cheaper model than the one doing your open-ended reasoning. Don't default to your most expensive model for a three-way bucket decision.
5. If none of the above apply, the step is probably deterministic and mislabeled as an AI step. This is the most common finding in audits of production agent pipelines.

Sizing the Model to the Step

Once a step is confirmed as genuinely model-territory, size the model to that step rather than to the system as a whole. A bounded classification step that a mid-tier model gets right 98% of the time doesn't need a frontier model that gets it right 99% of the time; the extra point rarely changes downstream behavior and can triple the cost of that step. Reserve the most capable model for the step where reasoning quality visibly changes the outcome, typically the open-ended language step, not the bucket-sorting step next to it.

How to Build the Audit Loop That Catches Silent Failures

A pipeline with a correctly drawn deterministic core still needs a final check, because the model steps that remain can produce fluent, plausible, wrong output, and fluency is exactly what makes that failure mode hard to catch by eye.

Step 1: Define ground truth for the output, not just a format. Before the audit step exists, know what "correct" means for this output type: does every number in the output have to trace back to a row in the source data? Does every claim need a citation to a specific field?

Step 2: Run a separate pass that checks output against source data, not against plausibility. This is the distinction that makes an audit loop work: it should ask "does this statement match this row" rather than "does this look reasonable." A plausibility check catches almost nothing, because hallucinated output is specifically optimized to look reasonable.

Step 3: Fail closed, not open. When the audit step finds a mismatch, the pipeline should block the output rather than pass it through with a warning. A warning that nobody reads in production is equivalent to no check at all.

Step 4: Log every audit failure with the specific mismatch, not just a boolean. This turns the audit loop into a source of eval data. Recurring failure patterns point directly at which upstream step needs to move from model to code, closing the loop back to the first section of this guide.

Common Questions

How do I know if a step is really deterministic or just looks like it?

Test it with the same input twice, several hours or days apart, and compare outputs byte-for-byte. A truly deterministic step (a calculation, a lookup, a format conversion) produces identical output every time. If a step that should be deterministic sometimes varies, it usually means a model call is hiding inside what should be a pure function, often through an unnecessary "let the model tidy this up" pass.

Doesn't handing more to code just move the maintenance burden instead of reducing it?

It shifts the burden to a place engineering teams already know how to manage. Deterministic code fails loudly, is unit-testable, and shows a diff when something changes. A model step fails silently and produces a different diff every time you look at it. Maintenance burden doesn't disappear, but it moves from "reading transcripts to guess why an output looks off" to "reading a stack trace," which is strictly cheaper at scale.

Can this pattern work with agents that need to take real-world actions, not just answer questions?

Yes, and the pattern applies more strongly there. Any action with a side effect (sending an email, charging a payment, updating a record) should be executed by code that the model calls with validated parameters, not generated freehand by the model as a command string. The model's job is deciding which action to take and with what parameters; executing it correctly every time is code's job.

What's the minimum audit loop worth building if I only have time for one check?

Check that every number and named entity in the model's output appears in the source data it was given. This single check catches the majority of production hallucination incidents, because most user-visible failures are the model inventing a specific detail rather than getting the overall structure wrong.

Does this pattern still apply to single-agent systems, or only multi-agent pipelines?

It applies at any scale. A single agent with five tool calls has the same decision to make at each call: is this step deterministic, judgment-based, or a bounded classification? The pattern is about how work is divided between code and model, not about how many agents are coordinating; a single well-scoped agent benefits from the same audit just as much as a five-agent pipeline does.

Key Takeaways

• Diagnose before you redesign: trace ten recent runs back to their inputs and confirm variance is actually in judgment steps, not in steps that should have one correct output.
• Default to code for any step with a single correct answer (fetching, validating, calculating, formatting). Reserve the model for reading unstructured input, writing for a human reader, and bounded classification.
• Size each model call to its step. A cheap model that clears the accuracy bar on a bounded task beats a frontier model doing the same task at three times the cost with no behavioral difference.
• Build an audit loop that checks outputs against source data, not plausibility, and fails closed on a mismatch so bad output never reaches a user silently.
• Treat audit failures as eval data. Recurring mismatches identify exactly which step still needs to move from the model to code.