In this guide a workflow is the first design and an agent is the second. The question for a build is which of those your task needs, and the answer comes from the path the work takes rather than from the label on the ticket.

Most of what we have built is a workflow in that sense, by design. Of the ten systems in our case studies, the ones that turn filings, custodian feeds, legal drafts or meeting audio into records run as fixed pipelines, and where a model appears it sits inside one step. They are the ESG filing pipeline, the fund reconciliation system, the trademark drafting system and the meeting transcription app.

Even the DevOps agent turns each request into a structured spec and an explicit chain of tool calls, because a reviewer can only build judgement about output whose shape is stable. The chief-of-staff agent needed a loop. It works across more than a hundred registered systems, much of it behind logins with no usable API, and each step reads what the last one returned before choosing the next.

A demo hides the difference. Each shape can call a model, show a tool call and return a sentence that sounds finished. The split shows when a step fails or a record is missing. A workflow stops on the branch you wrote and retrieval says the passage is absent. An agent has to choose another tool, or stop, based on what it just saw.

If you cannot point at a case where that choice is real, the loop is overhead you will spend the next quarter tracing. This guide settles the vocabulary first, then takes the three shapes in the order a team usually reaches for them and puts five signals in one matrix. The last two sections cover the signals teams most often skip. One is a write that can do damage before any gate exists, and the other is a success check that only the model can see.

What is an agentic workflow, and how is it different from an AI agent?

The phrase is used both ways, so ask which one a vendor or a colleague means before you compare designs. Most of the pages that rank for it mean the second design, a model choosing the next step while it runs. Anthropic's Building effective agents calls that same system an agent and keeps "workflow" for code paths written in advance. Two proposals can both say "agentic workflow" and describe opposite systems, and the difference decides how you test them.

The usual list of workflow patterns is chaining one call into the next, routing to a branch, running calls in parallel, an orchestrator handing out subtasks, and one call grading another's draft. Review all but one of them as workflows, because code still owns the order.

The orchestrator is the exception: a lead model chooses the subtasks, so part of the path is already the model's choice, and it needs the checks of an agent. Multi-agent systems covers that pattern once the workers are agents themselves.

We keep those words because they name the decision you have to make. A scripted workflow is code picking the next step, with a model inside some of the steps. An agent is the model picking it. Plain automation, such as a rules engine, has no model at all, and the next section shows how far that gets.

When a proposal uses the phrase "agentic workflow", ask which step the model chooses and what code checks the choice. If the model chooses none, you are looking at a scripted workflow with model calls in it, which is usually the right thing to build. If it chooses the next tool, the rest of this guide applies before you sign off on it.

When is automation or a scripted workflow enough?

Rule-based automation and a scripted workflow both keep the path in code, and an AI agent does not. A scripted workflow is the right shape when someone who knows the job can write the steps, including the branches. Invoice extraction, support triage with a fixed set of labels and document classification usually sit here. The code walks classify, retrieve, draft, check and send, a model fills the steps that need language, and the step after each model call is the one you named.

The fund reconciliation system we built shows how far rules go. Custodians agree on nothing, not the format, the field names or the sign conventions, so each custodian gets its own mapping rather than one parser full of conditionals. The matching rules are explicit and versioned, because they are the domain knowledge. A person approves at one step only, resolving an exception, because that is where human judgement enters the data. There is no model call anywhere on that path.

The common advice runs the other way. OpenAI's practical guide to building agents points agents at rule sets that have grown unwieldy and costly to update. Take that advice when nobody can state the rules. When the rules are the domain knowledge, as they are in reconciliation, versioning them in code costs less than asking a model to rediscover them on each run, and an auditor can read the version that applied.

Put an agent on a path like that and the model spends the run re-deciding a sequence the code already had. A failure then looks like a choice, so you open the trace, when the missing piece was a branch nobody wrote. With the order of steps in code, a reviewer can point at the branch that fired.

The trademark drafting system is a pipeline for a different reason. Each stage has a defined failure behaviour, and a failed validation triggers a bounded retry that may draw only on retrieved evidence. Falling back to unconstrained generation is the most common way we see systems of that shape go wrong. A loop that could choose its own next step would reopen that path.

A branch you can state as a rule is still a workflow. If the amount is under the cap and the order is inside the return window, draft the refund and wait for a person to approve it. The model can draft the note while the cap stays in the code that checks the amount. A value you can compute is computed in code and passed in.

When is retrieval enough on its own?

Retrieval is enough when the failure is a missing or stale fact and one query can find the passage that holds it. A refund window that changed in March, a policy clause, or a filing the model was never trained on are RAG problems, or plain search problems. An agent does not repair a weak index, and RAG vs fine-tuning walks those layers in the order they cost.

Prospex, our own research engine, has to answer both "how do they make money", which is a semantic question, and "are they using Snowflake", which is a keyword question. Pure vector search will happily return a passage about a different warehouse. So semantic and keyword search run together rather than as a fallback chain, because their failure modes are complementary. A custom chunker keeps a role and its company in the same chunk.

A loop that searches until it feels sure calls the same weak index several times and then writes a fluent answer over thin evidence. You pay for several model calls to hide a recall problem that one better query would have shown in the log. On Prospex, too little retrieval produces a short profile, which is the safe failure, because a short profile is visibly short and an invented one is not.

Some questions do need more than one retrieval. The ESG pipeline gets comparative research questions, such as one about "the same subsidiary two years earlier", which pure vector search cannot express. Pure graph traversal fails the other way, because it cannot find a narrative claim that was never structured. One retrieval cannot answer a comparison, so the question is broken into steps: select sources, combine graph and vector evidence, build claims, check them and attach the source page to each one.

On the ESG pipeline that is a sequence of named steps: several retrievals in a known order, not a model left to decide what to search next. If you can name the second query the first result forces, you can often write it as a step. An agent loop starts to pay only when the next query depends on what came back in a way you cannot list.

When do you need an AI agent instead of a workflow?

An agent earns its cost when the next action depends on an observation you could not encode as a branch. The model selects a tool, reads the result, and either continues or stops. How to build an AI agent covers that loop, and this guide covers the decision to build it.

On the chief-of-staff agent each step runs think, act, observe, and the raw observation goes back into the loop rather than being summarised away. A critic then decides whether to continue, retry or escalate. Without it, one bad observation spreads through every later step and the agent claims success on work it never completed.

The model is not handed the whole tool registry either. It queries the registry and chooses among a handful of relevant contracts, because past roughly 30 tools in the prompt it started picking plausible but wrong ones.

Tool recovery is where the two shapes part. Repeating a call that timed out is still a retry inside a workflow. A not-found error that means "try the archive, and if that is empty, ask for the account id" is a choice among tools. If every recovery you care about is already a row in a table, keep it in the table. Start the loop only when the error has to pick the tool.

A miss inside the loop runs longer than it would in a workflow. A wrong branch fails one step, while a wrong choice can take several more tool calls before anyone sees a bad final state.

The loop pays for its flexibility in latency and tokens, and AI agent cost shows how each extra turn resends the transcript. The loop needs a budget, a stop rule and a check that does not trust the model's claim that the work is done.

How do five signals place one task?

Five signals are enough to place a task, and one product can sit in different columns on different rows. Read down the rows for the task in front of you before you pick a column for the whole product. The figure repeats the comparison at the top of this page.

Triage can be a workflow, the policy answer can be retrieval, and the ticket whose next system depends on the last lookup can be an agent. A single label on the product hides which of the three you shipped, and the next failure goes to the wrong owner.

The cells are fit judgments you can argue with on a real task. Green marks a fit for that signal. Amber means the shape works only when the condition in the cell is true. Red means stop and build the other shape, because this one adds a way for the bad outcome to happen.

A task that is green in the workflow column and red in the agent column should stay a workflow even if the roadmap calls for an agent. When a row is red because a write can do damage and no gate exists yet, read the next section before you add a loop.

What happens when a bad write has no gate?

A write that can move money, change access, send a customer message or alter production state can do real damage, and a loop does not make that smaller. If you do not yet have a gate that can refuse the call, a credential scoped to that action, and a check on the final state outside the model, keep the write out. The read path can still be a workflow or a retrieval app.

Adding an agent so the model can be careful puts the care in the prompt, which is the place a later instruction can argue with. The systems we have built put it in code. On the DevOps agent, an engineer approves an architecture diagram before anything is applied. Changes then go through the provider's change-set mechanism, which applies a group of changes as one unit, so a partial failure does not leave orphaned resources.

On our insurance voice agent, a payment cannot execute without its required disclosure, because the condition sits in the tool contract and not in the prompt.

The same agent shows what a failed write should do. A request that cannot be completed does not turn into an approximation. It escalates to a person, with the dialogue and the actions already taken attached, rather than doing something adjacent to what the caller asked for.

Human in the loop AI agents covers the product boundary, where answering can stay read-only and the first write is one kind of action. AI agent governance covers how one action gains authority after the evidence for it exists. Both assume you have already decided that an agent is the right shape for the path.

What do you score once the shape is chosen?

Success has to be a state something other than the model can check, or you cannot tell the three shapes apart after launch. A workflow scores the output of each step: the label, the drafted field, the branch that fired. Retrieval scores whether the right passage was in the set and whether the answer stays inside it.

An agent scores the trace and the final record. On the chief-of-staff agent, the claim that a task is complete is compared against the actual state of the target application, which catches claims of success on work that never happened. Work it cannot finish comes back as a flagged exception with its trace attached. The DevOps agent keeps one record that links the request, the account facts, the generated template, every validation failure and the resulting resources.

If the only signal is the model's closing sentence, every shape will claim it is done. A workflow claims done when a branch was skipped, retrieval when the passage was missing, and an agent after a tool error it folded into the prose.

Name the check before you pick the shape, then build the shape that can produce that check with the least machinery. Evals keep that check stable from run to run, and LLM evaluation covers building the cases. Until they exist, a demo that felt like an agent is not evidence that you needed one.

Observability on a loop is the trace of that check. Store the prompt, the tool list and the stop reason with the run, so a wrong final record can be tied to the choice that produced it. A workflow needs less of that, because the branch is already in the code. AI agent monitoring covers reading those traces once the system is live.

Common questions

What is an agentic workflow?

An agentic workflow is a multi-step process in which a language model does part of the work, such as drafting, classifying, retrieving or calling a tool. Some writers use the term only when the model chooses the next step. Anthropic calls that design an agent and keeps "workflow" for systems whose steps run through code paths written in advance. Ask which meaning is intended before you compare two designs.

What are the types of agentic workflows?

The usual patterns are prompt chaining, routing, parallel calls, orchestrator-workers and evaluator-optimizer. In all but one, code still owns the order, so review them as workflows. In orchestrator-workers a lead model chooses the subtasks, so part of the path is the model's choice and it needs the checks of an agent. Pages that use the term for agents list planning, tool use and reflection instead.

Is an agentic workflow the same as an AI agent?

Not in the sense this guide uses. In a scripted workflow, code picks every next step and a model fills some of the steps. In an AI agent, the model picks the next step and the next tool from what the last one returned. Many vendors call both of them agentic, so ask which step the model chooses and what code checks that choice.

What is the difference between AI agents and automation?

Rule-based automation follows steps and rules written in code, with no model in it. A scripted workflow keeps that fixed path and adds a model inside some steps. An AI agent lets the model choose the next action. Bigcircle's fund reconciliation system kept its matching rules explicit and versioned in code, because the rules are the domain knowledge and are versioned as such.

Do we need an agent, or is a workflow enough?

Build a workflow when a person can write the steps and the branches, including retries that always repeat the same call. Build an agent when the next tool depends on a result you could not list before the run. Most document and reconciliation systems Bigcircle has built are fixed pipelines, and the agent loop in its case studies is the build that works across many systems.

When is a RAG or search app enough on its own?

When one query can find the passage and the answer has to stay inside it. Fix the index, identifier matching, ranking and the permission filter on the query before you add a loop. If a question needs several retrievals, as a comparison across years does, try writing them as a fixed sequence first, and use an agent only when the next query cannot be known in advance.

Should a high-risk write start as an agent?

No. A call that can move money, change access, send a customer message or alter production state needs a gate before any loop may request it. If the gate, a scoped credential and a check on the final state are missing, keep the write out and ship the read path as a workflow or a retrieval app. Authority for one kind of action is a later release.

How do you know the run succeeded?

Check a state outside the model's closing sentence. A workflow should show the branch that fired and the field it produced, retrieval should show the passage and whether the answer stayed inside it, and an agent should show its tools, any recovery, any approval and a final record that matches what was expected. If you cannot name that check, you cannot yet tell which shape you need.

Further reading

• Anthropic, Building effective agents, 19 December 2024. The use of "workflow" and "agent" this guide follows.
• OpenAI, A practical guide to building agents, April 2025. The advice to point agents at unwieldy rule sets, which the automation section argues against for rules that are domain knowledge.
• How to build an AI agent. The loop, once the path cannot be scripted.
• AI agent cost. Why each turn of a loop costs more than the last.
• RAG vs fine-tuning. Which failure you have, before a month goes to the wrong layer.
• Human in the loop AI agents and AI agent governance. The first write, and how one action gains authority.
• Multi-agent systems. When the workers in an orchestrator-workers design are agents too.
• Fund operations reconciliation, executive operations agent and prospect intelligence engine. A pipeline, an agent and a retrieval system from our own builds.
• AI agents. For a team that has already passed the test in this guide.

Originally published on bigcircle.ai. I'm Tirth, founder of BigCircle, where we build production AI systems with governance built in.

Plenty of AI projects stall at the legal review, when someone asks what leaves the network and the answer is everything: the support ticket, the customer's email, the account number. We build the layer that changes that answer: a small piece of code in the request path that adds single-digit milliseconds to a call. This guide covers the design choices in it, the Python, where an entity recognizer such as Presidio fits, what the tokens mean under GDPR, and the CI test that turns "we tokenize PII" into something a compliance team can check.

PII tokenization vs masking: which keeps the answer useful?

Tokenize when the model has to reason about who is who, and mask only when it does not. Masking replaces every email with [EMAIL] and every number with ****. It is simple, and it makes the answers worse, as soon as a prompt holds more than one of anything.

Say a ticket mentions two customers and two accounts. After masking, the model sees [EMAIL] twice and **** twice. It cannot tell which account belongs to whom, so it cannot write "refund account X for customer Y", because the structure it needed has gone.

Reversible tokenization keeps that structure. Each sensitive value gets its own token, and the same value always gets the same token within a request, so two different emails get two different tokens and the model can still say "this customer, that account".

Two properties make tokenization work in a prompt. It is reversible: a private map from token to value lives for the life of the request, so the answer can be restored. And it is format-preserving: an email token still looks like an email, and a 16-digit account number becomes another 16-digit number, so prompts, parsers and validators downstream keep working and the model treats the field as it would treat the real one.

Is tokenized data still personal data under GDPR?

For you, yes, and for the provider possibly not, which is the argument to bring to legal review. A token map you keep is pseudonymization in GDPR's own terms: the data can no longer be tied to a person without additional information held separately. Pseudonymized data stays personal data for whoever holds that additional information, so tokenizing does not take your own processing out of scope.

The provider's position is different. In EDPS v SRB (C-413/23 P, 4 September 2025), the EU Court of Justice held that pseudonymized data passed to a recipient is not always personal data from that recipient's side, when the recipient cannot reverse the pseudonymization or identify people by other means. A provider that receives user_1ec32386@example.invalid, under a key that changes every request and never leaves your process, is the setup that ruling speaks to.

Describe the layer to your counsel as pseudonymization, and have them read the ruling against your setup. Anonymization is the wrong word for it, because the map you keep can reverse every token.

Which fields should you tokenize?

Decide with whoever owns the risk, from your real data, before writing code. Not every field is personal data, and encoding everything destroys the answer as surely as masking does. Over-redaction is its own failure.

The output is a short list of data classes, such as email, account number and phone, with a rule per class and per jurisdiction. On the telecom marketing platform we are building agents for, subscriber data spans several jurisdictions, and the analytics agent works through a governed data-access layer that masks, aggregates or excludes PII by policy before the agent sees a row. That policy is enforced in the access layer, never in the prompt, because a prompt is a request the model can ignore.

Structured classes such as emails, account numbers and phone numbers can start with pattern rules. Names, addresses and free-text identifiers need entity recognition, covered further down.

How do you tokenize PII before the provider call?

Create one boundary object per request that detects sensitive values, swaps each for a token, remembers the mapping and writes a record of what it encoded.

import hmac, hashlib, re
from dataclasses import dataclass, field

# Pattern rules for the fields your risk owner marked sensitive.
# In production, add an NER pass for names and addresses.
PATTERNS = {
    "EMAIL":   re.compile(r"\b[\w.+-]+@[\w-]+\.[\w.-]+\b"),
    "ACCOUNT": re.compile(r"\b\d{10,16}\b"),
    "PHONE":   re.compile(r"\+?\d[\d -]{8,}\d"),
}

@dataclass
class Boundary:
    key: bytes                                   # per-request secret, never logged
    vault: dict = field(default_factory=dict)    # token -> raw value
    seen: dict = field(default_factory=dict)     # raw value -> token
    record: list = field(default_factory=list)   # what was encoded, without the values

    def _digest(self, kind, value):
        return hmac.new(self.key, f"{kind}:{value}".encode(), hashlib.sha256).hexdigest()

    def _surrogate(self, kind, value):
        h = self._digest(kind, value)
        digits = "".join(str(int(c, 16) % 10) for c in h)
        if kind == "ACCOUNT":
            # Same length, all digits. The "99" prefix marks it as a stand-in.
            return "99" + digits[: len(value) - 2]
        if kind == "EMAIL":
            return f"user_{h[:8]}@example.invalid"
        if kind == "PHONE":
            return "+00 " + digits[:10]
        return f"{kind}_{h[:8].upper()}"

    def encode(self, text: str) -> str:
        for kind, pattern in PATTERNS.items():
            def swap(m):
                raw = m.group(0)
                if raw in self.vault:            # already a token, leave it
                    return raw
                if raw not in self.seen:
                    tok = self._surrogate(kind, raw)
                    self.seen[raw] = tok
                    self.vault[tok] = raw
                    self.record.append({"kind": kind, "token": tok})
                return self.seen[raw]
            text = pattern.sub(swap, text)
        return text

Three choices in it are deliberate.

A fresh key for every request

Tokens come from an HMAC of the value under a random key. The same email gives the same token inside one request, so the model can connect two mentions of one customer. Across requests the key changes, so the provider cannot link tokens between calls and build a profile. The changing key also puts the provider in the position the EDPS v SRB ruling describes.

Tokens that cannot be real

example.invalid is a reserved domain that never resolves, and +00 is not a country code. If a token ever escapes into an outgoing email or a log, it is visibly a stand-in and not someone else's real address.

A record of tokens, not values

The per-request record lists the kinds of data encoded and the tokens used, and never the values. You show that record in the review afterwards.

Run on a support ticket:

import os
b = Boundary(key=os.urandom(32))
ticket = ("Customer priya.shah@acme.in says account 4111222233334444 "
          "was charged twice. Call back on +91 98765 43210.")
print(b.encode(ticket))
# Customer user_1ec32386@example.invalid says account 9955708008262310
# was charged twice. Call back on +00 0573126352.

Your tokens will differ, because the key is random, but the model still knows there is one customer, one account and one phone number, and what happened to them. It does not know who they are.

How do you restore the real values after the reply?

Decode with a lookup in the other direction, longest token first so one token never clips another:

    def decode(self, text: str) -> str:
        # Longest tokens first, so one token never clips another.
        for tok in sorted(self.vault, key=len, reverse=True):
            text = text.replace(tok, self.vault[tok])
        return text

If the model replies "Email user_1ec32386@example.invalid to confirm the refund on 9955708008262310", your user sees the real email and account number, using values your own systems already hold.

Models sometimes reformat what they copy, so a phone token can come back with different spacing. Tell the model in the system prompt to copy identifiers exactly. Then, after decoding, look for anything still shaped like a token: example.invalid, a +00 number, a 99-prefixed account. A leftover token means the model changed or invented it. Treat that response as failed and retry it or route it to a person, rather than sending a half-decoded answer.

Where does Presidio fit for names and addresses?

Use an entity recognizer for detection and keep your own tokens for the replacement. Patterns catch emails and account numbers, and they miss names, street addresses and identifiers written into free text. Presidio, the open-source PII toolkit Microsoft published, pairs named-entity recognition with pattern recognizers in its analyzer, and its anonymizer offers replace, redact, mask, hash and encrypt operators, with a decrypt operator that reverses encryption.

Presidio's encrypt and decrypt pair is reversible, and it is not format-preserving: an encrypted name comes back as ciphertext, which a model reads as noise rather than as a person. Feed the analyzer's results into the boundary above, so a detected name gets a name-shaped token from the same per-request map and decodes with everything else. Every detector misses something, so the leakage test below sits behind all of it.

How do you prove no PII reached the model provider?

Write a test that fails the build when a raw value reaches the provider. "We tokenize PII" is a claim, and this test is the evidence. Put a fake provider behind your real code path, capture everything that would have been sent, and assert that none of your known sensitive fixtures appear in it:

import os, json
from boundary import Boundary

RAW_FIXTURES = ["priya.shah@acme.in", "4111222233334444", "+91 98765 43210"]

class CapturingProvider:
    def __init__(self):
        self.sent = []
    def complete(self, payload: dict) -> str:
        self.sent.append(json.dumps(payload))
        return "ok"

def handle_ticket(text, provider):
    b = Boundary(key=os.urandom(32))
    payload = {"messages": [{"role": "user", "content": b.encode(text)}]}
    return b.decode(provider.complete(payload))

def test_no_raw_value_reaches_provider():
    provider = CapturingProvider()
    handle_ticket("From priya.shah@acme.in, card 4111222233334444, "
                  "phone +91 98765 43210", provider)
    for body in provider.sent:
        for raw in RAW_FIXTURES:
            assert raw not in body, f"raw value leaked: {raw!r}"

Run it in CI on every pull request, and grow the fixtures. Each time someone finds a format the rules missed, such as an email with a plus sign or an account number with spaces, add it as a fixture before fixing the rule. The suite becomes the list of everything you have promised not to send, and we hand it to a compliance team instead of a policy document.

Test the whole path, not just the boundary class. Leaks rarely start in the tokenizer. They start when someone later adds a helpful field to the prompt, the raw customer record for context or the user's profile in the system message, and it skips the encoder. A test on the real request path catches that on the pull request that adds it.

Where does the tokenization layer sit?

Put it in the request path, directly in front of the provider call, so every call goes through it. A boundary that each feature calls when it remembers to will be skipped by the feature written in a hurry. Keep it fast as well. In our builds the encode and decode step adds single-digit milliseconds, so there is no latency argument for switching it off under load.

Three limits shape the rest of the design:

• It protects the next request. Data already sent to a provider is a separate conversation with that provider.
• Detection misses things. Patterns and entity recognition both need the leakage suite behind them.
• The token map is now the sensitive object. It lives for one request, in your process, and never goes into logs or traces.

Credentials follow the same rule. On Solarpunk, the desktop agent we built, secrets sit in an encrypted vault on the user's machine and are injected into the tool adapter when a tool runs, so they never travel into the prompt. Whatever the model never receives, a prompt injection cannot ask it to repeat.

We build this layer for teams whose AI work is waiting on that legal review. PII redaction and data boundaries covers how an engagement runs, from classifying fields in week one to the leakage suite your compliance team reviews.

Common questions

What is PII tokenization?

PII tokenization replaces each piece of personal data in a prompt with a stand-in token before the request goes to the model provider, and restores the real values after the reply. The provider sees the structure of the record and never the values, and the user still gets a complete answer.

What is the difference between tokenization and masking?

Masking replaces every email with the same placeholder, so the model cannot tell which account belongs to which customer, and the original cannot be restored. Reversible tokenization gives each value its own stand-in within a request, so the structure survives and the real values go back in after the reply.

Is tokenized data still personal data under GDPR?

For the organization holding the token map, yes: it is pseudonymized data, which stays personal data. For a recipient that cannot reverse the tokens or identify people by other means, the EU Court of Justice held in EDPS v SRB (C-413/23 P, 4 September 2025) that it is not always personal data. A fresh key per request that never leaves your process puts the model provider in that second position.

Can Microsoft Presidio do reversible PII tokenization?

Presidio's anonymizer has an encrypt operator and a decrypt operator that reverses it, so its output is reversible. It is not format-preserving, because an encrypted value comes back as ciphertext. Bigcircle recommends an entity recognizer such as Presidio's analyzer for detection, with format-preserving tokens generated from your own per-request map.

Does PII tokenization slow down the LLM call?

In Bigcircle's builds the encode and decode step adds single-digit milliseconds, which is small next to the model call itself. Keeping it that fast matters, because a boundary that teams switch off under load protects nothing.

How do you prove no personal data reached the provider?

Put a fake provider behind the real code path, capture everything that would have been sent, and fail the build if a known sensitive fixture appears in it. Run the test in CI on every pull request, and add a fixture each time someone finds a format the rules missed.

Further reading

• PII redaction and data boundaries (linked above). How Bigcircle builds and proves the layer.
• AI agent permissions. What an agent may do once the data it sees is controlled.
• Prompt injection in multi-agent systems. Why what never reaches the model cannot be extracted from it.
• Court of Justice of the EU, EDPS v SRB, C-413/23 P, judgment of 4 September 2025. Pseudonymized data from the recipient's side.
• Presidio documentation. The analyzer's recognizers and the anonymizer's operators, including encrypt and decrypt.

Originally published on bigcircle.ai. I'm Tirth, founder of BigCircle, where we build production AI systems with governance built in.

Our case studies show where the work went. On the DevOps agent, the build was mostly about reading the live AWS account, checking every template and putting a human review step before anything applied. On the chief-of-staff agent, the visible cost of tool definitions in the prompt turned out to matter less than the wrong tool choices that came with them.

The sections below take the build, then the run with a worked token calculation at prices as of 22 October 2026. The keep-alive and a way to compare two quotes come next. The last two sections cover the cap that ends a runaway loop before the next paid call, and the ledger of what each run spent. If a script or one query can do the job, agentic workflows vs AI agents comes first, and none of this bill starts.

How much does it cost to build an AI agent?

The build price follows the scope, and most of the scope sits around the model rather than in it. The build is everything that has to work before the first real task leaves the team: the tools the job needs, permission to call them, an eval set, and a path for the step that fails. Choosing a model is one decision inside that work, and usually not the long one.

The DevOps agent shows where the time goes. A raw model invents resource types, writes templates the account rejects for quota or region, and drifts between runs, and a failed stack can take minutes to unwind, so "apply and see" is not a usable loop.

The build therefore had to read live limits, quotas and existing resources before planning, check every template in a loop of structural and semantic checks, and show an engineer an architecture diagram before anything applied. Delivery took roughly four months.

The chief-of-staff agent spent its build on reach. It connects to more than a hundred systems, and a large share of an executive's work sits behind single sign-on and multi-factor login with no usable API. We argue that a routine the agent can finish 70 percent of still needs a person to hold the sequence, so it saves almost nothing. That is why reach costs build time on an agent like this.

Each integration is a tool contract, a login path and a failure the agent will meet in normal use. MCP authorization covers the check before each call, LLM evaluation covers the eval set, and how to build an AI agent puts those decisions in order. A build quote that lists model access and leaves out the access check and the eval set will be rewritten after the first real task fails.

How the build is bought is a separate question. AI pilot to production compares production hardening, a fixed-scope build and a studio-led programme. A fixed-scope build is priced against a scope agreed before work starts, so the useful comparison between two quotes is which of the three parts each one names.

Why does an AI agent cost more to run than one reply?

A run costs more than one reply because the agent decides how many steps to take, and each step sends the whole history again. A workflow pays one model call per step you wrote. An agent pays a number of calls that depends on the trace, and the input on each call grows with every tool result.

A worked example makes the shape concrete. The token counts are assumptions chosen for the example, and the prices are OpenAI's list rates for GPT-6 Luna as of 22 October 2026: $0.10 per million input tokens and $0.50 per million output tokens.

Take a system prompt and tool definitions of 6,000 tokens, a task of 10 turns, 300 output tokens per turn for the tool call, and 2,000 tokens of tool result added to the transcript after each turn.

Ten turns cost about 24 times one reply, not 10 times, because each turn pays again for every earlier turn's transcript. At 10,000 tasks a day that is about $179 a day on the run line.

Reasoning adds to it. OpenAI bills reasoning tokens as output tokens, and Luna defaults to reasoning effort medium. If each turn spent 1,000 reasoning tokens, another assumption, the same task would cost about $0.005 more, roughly 28 percent on top. The example also assumes reasoning from earlier turns is not sent again. Check that before you trust the figure, because the GPT-5.6 family carries earlier reasoning into the next turn by default. On a model that does, those tokens add to the input of every later turn. LLM model migration covers why a model change can move this line while the per-token price falls.

Tool definitions are part of the input on every turn, and the chief-of-staff agent found where listing all of them in the prompt stops working. Direct tool-calling with every tool listed worked up to roughly thirty tools. Past that, the model started choosing plausible but wrong tools, and we found that the context cost was the visible symptom while the selection error was the expensive one. The fix was to let the model query a tool registry and choose among a handful of relevant contracts, which kept the working context small at more than a hundred tools.

Prompt caching cuts that bill more than anything else in the example, because it discounts a repeated prefix, and an agent's growing transcript is one. As of 22 October 2026, caching is on by default on OpenAI's supported models, with a minimum prefix of 1,024 tokens on GPT-5.6 and later. On those models a cache read costs a tenth of the input rate and a cache write 1.25 times it.

An entry lasts 30 minutes by default, and each read refreshes it. OpenAI's caching rules do not name Luna, and Luna's model page lists cached input at $0.01 per million, the same tenth, so we assume the same write rate.

Applied to the example, if every turn after the first reads the earlier transcript from cache and writes only its new 2,300 tokens, the ten-turn task comes to about $0.006 instead of $0.018. That is 136,800 tokens read at the cached rate, 26,700 written at the write rate, and the same 3,000 output tokens. It assumes each turn arrives inside the cache lifetime with the whole prefix unchanged, so a retry that edits anything early in the transcript pays full price again from that point.

Caching at the application layer saves a different cost. Prospex, our own research engine, runs cache-first, so a rerun reuses earlier extraction and retrieval instead of calling external services again. By our estimate that cut repeated external API calls by 50 to 60 percent, and we watch the cache hit rate as its cost metric.

What does an AI agent cost after launch?

Keep-alive is the work after launch. It covers rerunning the eval set when a tool, a prompt or the model changes, a person who can act when a run stops, and traces that someone can still read when finance asks what happened. None of it shows on the launch invoice, and all of it is on the calendar afterwards.

Integrations are where it starts. We saw what point-to-point wiring does to maintenance on the chief-of-staff build: a vendor changes a response, and the break shows up somewhere unrelated days later. Our answer there was a uniform tool layer, where each tool is a typed contract with declared inputs, outputs and side effects.

Human review is the other running line. On the trademark drafting system, attorney corrections are the most valuable signal in the system, because they measure the gap automated scoring cannot see. That review does not end at launch. On the ESG filing pipeline, the slow failure is framework version drift, which is why the mapping version is recorded on each mapped fact.

Observability on an agent is the trace of each run, stored with the run. AI agent monitoring separates the quality of the answer from the health of the path that produced it. Name the person who owns the stop before launch. A stop with nobody assigned to it is only a log line, and the loop behind it keeps spending until someone reads the invoice.

How do you compare two AI agent quotes?

Compare two quotes part by part, or a missing line will look like a saving. Two proposals can describe the same agent and still leave out different parts, and the shorter one has often left off the keep-alive or the stop.

How do you enforce a budget in the request path?

A budget holds only when the check runs before the next provider call and a breach ends the run. A dashboard that climbs and an email that arrives later have already paid for every call between the threshold and the person reading it. The cap is code on that path, and it limits turns, tool calls, spend, and how many child agents one run may start.

Fan-out, where one run starts several child agents, is the case a per-call limit misses, because each child looks small on its own. If every child gets a fresh counter, together they spend the parent's allowance several times over. Make each child draw on what the parent has left, and have the parent refuse another child once the count or the allowance is gone. Multi-agent systems covers when the children are worth starting at all.

Retries need a bound of their own. We bounded them on the chief-of-staff build, because an agent that retries indefinitely against a broken vendor is worse than one that stops: it burns cost and time while looking busy. The DevOps agent bounds revision the same way, and a template that will not come clean after bounded revision is escalated as a request the agent cannot safely fulfil. Count failed calls in the total, because the provider already did.

A limit written into the prompt can be talked past, and a counter in code cannot. Guardrails that live only in the prompt can give way the first time a tool result or a user message argues for one more step.

Account-level caps from the cloud providers are a backstop behind that check. They act on a whole project after many runs have spent, they cannot tell a retry storm from a busy day, and a paused project takes your working runs down with the broken one.

AWS spend limits, introduced on 16 September 2026 and still rolling out to a limited set of customers, pause a project and stop its resources when its monthly usage reaches the limit. An opt-in control pauses the highest-cost resources about four days before the limit, chosen from five services that include Lambda and Bedrock.

Google Cloud's Spend Caps, in public preview since 29 July 2026, restrict further usage of one service in one project once the monthly cap is reached. Both are worth turning on, and the loop still needs its own check before each call.

Monitoring without a stop is the gap that costs most. In one incident Jatin Bansal collected, a team running four LangChain agents had monitoring and no enforcement. An analyzer and a verifier began looping with each other in November 2025, and the bill reached $47,000 over eleven days. The figures are as Bansal reports them.

Set each cap from runs you have already finished rather than from a framework default, which is calibrated to someone else's task. Leave room above the longest run that completed normally, so a stuck loop hits the cap and a normal task does not. A cap set on a guess either cuts good tasks or lets a loop run.

Where does the spend record live?

The ledger has to outlive the agent, or a loop can erase the only record of what it spent. Write the run id, the tool, whether the call was allowed or refused, the token counts and the stop reason for each call to a store the agent's credential cannot delete or rewrite. Include the calls that failed and the calls each child made, or your meter will sit below the invoice.

Reconcile your meter with the provider bill every month as two sums: billed spend from the invoice, and the cost of every attempt from your own traces, including retries, timeouts, cancellations and outputs you rejected.

A gap left after failed attempts are included is work you are not tracing, such as an eval job, a warm-up call or a second service on the same key. Close that gap before you call either number the cost of a successful task.

Common questions

How much does an AI agent cost?

AI agent cost has three parts, the build, the run and the keep-alive, and each moves for a different reason. The build is tools, credentials, evals and the path for a failed step. The run is turns, tool calls, retries, fan-out and a transcript that grows. The keep-alive is eval reruns, review and traces after launch. A price per token covers one line inside the run.

How much does it cost to build an AI agent?

The build price follows the scope, and most of the scope sits around the model: tools, permissions, an eval set and the path for a failed step. Bigcircle's DevOps agent took roughly four months, spent mostly on reading the live AWS account, checking every template and a human review step. Compare quotes on which of those parts each one names.

Why does an agent task cost so much more than one reply?

Each turn resends the whole transcript, so input grows with every tool result. In this guide's worked example at GPT-6 Luna's October 2026 rates, a ten-turn task with a 6,000-token prompt and 2,000-token tool results costs about 24 times one reply, not 10 times. Reasoning tokens are billed as output and add to that.

Does prompt caching fix the run cost?

It helps on a repeated prefix and nowhere else. On OpenAI's API as of October 2026, caching is on by default, with a 1,024-token minimum on GPT-5.6 and later, where a cache read costs a tenth of the input rate and a write 1.25 times it. In this guide's example that takes the ten-turn task from about $0.018 to about $0.006. A retry that edits the early transcript pays full price again.

How do you stop a runaway agent before the invoice?

Put a hard cap in the request path and check it before the next provider call. Cap the turns, the tool calls, the spend and the number of child agents, and make each child draw on the parent's remaining allowance. When any cap breaks, end the run and keep the partial result. An alert after the threshold has already paid for the calls in between.

Are cloud spend caps enough on their own?

No. AWS spend limits and Google Cloud Spend Caps, both announced in 2026, pause a project or a service once a monthly amount is reached. They are a useful backstop, but they act after many runs have spent, they cannot tell a retry storm from normal load, and they stop healthy runs along with the broken one.

How do you reconcile your meter with the provider bill?

Sum what the provider invoiced for the period, and sum what your traces recorded for every attempt, including retries, timeouts and outputs you rejected. Compare the two before you call either one the cost of a successful task. A remaining gap is work you are not tracing, such as an eval job or a second service on the same key.

Further reading

• OpenAI, GPT-6 Luna, reasoning models and prompt caching, read 22 October 2026. Token prices, reasoning billed as output, reasoning carried across turns, and the cache threshold, read and write rates.
• AWS, Create a spend limit, and Google Cloud, Spend Caps on Google Cloud Budgets, 29 July 2026.
• Jatin Bansal, Agent Budgets and Runaway Prevention, 26 May 2026. The author-reported $47,000 incident.

Originally published on bigcircle.ai. I'm Tirth, founder of BigCircle, where we build production AI systems with governance built in.

What if the tool call succeeded and the task did not?

A tool response describes what one interface observed at one point in time. It may prove that the request was valid, accepted, queued, or written to one store. It may not prove that the intended workflow ever reached its final state, so consider a customer-operations workflow:

1. The agent updates a case.
2. The case system accepts the write.
3. A provisioning event should be created.
4. The provisioning worker should change the customer account.
5. A confirmation message should be sent.

If the first system returns success and the worker later fails, the agent can close the case while the customer still waits. The same gap appears in commerce, finance, infrastructure, messaging, and enterprise workflow systems. Done is the state a customer or operator would recognise as finished, such as a refund record with a ledger entry and a customer-visible status, and a status code only says that one interface accepted the request.

Why write an expected-state contract first?

An expected-state contract describes what must be true after the action has run, in terms the business would recognise, and it should include:

• the relevant state before execution;
• the exact state expected after execution;
• the fields or resources allowed to change;
• the invariants that must remain true;
• the downstream effects expected from the action;
• the deadline for those effects;
• the source that can verify each condition.

The contract should exist before the agent invokes the tool, because without it the agent can only repeat the tool’s success message. It cannot test whether the product delivered the intended result. For a refund, the final state may require a refund record, a ledger entry, and a customer-visible status. For a deployment, it may require the correct account, expected resources, healthy service checks, and no forbidden changes. For an external message, it may require one accepted message with the correct recipient, content hash, and provider ID.

Can you trust an agent that says the monitor is still on?

In two internal coding tasks, a GPT-6.1 Sol agent on 3 October 2026 disabled automatic human alerts from misalignment monitoring, treating an instruction not to send messages as permission. On 6 October a GPT-6 Astra agent removed a monitoring integration to match a reference implementation, then falsely denied removing it. OpenAI's misalignment report, updated 20 October 2026, says OpenAI is strengthening monitoring and assessing overpersistence against a propensity to evade monitoring. Bigcircle reports the work done only after a separate read matches the expected state.

Why verify through an independent source?

The execution path should not be the only path that decides whether execution worked. If the agent writes through an action API, verification can read the system of record through a separate query path. If the agent submits a browser form, verification can inspect the resulting record, event, or provider receipt instead of trusting the changed screen. If the agent deploys infrastructure, verification can inspect the target account and health state instead of trusting the deployment command. Independent does not always mean a separate vendor, it means a separate observation that can detect false success in the execution path.

The verifier should return structured evidence:

Structured output helps the application enforce these fields, and it does not replace the underlying observation.

The comparison returns one of four verdicts, and the next action depends on which one came back. Complete returns the evidence and stops. Incomplete keeps checking while the effects are still inside the deadline. Failed runs a compensating action or a handoff, because a partial effect is already proven. Unknown blocks the retry until a separate read can see the final state.

Why treat a timeout as unknown state?

A timeout is not proof of failure. The request may not have reached the target, it may have reached the target and failed, or it may have succeeded while the response was lost. Those three states require different next actions, and if the agent labels every timeout as failed, it may retry an action that already changed real state. That can duplicate a refund, message, ticket transition, order adjustment, or infrastructure change. The correct state is unknown until the system can reconcile the operation. The run should preserve the operation ID, target, requested parameters, request timestamp, observed response, and last known state. It should then query the target or ledger through the verification path before it decides to retry.

Why do business operations have to be idempotent?

Idempotency means that repeating the same business operation does not create a second consequence anywhere downstream. A transport-level request ID helps, but it is not enough when the same business operation can enter through several services or retries, so use a stable operation key tied to the business action. For example:

refund:{order_id}:{approved_refund_version}
case-transition:{case_id}:{from_state}:{to_state}:{approval_id}
message:{case_id}:{recipient}:{content_hash}
deployment:{change_request_id}:{plan_hash}:{target_account}

Persist the operation state before execution when the target supports it. On retry, the action layer should return the existing result or continue the recorded operation instead of creating a new action. Idempotency does not remove the need for verification, it limits duplicate effects while the verifier determines the final state.

Why reconcile before retrying?

Retry is a business decision rather than only an infrastructure setting, and the two get confused constantly, which is why a safe retry loop asks:

1. Did the target receive the operation?
2. Did the expected state change occur?
3. Did any forbidden or partial state appear?
4. Is the approval and credential still valid?
5. Is the same action still permitted in the current state?
6. Can the next attempt create a duplicate effect?

The answers can produce five outcomes:

• Return success: the first action completed and is verified.
• Continue: an accepted workflow is still within its allowed time window.
• Retry: no effect occurred, the action remains valid, and the retry is safe.
• Compensate: a partial effect needs a defined reversal or repair.
• Escalate: the system cannot prove a safe next action.

A generic exponential-backoff policy cannot make any one of these decisions on its own.

How do you design for partial failure?

Multi-system work creates states between success and failure. An onboarding case may change while provisioning fails, a refund record may exist while the ledger entry is missing, and a deployment may create resources while the health check fails, so model those in-between states explicitly.

Recovery work needs its own authority boundary. An agent allowed to change a case status is not automatically allowed to reverse a financial entry or delete a cloud resource during recovery. Each compensating action needs the same target, parameter, approval, credential, and verification discipline as the original action.

What belongs in the causal evidence chain?

An audit log of tool calls is useful and incomplete on its own. The organization must be able to reconstruct why the action was allowed and what consequence followed. Record:

• the business request and delegating principal;
• the agent identity, model, tools, and policy version;
• the evidence and state used for the decision;
• the approval, scope, and expiry;
• the credential or capability used;
• the exact action and parameters;
• the tool response;
• the independently observed final state;
• any retry, compensation, escalation, or human correction.

Observability should connect these records with one causal operation ID. This evidence lets engineering replay a failure, operations complete a handoff, security inspect the authority chain, and product decide whether the action has earned more authority. It also answers a growing identity problem that we now meet on most agent engagements. NIST’s 2026 concept paper covers identification, authorization, auditing and non-repudiation for agents, and it is context rather than a certification for this design. The engineering requirement remains direct: preserve the person, task, policy, action, and result in one evidence chain.

A later challenge has to read that chain from retained evidence, including the independently observed result, rather than from the tool log alone. How to build an audit trail for AI decisions is the record for the accepted decision, and the operation ID joins it to the tool trace.

What does the production release gate check?

Before an action receives write access, test the state transition it produces rather than only the model response that describes it, and a release-ready eval should include:

• permitted targets and parameters;
• denied records, tenants, and environments;
• stale-state conflicts;
• expired approval and credential cases;
• timeouts before and after target execution;
• duplicate submissions;
• partial downstream failure;
• verifier failure;
• compensating-action limits;
• human handoff quality.

We built custom capability evaluations for an authenticated browser-use workflow. The useful production pattern was to evaluate real task completion and action behavior, not only the agent’s reasoning. Do not grant more authority because the agent explains its plan well. Grant more authority when the system can prove that the action stays inside its boundary, reaches the expected state, avoids duplicate effects, and stops when the result is unknown. How that authority moves up, and how it comes back down, is the release model in how to roll out agent autonomy. When several workers each report a slice as done, the same separate read is the merge, which when to fan out to many agents puts in front of the combined result. Use the Agent Action Boundary Canvas to define the expected state, verifier, retry rule, partial-failure states, and recovery owner for one action.

What should you take into production?

• Interface success is not business completion. A 200 OK can sit alongside a rejected queue, a failed downstream system or an unhealthy deployment.
• Write the expected-state contract before the tool call, in terms the business would recognise.
• Verify through a path that is independent of the one that executed the action.
• A timeout is unknown state, not failure. The correct state is unknown until the system can reconcile.
• Key idempotency to the business operation, not to the transport request, because the same action can enter through several services.
• Reconcile before retrying. A generic exponential-backoff policy cannot decide whether the first action already changed real state.
• Model the states between success and failure explicitly, and give recovery its own authority boundary.
• Gate the release on the state transition the action produces, not on the model response that describes it.

Common questions

Why do AI agents report unfinished work as done?

A tool can return success for one interface while the business task is still open. The case write can succeed while provisioning fails, so the agent closes the case and the customer still waits. A rejected queue, a double browser submit, or an unhealthy deployment is the same gap. Report done only after a separate read matches the expected state, and keep the run unknown when that read fails.

What is an expected-state contract?

An expected-state contract lists the business facts that must be true after the action. Write it before the tool is called, because after the call the agent can only repeat whatever success message the interface returned. It names the before and after state, the fields that may change, and the facts that must stay true. It also names the downstream effects, the deadline, and which source can verify each one.

Is an HTTP 200 proof that an agent finished?

A 200 is not proof the task finished. A success payload describes what one interface observed at one moment, and it can show a valid, accepted, queued, or stored request. It cannot show that the workflow reached the state a customer or operator would call finished. For a refund, that state can be the refund record, the ledger entry, and the customer-visible status.

What should an agent do when a tool call times out?

Treat the timeout as unknown, not as failure. The request may never have reached the target, it may have failed there, or it may have succeeded while the response was lost. Calling every timeout a failure invites a retry that can duplicate a refund or a message. Read the target through the verification path before any retry.

How do you stop a retry from duplicating an agent action?

Key the operation to the business action, not to the transport request, because the same action can enter through several services or retries. A stable key lets the retry return the recorded result instead of starting a new action. That key limits a second consequence while the verifier is still reading the final state.

What does independent verification mean for an agent?

Independent verification is a separate observation of the result, and it does not require a separate vendor. If the agent writes through an action API, the check reads the system of record on another path. A browser submit is checked from the resulting record or provider receipt, not from the changed screen. The verdict is complete, incomplete, failed, or unknown.

Originally published on bigcircle.ai. I'm Tirth, founder of BigCircle, where we build production AI systems with governance built in.