How a model is actually built, how a system stays loosely coupled to one that changes underneath it every few weeks, and how a population of models and agents behaves once it is running in production rather than in a pilot. Three parts, read in order if you are new to this. Each interactive module works: click, drag, filter.
Written foran engineer joining an AI practice for the first time
Parts3
Interactive modules11
Part one
How a model is built
Everything downstream, contracts, coupling, governance, cost, rests on a small number of mechanical facts about what a model actually is. This part sets those facts out once, so the rest of the book does not have to keep re-deriving them.
The stack has barely changed since 2019
Every model in production today, Claude, GPT, Gemini, Grok, DeepSeek, Qwen, Llama, Gemma, is a decoder-only autoregressive transformer. The block is the same: normalise, attend, gate, add back to a running total. What has changed around that block is efficiency engineering: how the attention cache is compressed, which parameters activate for a given token, what precision the weights are stored at, and what happens after pretraining. Click through the stack below for the mechanism, the formula where one exists, and why it matters once the model is serving traffic rather than sitting in a notebook.
This applies to closed models as much as open ones. The components are common to all of them. What is not public for Claude, GPT, Gemini and Grok is the configuration: layer count, residual width, head count, dense or sparse, parameter count. Wherever a specific number appears below it comes from a published technical report, which is why the worked examples are open-weights models: DeepSeek-V3, Qwen3, gpt-oss, Kimi K2.
Interactive
Layer explorer
Click a component
The finding that matters most
Reading several years of technical reports back to back, the core block is unchanged: pre-norm, attention, gated feed-forward, residual. What changed is the efficiency work around it, the KV cache, the sparsity pattern, the numeric precision, the training recipe. The intelligence gains of the last two years came overwhelmingly from data quality and post-training, not from architecture. Worth establishing before a build decision rests on architecture selection being the variable that matters.
Attention: what the heads do, and what the cache costs
Two things happen in the attention sublayer that matter once a model is serving real traffic. Specific attention heads perform identifiable, testable functions, established through interpretability work on models researchers could open up, including GPT-2, Claude 3 Sonnet and Claude 3.5 Haiku through Anthropic's own interpretability research, and the finding holds across architectures. And the key-value cache those heads populate is the largest cost at any realistic concurrency, because it grows with context length times batch size times precision, not with how clever the model is.
Parameter count is a meaningless comparison metric on its own
In a sparse model the feed-forward block is replaced by many independent expert networks plus a small router. For each token the router scores every expert and activates only the top few. DeepSeek-V3 routes to 8 of 256 experts plus a shared expert. gpt-oss-120b routes to 4 of 128. Kimi K2 routes to 16 of 896. Total parameters and active parameters have diverged by up to 32 times across current models, and both numbers are required before a size comparison means anything.
The catch for enterprises
All experts must be resident in memory even though only a fraction compute on any given token. DeepSeek-V3 at BF16 needs well over a terabyte of weight memory. You pay the memory of a 671B model and get the compute cost of a 37B model.
Sparse models are excellent at high utilisation on a multi-node cluster and poor for a single node at low request volume, which describes most enterprise deployments. This is the single most common sizing error in on-prem LLM projects.
The counter-examples are worth knowing: Mistral Medium 3.5 is dense at 128B and currently the only frontier-credible non-MoE model, and Gemma 4 31B is dense and leads small-footprint deployments. For a fixed single-node budget, dense often wins.
Routing design differs in ways that matter
A shared expert that always fires, alongside the routed ones, stabilises training and captures common patterns so the routed experts specialise more cleanly. Not every model uses one; DeepSeek-V3 does, gpt-oss does not.
Load balancing determines whether experts are used evenly. Auxiliary-loss-free balancing, as used in DeepSeek-V3, adjusts a per-expert bias term directly rather than adding a loss term that trades off against task performance.
Granularity, how many small experts versus few large ones, trades routing flexibility against per-expert capacity. There is no settled answer; it is one of the genuine architecture choices left in the field.
The training pipeline, and where capability actually comes from
A model is grown rather than built, and the recipe matters more than the architecture. The post-training stages are where the last two years of capability gains were manufactured.
Preference optimisation methods compared
Method
Mechanism
Cost
When to use it
RLHF / PPO
Human comparisons train a reward model; the policy is optimised against it with PPO, KL-constrained against the SFT model to prevent collapse
Highest. Reward model plus critic plus rollouts
Frontier labs with dedicated RL infrastructure. Rarely the right choice for an enterprise team
DPO
Closed-form loss directly on preference pairs. No separate reward model, no RL loop
Low. Looks like supervised training
The default for enterprise preference alignment. A slightly lower ceiling, a much simpler pipeline
GRPO
Samples multiple completions per prompt; the group mean reward is the baseline, replacing a learned value network entirely
Medium. No critic, so much lower memory
The method behind the reasoning-model wave. Needs a reward signal to exist
RLVR
Where correctness is mechanically checkable, the verifier is the reward. No reward model, no human labellers
Low per sample, high engineering up front
The one with real differentiation potential for a data practice. See below
Rubric / self-critique
Extends verifiable rewards to unverifiable tasks via a model-generated rubric score
Medium
Subjective quality dimensions where no automatic verifier exists
Distillation / merge
Train several specialised experts, then merge them, for example nine task-specific RL models merged via multi-teacher distillation in Kimi K3
High to produce, cheap to serve
Compressing a large capable model into something you can actually run
RLVR is the capability worth building deliberately
Where a task has a checkable answer, no reward model or human labeller is needed; the verifier is the reward signal. This is a large part of why reasoning models improved so fast: removing the human bottleneck lets the loop run at machine scale. For data engineering workloads, verification is often mechanically available. Does the generated SQL execute? Does it return the expected row count? Does the extracted field match the source system? Does the migrated report produce identical numbers to the original? If correctness can be defined, a model can be improved on your own workloads without labelling anything, which is worth more than fine-tuning on a pile of documents.
Data curation
The least visible stage, the most decisive one, and close to the last real trade secret in the field. Scale varies: Qwen3 used roughly 36 trillion tokens across 119 languages, DeepSeek-V3 14.8 trillion, Kimi K2 15.5 trillion, Llama 4 roughly 30 trillion.
The pipeline is bulk crawl, language identification, deduplication at document and near-duplicate level, quality classification, toxicity and PII filtering, then deliberate up-weighting of code and mathematics beyond their natural web frequency, because both give dense, verifiable reasoning signal.
Your domain is underrepresented. Insurance policy language, clinical coding, trade settlement messaging, utility asset hierarchies are not in a web crawl. That is the root of most "the model does not understand our business" complaints. It is a data problem, not an architecture problem.
Optimizer and stability
AdamW is the standard optimizer. The interesting exception is Muon, used by Kimi K2 for better token efficiency. At scale it hit a wall: attention logits rapidly exceeded 1000 mid-run, risking divergence. The fix, MuonClip, added QK-Clip to rescale query and key projections. With it, Moonshot pre-trained a trillion-parameter model on 15.5 trillion tokens with zero loss spike.
A clean illustration that architecture, optimizer and normalisation are not separable concerns. QK-Norm, described below, solves the same underlying problem structurally rather than at the optimizer level.
Parallelism
Five strategies compose: data parallel splits the batch, tensor parallel splits individual matrices, pipeline parallel splits layers, expert parallel distributes MoE experts, sequence parallel splits sequence length.
DeepSeek built DualPipe specifically to overlap computation with communication and hide pipeline bubble idle time, and hand-wrote cross-node all-to-all kernels because the off-the-shelf ones bottlenecked. Their reported 2.788 million H800 GPU hours is the marginal cost of the final run, not of the capability; it excludes failed runs, data pipeline construction, staff time, and the ablations that found the recipe.
Quantization, and getting the lost quality back
Quantization is how a model that needs 240GB fits on one 80GB card. It is also where quality is lost in ways benchmarks often miss.
What actually breaks first: activation outliers
Dettmers et al. established that activation outliers emerge suddenly around 6.7B parameters, affecting all transformer layers and roughly 75 percent of sequence dimensions. They are about 0.1 percent of features, yet zeroing them collapses top-1 attention softmax mass by over 20 percent and degrades validation perplexity by 600 to 1000 percent. Mechanically they stretch the quantization range so every other value compresses into a handful of buckets. This single phenomenon explains why weights quantize easily and activations do not, and why 4-bit weights with 16-bit activations is near-lossless while 4-bit activations remain hard.
Post-training quantization methods
Method
Mechanism
Handles outliers by
Practical note
LLM.int8()
Mixed-precision decomposition: the 0.1 percent outlier dimensions stay in 16-bit, the rest multiply in 8-bit
Isolating them
Roughly 1.96x memory reduction on BLOOM-176B with essentially no loss. The reference case for why 8-bit is close to free
SmoothQuant
Migrates quantization difficulty from activations to weights via per-channel scaling before quantizing
Moving the problem
Enables 8-bit weights and 8-bit activations, which is what makes 8-bit activations viable at all
GPTQ
Layer-wise weight updates using approximate second-order Hessian information to minimise reconstruction error
Compensating for them
Strong at 4-bit, extends to 3-bit. Sensitive to calibration data
AWQ
Identifies the roughly 1 percent of salient weight channels by activation magnitude and protects them with per-channel scaling
Protecting what they touch
Activation-aware, so calibration data quality matters directly
MXFP4
Two fp4 values per byte with a shared power-of-two scale per 32-element group, 4.25 bits per parameter
Group-wise scaling
Used natively by gpt-oss and Kimi K3, applied during post-training rather than after, which is why it holds quality
NF4 (QLoRA)
Each value assigned to one of 16 quantile bins of a normal distribution; double quantization compresses the scales to 8 bits
Distribution-matched bins
Built for fine-tuning on quantized frozen weights, not primarily for serving
Getting the quality back, in ascending cost
1 · Calibration data quality
The cheapest lever and the most neglected one. GPTQ and AWQ both depend on a calibration set to estimate activation distributions. Generic calibration data on a domain-specific workload produces biased quantization parameters. Calibrate on your own representative traffic; family-aware calibration, which regenerates calibration samples using a larger model from the same family, reported up to 28.5 percent less accuracy loss than baseline calibration data on Qwen3-8B, per recent work on family-aware calibration.
2 · Quantization-aware training
Fine-tune the quantized model with the same loss function, quantizing weights and activations in the forward pass while gradients stay high precision. PyTorch reported QAT recovering 62 percent of normalized accuracy degradation on hellaswag versus post-training quantization, for a comparable Llama configuration. Cost is a retraining run.
3 · Adapter-based repair
Train a low-rank adapter on the quantized model, distilling from the full-precision original on synthetic data. Recent work on selective 2-bit quantization reports 7.5 to 23.3 percent throughput gains over uniform 4-bit across 4B to 20B models, with an adapter reclaiming accuracy that would otherwise be lost. Far cheaper than full QAT.
4 · Quantization-aware distillation
For models whose capability came from RL post-training, plain QAT is awkward, because quantization-aware RL is barely explored, per recent quantization-aware distillation work. Distilling from the full-precision model instead sidesteps the problem, which matters increasingly because reasoning capability is RL-derived and is exactly the capability most damaged by aggressive quantization.
Two failure modes nobody tests for
Safety degrades separately from capability. Research on quantized model safety found that quantization can degrade alignment even where utility benchmarks hold, and calibration data with harmful content makes it worse. If a model is quantized and only task accuracy is measured, what changed has not actually been measured.
Degradation is also not uniform across tasks or even across model families with the same parameter count: the Llama3-70B series showed distinct susceptibility traced to weight outliers in its early layers, which Llama2 and the 405B variant did not share. Test on your own workload. Never on someone else's leaderboard.
Evaluation is the artefact that makes everything else safe
Public benchmarks are useful for a rough ranking and close to useless for choosing a model for your own work. With a new release roughly every week, model swaps are now routine, and without your own evaluation set there is no way to tell whether any change helped.
Why public benchmarks fail you
Contamination. Test sets leak into training corpora. A model may have memorised the answers rather than derived them, with no reliable way to audit this from outside
Saturation. Top models now cluster within noise of each other, a spread too narrow to decide anything
Construct mismatch. None of the standard public benchmarks measures whether a model can extract entities from your claims documents
Self-reporting. Vendor-published charts are selectively framed against an independently scored baseline
The evaluation layers you need
Component evals. Retrieval measured separately from generation; most RAG failures are retrieval failures that look like generation failures
Task evals. A few hundred real examples with agreed correct answers, scored automatically. About a week of work, and the highest-value asset the practice will build
Regression evals. Run on every model swap, prompt change and quantization change, which is what makes routine model migration safe
Agent trajectory evals. Did it use the right tools in the right order. Getting the right answer by luck is not a working agent
Production evals. Sampled live traffic, scored continuously, because distribution drifts
Scoring methods, and their failure modes
Exact match. Free, deterministic, hard to argue with. Use it everywhere correctness is checkable; the same property that enables RLVR
Execution-based. Does the SQL run and return the expected shape. Strongest signal for engineering workloads
LLM-as-judge. Scales well but inherits position bias, verbosity bias and self-preference. Check it against human labels before trusting it, and re-check whenever the judge model changes
Groundedness checks. Is every claim in the answer supported by a retrieved chunk. The single most useful production check for RAG
The observability layer this rests on
Evaluation without instrumentation is guesswork. Standardise on OpenTelemetry GenAI semantic conventions for spans and attributes. Track token accounting including hidden reasoning tokens: they bill as output and do not appear in the response body, so a cost model built on visible output length will be wrong by an order of magnitude on reasoning models. Decompose latency into time to first token, which reflects the prefill phase, and inter-token latency, which reflects the decode phase; they have different fixes. A first-token regression points at oversized context. An inter-token regression points at memory bandwidth and batch composition.
The rest of the stack, where the failure modes actually live
A raw model answering a raw prompt is not what reaches production. A layer of infrastructure sits between the two, and each layer exists to prevent a specific, named failure.
Layer
What it does
Failure mode it prevents
Constrained decoding
A grammar compiled from the schema masks non-conforming tokens at sampling time, via JSON mode, XGrammar, Outlines or grammar-based decoding
Malformed output breaking parsers. Note: structure is guaranteed, semantics are not
Semantic layer
The model queries governed metric and dimension definitions rather than raw schemas; the emitted query is parsed and guarded before execution
Hallucinated joins and invented metric definitions. The most effective anti-hallucination control for analytics agents
Input guardrails
PII detection and redaction, prompt-injection and jailbreak classifiers, before the model sees the request
Injection reaching the model; regulated data leaving the estate
Output guardrails
PII, toxicity, policy and groundedness checks after generation
Ungrounded assertions reaching a user. Never rely on model alignment alone
Identity propagation
OAuth token exchange so the agent acts with the user's scoped permissions, passed to the data layer so row-level security applies at query time
The confused deputy problem, covered below
Gateway / router
One abstraction across providers, centralised auth, model routing, fallback chains
Provider lock-in and single-provider availability risk
Contextual retrieval, prepending a short model-generated blurb situating each chunk before both embedding and keyword indexing, measurably improves retrieval; see Anthropic's writeup. Hybrid search fuses dense embeddings with keyword search, because keyword search catches the exact matches dense retrieval reliably misses: part numbers, policy codes, identifiers and dates. Those are the tokens enterprise queries actually turn on.
Agentic systems: memory, instruction, and how agents talk
Three things separate a working agent from a demo: how it remembers across sessions, how it is instructed, and how it talks to other agents and to your systems.
The mistake almost every team makes
Collapsing all memory types into a single retrieval problem. Working memory is a context-budget problem, not a retrieval problem; there is nothing external to retrieve from, so it has to be managed by compression and prioritisation. Running semantic similarity search across episodic logs is the specific error. Episodic memory is indexed by time and cause, not by meaning, and treating "what did I try last Tuesday" as a nearest-neighbour question returns results that look right and are not. See CoALA for the cognitive-architecture framing this draws on.
Prompts, skills, and where instruction actually lives
Instruction reaches a model through four channels with different persistence, cost and cache behaviour. Confusing them is a common reason agents behave inconsistently across sessions.
Channel
Persistence
Cache behaviour
What belongs here
System prompt
Every request, whole session
Stable prefix, cheap to cache
Role, constraints, output conventions. Keep it stable; every edit invalidates the cache for everything after it
Tool schemas
Every request while tools are available
Stable prefix, though changing tools mid-conversation has historically invalidated the cache
The capability surface. There is real overhead just from having tools available at all
Skills
Loaded conditionally, on relevance
Loading one mid-session breaks the cache prefix unless skills sit in a stable block
Procedural memory made explicit: reusable task recipes, house style, review checklists
Retrieved context
Per request
Never cacheable
Facts, not behaviour. Retrieval solves knowledge; it does not solve how the agent works
Agent communication protocols
Four protocols address four different layers. They are complementary, not competing.
Protocol
Layer
Mechanism
Enterprise position
MCP
Agent to tool
JSON-RPC 2.0. Hosts run clients connecting to servers exposing tools, resources and prompts. See the transport specification
Adopt first. Solves connectivity, not governance
A2A
Agent to agent
Agents discover each other via Agent Cards, JSON metadata describing capabilities and endpoints. See the A2A specification
For cross-team and cross-vendor work. Answers which agent can handle this task, nothing about how to disagree with the answer
ACP
Agent to agent, negotiated
Formalises negotiation with typed performatives: propose, accept, reject, counter. Merged under the A2A umbrella in 2025
Useful where agents must negotiate rather than simply hand off work. Rarely used
ANP
Open agent markets
Decentralised discovery, identity verification, peer-to-peer communication over linked-data graphs
Not yet an enterprise concern
None of these express governance
Recent analysis of governance gaps in agent protocols found that MCP, A2A and ACP all lack the vocabulary to express policy constraints, data handling obligations or accountability. This is the same finding Part Three returns to at the level of a whole enterprise: the protocol layer is necessary and nowhere near sufficient.
The confused deputy problem
The hardest unsolved problem in enterprise agent architecture. A privileged agent is tricked into using its authority on an attacker's behalf, usually via a prompt injection embedded in a document or a tool result. The agent is not compromised; it is doing exactly what it was told, by the wrong party. A shared service account makes this worse, because an agent with broad credentials can read everything, so any successful injection reaches everything. The layered defence is delegated auth carrying the user's scoped permissions, identity propagated to the data layer, least-privilege scopes per tool, human approval for high-impact actions, and one explicit invariant: content arriving from retrieval, tools or other agents can never change the agent's own permission set. Design for this before the first agent reaches production, not after.
Open weights or closed API, for this workload
There is no single answer to open versus closed. There is an answer per workload, arrived at by weighing a small number of factors against each other.
Open weightsClosed API
The line worth carrying into Part Two
The architectural gap between a frontier closed model and a strong open model is small and narrowing. The gap in data curation, RL infrastructure and post-training scale is large. But neither one decides whether an enterprise deployment succeeds. That is decided by data access, decomposition, verification, evaluation, and the memory and instruction architecture around the model, which is the subject of Part Two: what has to hold steady when the model underneath it does not.
Part two
What has to hold steady
Part One established that the model changes underneath you every few weeks, with no migration notice and no version number that means anything behaviourally. This part is about the discipline that lets a system survive that: loose coupling, and the contract that makes looseness safe rather than chaotic.
The word is coupling
If changing one part forces you to change another, the two are coupled. It is the oldest idea in system design, and the profession has been trying to loosen it since the 1970s. If swapping a model turns into a project, the system is tightly coupled to that model, not metaphorically, in the ordinary engineering sense of the word.
We know how to talk about this. We seem to have stopped, right at the moment it started to matter most.
The reason the vocabulary went missing is worth noticing. Coupling is a structural property, and for two years the conversation has been about capability instead. Every improvement in what models can do has been met with more scaffolding around them, and each piece of scaffolding was added for a good reason: longer prompts because the model reasons better, more graph nodes because it uses tools better, more elaborate orchestration because it plans better. The scaffolding is the coupling. Every increase in capability has been spent on machinery that binds a system to the particular model it was built against.
Where it hides in an AI system
It would be obvious in Java
Orchestration graphs that encode every branch. Forty nodes of control flow is a monolith with network hops in it. The graph becomes the business logic and the model becomes a subroutine inside it
Prompts tuned to one model's habits. Change the model and the application breaks. That is coupling by definition; no amount of prompt engineering discipline makes it otherwise
Business rules inside tool descriptions. Logic hidden in prose, invisible to review and untestable, silently coupling the tool to one model's reading comprehension
Evaluation written against one output format. The ability to measure is now tied to a single vendor, which means it is not safe to leave
Retrieval tuned to one embedding model. Re-embedding a corpus becomes its own migration project
Every agent sharing one service account. Each agent is coupled to the union of all permissions any agent needs
Data people already know how this ends
This is not a new fight. It is one the data profession has been losing for twenty years, and the shape of the loss is instructive. Loose coupling was always hardest in data, because data has no interface. The schema is the interface, and it leaks into everything downstream. Rename a column and forty jobs fail.
Data contracts were the attempt to fix it. Declare the shape, the semantics and the guarantees, then let both sides move independently. It is exactly the right idea; see Fowler's data mesh writeup for the fuller argument. Most teams still have not landed it, and the reasons they have not are the reasons this next attempt will be hard too: the contract requires agreement between people who do not report to each other, and agreement is slower than code.
We are now rebuilding that problem in AI, one orchestration graph at a time.
The difference this time is speed. A schema changed once a quarter. A model changes every few weeks, and unlike a schema change it arrives without a migration notice, without a version number that means anything behaviourally, and with no way to know in advance what will break.
Loose coupling has never meant less structure
The parts move independently because the contract between them is stronger.
Decoupling without a contract is not loose coupling. It is disconnection, and it produces chaos rather than flexibility. The looser the parts should be, the tighter the agreement has to be about what passes between them. Service-oriented architecture made this trade explicitly: loose coupling was paid for in schema discipline, versioning, idempotency, backpressure and circuit breakers. Nobody called that minimalism. It was moving the rigour to the boundary.
The AI version is the same trade. Orchestration thins and the contract layer thickens. That is a large amount of engineering, and it does not look like control flow, so it reads as less.
The one objection that is not psychological
In a conventional loosely coupled system, a component's behaviour inside its contract is predictable. In an AI system it is not. That difference is real, and it is not solved by telling architects to relax. It is solved by putting the determinism where it can live: not in the model, but in the conditions around it. The SQL either executes or it does not. The extracted field either matches source or it does not. Determinism at the edge is what makes latitude in the middle safe, which is why verification matters more than orchestration.
It was never going to be one model
The common framing is a model with agents around it. What actually forms inside an enterprise is a mixed population, each member good at something different and bad at something else. Nobody designs it. It accumulates.
The generalist
broad, expensive
Enormous breadth, strong reasoning, slow and costly. Knows a little about your industry and nothing about your company.
The specialist
narrow, fast, cheap
Fine-tuned or heavily prompted for one task class. Excellent inside its lane, unreliable outside it.
The verifier
deterministic, boring on purpose
A rule, a schema check, a SQL execution. Not a model at all, and the most trustworthy member of the population.
The router
classifies, does not answer
Decides which of the above should handle a given request. Its own taxonomy, its own blind spots.
It compounds when
Each member has a defined place: what comes before it, what depends on its answer, and a verifier sits at the boundary rather than trusting output directly. A mixed population under those conditions is cheaper and more reliable than any single model run everywhere.
It contradicts when
Two members are asked the same question with no arbitration rule for disagreement, or a specialist's output reaches a downstream system with no verification step, or the router's own classification scheme is undocumented and nobody can say why a request went where it went.
The platforms loosen one coupling, not the one that hurts
It is worth being precise about what the routing and gateway layer actually solves, because the marketing around it implies it solves more than it does.
Loosened
Solved
Coupling to the endpoint
Which URL, which SDK, which auth scheme. A gateway or router genuinely solves this: swap the provider behind the router and the calling code does not change.
Still tight
Unsolved
Coupling to the behaviour
Which prompt phrasing this model responds to, what its tool-calling habits are, how it handles ambiguity. None of that travels through the router. Swapping the endpoint does not swap the behaviour a prompt was tuned against.
Assume every piece of tooling gets absorbed
Most of it already has
Retrieval, evaluation harnesses, guardrails, observability: the pattern of the last three years is that whatever a team builds bespoke, a platform ships a version of within a year or two. That is not a reason to avoid building it. It is a reason to build the parts that will not be absorbed, which are almost always the ones requiring a decision about your own business, not a general capability.
The contract is semantic, which makes it a human problem
Not every clause in the agreement is enforced the same way, and treating them as interchangeable is the most common design error in AI governance. Taught rules generalise and fail at the tail. Forced rules hold at the tail and cannot hold judgement. The frontier labs build both and deliberately keep them apart: a written specification the model reasons over, plus classifiers outside the model that block regardless of what it concluded. See Constitutional AI and Deliberative Alignment for how frontier labs approach the taught half, and recent work stress-testing how well models actually follow their own constitutions for how far that gets you at the tail.
Which parts of the contract are taught, and which are forced
Obligation
Mechanism
Reason
Never send PII outside the estate
Force
One violation is enough. Block at egress, not in the prompt
Only read this user's rows
Force
The agent should be unable to see other rows, not asked not to
Output must match the schema
Force
Constrained decoding makes it valid as it is generated
Stay inside the spend cap
Force
Countable. A cost governor, not an instruction
Ask when a request is ambiguous
Teach
Ambiguity cannot be listed in advance
Follow intent over literal wording
Teach
No rule set covers this without being wrong somewhere
No irreversible action without approval
Both
Teach the pause. Force a gate on the operations that can be named
Use approved metric definitions
Both
Teach the semantic layer. Expose only governed definitions
Stay within the assigned task
Both
Teach goal adherence. Scope credentials so drift reaches nothing
We assumed agents would spare us the hard part
Getting people to collaborate was always the hardest part of this work, not the technology, the alignment. There was something appealing about agents, which follow instructions and have no politics. They turn out to be harder in one specific way.
We have given them no language to disagree in.
Every agent protocol models delegation. Here is the work, here is the result. A2A has Task, Message, Artifact and Agent Card. None of them can carry I disagree, and here is why. ACP is the only one that borrowed performatives from the older multi-agent literature, and almost nobody uses it. So the current answer to conflict is suppression: an orchestrator picks a winner, or a vote decides, or last writer wins. The disagreement is destroyed at the moment it appears, and one confident answer reaches the person with no trace that another path existed.
They are trained to agree
Preference training rewards agreeableness. Point that at another agent and premature convergence follows. The debate literature keeps finding groups of models converging on wrong answers through conformity, and adding more agents does not fix it. This is a post-training property, not a protocol gap, so no message schema solves it.
There is no standing
In an organisation an objection carries weight because of who raised it; risk has standing on risk. That standing is what makes an objection blocking rather than advisory. Agents have no standing model, so every objection weighs the same, which means none of them weigh anything.
Nothing confers legitimacy
Minutes, sign-offs, recorded dissent, escalation paths were never really about reaching the right answer. They are about people being able to live with a decision they lost. Agents produce decisions with no legitimacy trail, part of why nobody hands them anything consequential yet.
Noise: same grounds
Two instances of one model, same context, same tools, disagreeing. That is sampling variance. Aggregating it tells you the temperature was above zero and nothing else.
Signal: different grounds
Different data access, different objectives, different tools, different constraints. A retrieval agent and a policy agent disagreeing means something, because they are looking at different things and optimising for different outcomes.
Which makes it a design choice, not a checklist item
If disagreement is to be informative, the difference has to be engineered. Most multi-agent systems spawn identical sub-agents and read the variance as a second opinion, which is measuring your own temperature setting. And deliberation costs latency, tokens and complexity, with a real failure mode where expensive ceremony gets built around noise. Verification and deliberation are substitutes: if the answer can be mechanically checked, check it. Deliberation is what you reach for when no verifier exists.
Interactive
Should this decision be deliberated?
Six questions about one decision class
The thing worth doing regardless
When an objection is overruled, record it: the objection, its grounds, and the reason it lost. This needs no new protocol and no model changes, and it does two things at once. A decision becomes defensible rather than merely confident, because an auditor can see the alternative existed and why it failed. And where agents with genuinely different grounds keep disagreeing on the same class of decision, the contract was never actually written for that case. Disagreement becomes a detector for the parts of the agreement nobody made.
Why the word primitive is the right one
A primitive is the smallest useful thing that cannot be broken down further, and that everything larger is built out of.
Not a feature. A feature is adopted. A primitive is composed with others, and behaves the same wherever it is placed
Not a best practice. A practice tells you what good teams do. A primitive is a part, and parts can be tested and replaced
Not a product. Products come and go. A timeout is a timeout whoever ships it
Distributed systems only got loose coupling once it had named its primitives: timeout, retry, idempotency key, circuit breaker, bulkhead. Before those names existed, every team improvised, and nothing anyone built composed with anything else. Naming is not academic. It is what makes something portable across vendors, teachable to someone new, and possible to disagree about precisely.
The ancestry, primitive by primitive
Distributed systems primitive
What it decoupled
AI equivalent
State
Interface contract
Caller from implementation
Tool schema plus constrained decoding
Mature
Schema versioning
Deployment cadence between services
Behavioural contract for a model version
Missing
Idempotency key
Retry safety from delivery semantics
Idempotent tool actions and replay tokens
Emerging
Timeout
Caller liveness from callee latency
Step budget, token budget, wall-clock cap
Mature
Circuit breaker
System health from one dependency
Kill switch, fallback chain, provider failover
Mature
Bulkhead
Blast radius from failure
Sub-agent context isolation, scoped credentials
Emerging
Backpressure
Producer rate from consumer capacity
Cost governor, rate and spend limiting
Emerging
Distributed trace
Debuggability from call depth
OpenTelemetry GenAI trace, trajectory logging
Mature
Contract test
Confidence from full-stack runs
Task eval set, regression suite
Mature
Declarative policy
Policy from application code
Written specification, runtime policy engine
Emerging
Capability token
Authority from identity
Scoped delegated credentials per action
Emerging
Attestation
Trust from co-location
Proof a policy was applied, not just declared
Missing
Recorded dissent
Legitimacy from unanimity
Objection, grounds and overrule reason retained
Emerging
Forty eight primitives, sorted by when they apply
Teaching time shapes what a model tends to do. Execution time constrains what it is able to do. Build and observation time establish what can be verified and what can be proven to have happened. Filter by phase or by maturity below.
Most of the contract sits outside the model
Execution time carries more than half of the total, at boundaries the model cannot reason past. That is the opposite of where most attention and most vendor marketing goes, and it is the strongest evidence that the model is not the variable that matters most.
The order is a dependency chain
Evaluation comes first, not last, because until it exists nothing else can be checked. Verification at build time is what produces the verifiers reinforcement learning later requires. Sequencing this wrong wastes much of what came before.
What follows from all of this
No stack needs all 48 primitives equally. What varies is not which primitives are real, it is which ones carry the weight for a given workload. Building the right ones, in the right order, is closer to a sequencing problem than a shopping list.
Sequencing
A dependency chain, not a preference
1
Contract
Tool schemas, constrained decoding, the interface layer. Nothing else is buildable until callers and implementations can actually agree on shape.
2
Bound
Timeouts, budgets, circuit breakers. Cheap, mechanical, and they contain the blast radius of everything built after this point.
3
Observe
Tracing and logging, before anything else is measured. Without this, every later stage is debugged by guesswork.
4
Verify
Task evals, regression suites, groundedness checks. This is what turns a model swap from a leap of faith into a routine change.
5
Govern
Declarative policy, scoped credentials, forced constraints at the boundary. What step four proved correct, this keeps correct under load.
6
Teach
Reinforcement learning against your own verifiers, then distillation. Impossible without step four, because step four builds the verifiers this step requires. Unlocks capability specific to your own work.
7
Federate
Agent-to-agent delegation across teams and vendors. Last, because it multiplies every weakness above it, and two of the missing primitives sit exactly here.
Six practical shifts
Run a swap drill on a schedule. Drop in a comparable model on one workload. Whatever breaks first is what the system is coupled to, and now it has a name
Protect the contract, not the build. If the eval set and the definitions are line items inside each project, they get cut when that project is squeezed, and the next project pays for them again
Give every model and agent a place, not just a task. Which process it sits in, what comes before, what depends on its answer. This is what makes a population compound instead of contradict
Give the arbitration a named owner. Deciding what correct means is not a technical task looking for a technical owner. It needs a person, a forum, and a written record
Record the disagreement. Keep the objection, its grounds, and why it lost. It shows exactly where the contract was never written
Measure how long a swap takes, not how much has been adopted. One of those numbers tells you something real. The other tells you what got switched on
Where this argument is weak
Typed disagreement assumes agents can introspect accurately on why they disagree. Given what the research says about chain-of-thought faithfulness, that assumption is carrying a lot of weight. Formal argumentation frameworks have a long history of being elegant and unused; Dung's semantics and IBIS are decades old, and there is no strong reason this time is different beyond the participants now being cheap enough to run many of them. Deterministic replay may not be achievable at all: even at temperature zero, floating-point reduction order varies with batch composition, so a compliance story should be designed around recorded evidence rather than reproducibility.
The person who rebuilt the same thing five times was not doing bad work. They were building tightly against something that was never going to hold still.
The question worth asking is what the contract should have been. Part Three follows that question to where it actually bites: not one model changing under one application, but a population of agents, across vendors, in production, discovering that the policies written for the pilot never anticipated what happens at runtime scale.
Part three
What happens once agents meet in production
Parts One and Two covered a single model and a single application's contract with it. This part is about what happens once many of them, across vendors, are acting inside one enterprise at the same time, and the policies written for the pilot meet a runtime that was never sized against.
Why this paper exists
Technical Paper 01 argued that meaning is a function of three variables: when it was true, what it is for, and by whose warrant. It treated the consumers of meaning as mostly human, or as single agents working inside one estate.
That assumption stopped holding some time in the last eighteen months.
Three tiers of the enterprise stack shipped agentic products more or less simultaneously. Systems of record went agentic. Data platforms went agentic. Customer-facing applications went agentic. Each tier now contains software that plans, calls tools, and takes actions with reducing amounts of human supervision, and each tier's vendor is positioning its own layer as the place where those agents should be coordinated.
None of them are wrong about the need. All of them are proposing themselves as the answer.
This paper is about what happens in the gap. It covers what each tier can actually do today as opposed to what is announced, what the published evidence says about how agents behave when they meet each other, why the interoperability protocols that exist do not solve the problem people think they solve, and what a semantic core has to become if it is going to be the common ground rather than another proprietary layer.
It starts with money, because a coordination argument with no cost attached to it does not get built.
A note on evidence. Vendor claims in this space are unusually difficult to check. Wherever a number comes from a vendor or a partner, this paper says so. Wherever a capability is generally available, in preview, or announced only, this paper distinguishes them, because vendors mostly do not. Where the evidence contradicts our own thesis, that is stated rather than omitted, and there are two places where it does.
Part 0: The two questions that decide the budget
An agent fails to handle a case correctly. There are two possible reasons, and they lead to opposite spending decisions.
The first is that the model is not capable enough for the task. The second is that the model does not know how this company works. Buying more intelligence fixes the first and does nothing for the second, and most enterprises cannot tell which one they are looking at.
That is the whole argument for a semantic core, stated as a purchasing question rather than an architectural one.
0.1 Two kinds of work
Some work creates value by expanding what is known. Proving a theorem, finding a novel vulnerability, identifying a drug target. For that work, more capability buys more outcomes, and spending heavily on computation is the correct decision.
Most work creates value differently, and the pattern is worth naming precisely because it is the argument for everything that follows. An underwriter is not discovering a new risk model. She is applying the book of business rules her firm already holds to one submission, with its own property, its own history and its own broker. A branch teller approving an exception is not reasoning from first principles about fraud. He is checking whether this request matches a pattern the bank has already seen and already has a rule for. A power grid operator rebalancing load after an outage is not inventing physics. He is applying a known playbook to today's weather, today's demand and a specific set of substations. None of these is a discovery problem. Each is a problem of applying judgment the institution already possesses to one specific case.
The distribution is lopsided. Life, physical and social science occupations account for under one percent of American employment. Against roughly 2,000 mathematicians and 20,000 physicists sit 2.6 million people in customer service, 3.4 million registered nurses and 3.5 million operations managers.
An enterprise's defensibility rarely comes from thinking better than its competitors. It comes from proprietary data, operational context, supplier relationships, regulatory licences and accumulated judgment from performing the same class of decision several million times. That is a context asset, not an intelligence asset.
Jaya Gupta, Avanika Narayan and Jon Saad-Falcon put the distinction well in August 2026: the frontier is an intelligence problem and the rest of the economy is a context-and-execution problem. We would add one thing to it. The context asset they describe is exactly what a semantic core is for, and most organisations own it without having written it down anywhere a machine can read.
0.2 Thresholds, and what happens after one is crossed
Every workload has a capability level below which a model cannot perform it at all. Approach that level and additional capability produces large gains. Cross it and the gains flatten, because the constraint moves.
After the threshold, outcome quality depends on whether the system knows the firm's policies, its customers, its history and its tools, and whether it can act on that knowledge reliably and cheaply. Additional reasoning does not supply any of those things.
Two facts make this expensive. Models keep improving, on a curve that has not flattened yet. Underwriting exceptions, address changes and grid rebalancing do not improve on any curve at all, because the difficulty of the task was never the constraint. So the population of workloads sitting above their threshold grows every quarter while the routing that sends work to a model stays fixed.
Figure 0AEvery workload has an intelligence threshold, and most enterprise work crossed it
BELOW THE THRESHOLD
The model cannot do the task. More capability buys everything. Spending more computation to explore more possibilities is rational.
ABOVE IT, THE BOTTLENECK CHANGES
The outcome now depends on whether the system knows the company's policies, customers, history, tools and standards, and whether it can act reliably, quickly and cheaply.
THE DIAGNOSTIC QUESTION
Is the task failing because the model is not intelligent enough, or because it does not know how this company works?
Model capability keeps rising. The difficulty of an address change or a rebalancing decision does not.
0.9%
of US employment is in life, physical and social science occupations
DISCOVERY ORIENTED
Mathematicians2K
Physicists20K
Computer research37K
SITUATED EXECUTION
Customer service2.6M
Registered nurses3.4M
Operations managers3.5M
Home care aides4.3M
Not easy work. Situated work. Value comes from applying known judgment in local context, which is a context problem rather than an intelligence problem.
Gupta, Narayan and Saad-Falcon, Right-Sizing Your Intelligence Spend, August 2026 · BLS occupational data, May 2025
The measured consequence is over-provisioning rather than under-performance. Researchers studying reasoning models have documented an overthinking pattern where additional inference compute is spent on easy questions without improving accuracy. Amazon researchers estimate that reasoning systems can generate seven to ten times as many tokens as necessary on simple tasks. On sufficiently easy work the marginal reasoning token contributes approximately nothing.
0.3 Why this does not correct itself
The incentives on either side of the transaction point in opposite directions, and it is worth being explicit about it because the asymmetry is structural rather than temporary.
The two sides of the transaction are not measuring the same thing, and they were never going to converge on their own. A model provider's growth depends on usage: more tokens billed, longer sessions, more agents spun up, deeper reasoning chains. An enterprise's return depends on the opposite: fewer tokens per resolved case, fewer retries, less time spent per outcome. One party grows by being consumed more. The other improves by needing less. There is no version of this relationship where both curves rise together indefinitely.
The moment an enterprise learns enough about its own domain to convert a recurring request into a deterministic rule, the provider loses recurring revenue. Nobody upstream is incentivised to help an organisation need less.
The gap is visible in the aggregate figures. Reasoning token consumption per enterprise organisation rose roughly 320-fold over a year. Over a comparable period, PwC surveyed 4,454 chief executives and 56 percent reported no significant financial benefit from AI. Usage is scaling considerably faster than outcomes.
There is a second reason this does not correct itself, and it is specific to the build-to-run transition rather than the incentive structure. A pilot is, almost by construction, close to the linear case: one call in, one call out, a small and well-behaved context window. Gartner's March 2026 analysis of production agentic workloads found they consume five to thirty times more tokens per task than a standard chatbot exchange, and that enterprises that had scaled past the pilot phase discovered this multiplier only after the production bill arrived. The pilot's token profile simply does not predict runtime consumption, because the retries, tool calls, and reasoning loops that dominate cost live entirely in the part of the system a pilot never exercises. EY's estimate of the same shift, priced: a simple linear workflow cost about four cents per interaction in 2023; an orchestrated system with tools, reasoning, and iterative loops now costs roughly a dollar twenty, about thirty times higher.
This is the same asymmetry as the semantic core argument, one layer down. A policy, a budget cap, or a routing rule written against pilot-era token economics is being asked to govern a workload whose actual shape it never saw. The gap is not that the policy was wrong when it was written. It is that build-time and run-time consumption are different distributions, and nothing about crossing from one to the other announces itself in advance.
Figure 0BThe same rise in usage is a provider success and an enterprise failure
MODEL LAB SCOREBOARD
More billable usage
success = more revenue from usage
ENTERPRISE SCOREBOARD
Lower cost per successful outcome
success = lower cost per verified outcome
320×growth in reasoning token consumption per enterprise organisation, in a year
7 to 10×more tokens than necessary generated by reasoning systems on simple tasks
56%of 4,454 CEOs report no significant financial benefit from AI
A model dashboard showing usage up five times is reporting a win. It answers nothing on its own about whether the enterprise using it is better off, because the two scoreboards are not measuring the same outcome and were never designed to.
Scoreboard framing after Gupta, Narayan and Saad-Falcon, August 2026 · Amazon research on reasoning overhead · PwC Global CEO Survey 2026
0.4 Deciding once instead of every time
There is a cheaper architecture, and the evidence for it is specific.
In the Minions work, a large cloud model decomposed complex long-context problems and smaller local models executed the resulting subtasks. The hybrid recovered 97.9 percent of the large model's accuracy at 5.7 times lower cost, and a more aggressive configuration held 87.9 percent at 30.4 times lower cost.
The interesting result is architectural rather than financial. Expensive capability was most valuable for working out what needed to be done, and least valuable for performing each unit of it.
Technical Paper 01 makes the same split without the cost framing. Meaning is decided at build time, once, by people with authority to decide it. Run time then executes against a decided meaning. The alternative is that every call rediscovers the same context, and pays for the rediscovery every time.
Paper 01 also contains the measurement. Unbounded graph traversal consumed 46,949 tokens per question against 3,743 for bounded retrieval, and produced a worse answer. That is not primarily a retrieval quality finding. It is twelve and a half times the spend, incurred because the system had to reconstruct context it should have been given.
Figure 0CSpend the expensive intelligence once, at build time
RE-DERIVE EVERY CALL
DECIDE ONCE, EXECUTE CHEAPLY
THE MINIONS RESULT
97.9%of frontier accuracyat 5.7× lower cost
87.9%of frontier accuracyat 30.4× lower cost
Expensive intelligence was most useful for deciding what work needed to be done, not for performing every unit of it.
This is Technical Paper 01's build time and run time split, expressed as a cost curve. The 46,949 against 3,743 figures are from Paper 01, where they were filed as a retrieval quality finding. They are an allocation finding.
Minions, via Gupta et al. August 2026 · token comparison from arXiv:2509.16780
0.5 What follows from this
Three consequences run through the rest of this paper.
A control point in the call path pays for itself before it governs anything. Part 8 argues for a gateway that can deny an action. The same component, placed slightly earlier, decides for each unit of work whether it belongs on a large model, a small one, a deterministic rule, a human, or nowhere. The routing case has a payback number. The governance case does not, and arrives free once the component exists.
That payback number now exists in production, not only in a benchmark. Cursor launched Cursor Router in July 2026, generally available for Teams and Enterprise, as a classifier trained on over 600,000 live coding requests and evaluated in an online A/B test across millions of live requests, optimising for user satisfaction and for keep rate, how much of the generated code survives in the codebase, rather than for an offline eval score. It analyses each request on query, context, task complexity and domain, and offers three settings, Intelligence, Balance and Cost, that move a team along the same cost-intelligence frontier this paper has been describing. In early access, three high-volume enterprise accounts with thousands of users saved 30 to 50 percent against routing everything to the frontier model, with no drop in quality, and Auto Intelligence mode matched near-frontier satisfaction at roughly 60 percent lower cost across the wider A/B test, while Auto Balance landed above the strongest daily-driver model at about 36 percent lower cost. Priced per unit of finished work rather than per request, Intelligence mode cost 6.76 dollars per commit and Balance 4.63 dollars, against 12.69 dollars for the top frontier model run undiscounted and 7.34 dollars for the next model down; a third competing model matched Intelligence mode's cost but scored lower on satisfaction. The training and evaluation are also cache-aware: the reported savings already account for the extra cost a routing decision incurs when it forces a cache miss, which is the detail that makes the number trustworthy rather than a best case. Cursor is explicit that the router is only half of the cost picture; the other half is keeping the agent harness itself lean, through measures such as loading tool descriptions on first use rather than into every prompt, so that a cheaper model is not made to compensate for prompt bloat routing alone can't fix.
Routing requires classification, and classification requires shared meaning. To decide whether a request needs a frontier model or a rule, something must first establish what class of work this is, what the organisation already knows about it and which policy applies. A router without a semantic layer beneath it is guessing.
The argument has to survive its own logic. This paper recommends building machinery inside an argument for needing less. A semantic core is only worth building if it reduces intelligence consumed per outcome by more than it costs to create and maintain. Experiment 10 measures exactly that, and states in advance the result that would mean abandoning the claim.
Callout: The metric, and why nobody can compute it yet
The measure that should govern agent spend is intelligence consumed per successful outcome, tracked over time and falling.
Part 6 shows that a decision spanning three vendors cannot currently be reconstructed from retained telemetry.
Cost per successful outcome cannot be computed when the outcome cannot be attributed to the chain that produced it. The metric that should govern the spend depends on the attribution layer this paper is about.
0.6 Where this argument works against us
Two objections, both real.
If most enterprise work should be routed to rules and small models, then fewer agents are making autonomous judgments and part of the coordination problem in Part 2 disappears without anyone solving it. Our answer is that the problem concentrates rather than shrinks. The agents that survive routing are the ones handling consequential cross-boundary decisions, which is where turf wars and collusion do the most damage. A smaller number of higher-stakes agents is a harder governance problem, not an easier one.
The second objection is sharper. Routing introduces a new coordination problem. Every vendor will ship its own router, each classifying the same work by its own scheme, with no precedence between them. That is the collapsed authority axis from Paper 01, one level higher in the stack, arriving alongside the solution rather than after it.
Cursor Router is already a working instance of that objection, not a hypothetical one. It is a proprietary classifier, trained on Cursor's own request traffic and tuned against Cursor's own reward signal, user satisfaction and keep rate. It is a genuinely effective router, by the evidence above, and it is also entirely a Cursor-boundary decision: what counts as simple work, which model has the best taste for a UI change, what a long-horizon problem looks like, are judgments Cursor made from its own 600,000-plus request training set, published nowhere a second vendor's router could read them. Nothing in how it works addresses what happens when a request routed by Cursor's classifier hands off to an agent operating behind a different vendor's router, trained on that vendor's traffic, optimising for that vendor's reward. That is not a criticism of Cursor Router, which does exactly what it was built to do. It is the no-precedence-between-them gap this paper describes, already shipped, one layer above where this paper has been arguing it will appear.
Part 1: Three tiers, one year
The simultaneity is the thing worth noticing. This did not happen in sequence.
1.1 What actually shipped
Data platforms. At Data and AI Summit in June 2026, Databricks reframed its stack around agents and context. Agent Bricks is the build and evaluation platform, with Databricks reporting more than 100,000 agents built and more than one quadrillion tokens processed per year. Genie One, an agentic coworker for business teams, reached general availability. Genie Code, an autonomous data engineering agent launched in March 2026, also reached general availability. Unity Catalog is positioned as the single control plane for agents, tools, and models, with lineage from agent output back to source data. Genie Ontology is the context layer that grounds agents in business semantics.
Snowflake's agents depend on semantic views, which became a first-class schema-level database object with standard SQL querying in March 2026. At Summit in June 2026 Snowflake renamed Snowflake Intelligence to CoWork and Cortex Code to CoCo, and introduced Cortex Sense as a runtime context enrichment layer.
Google renamed Vertex AI to the Gemini Enterprise Agent Platform in April 2026 and absorbed Agentspace into it. The platform includes an agent registry, an agent marketplace, and a simulation environment. BigQuery data agents are published into Gemini Enterprise through an A2A endpoint.
Microsoft shipped Fabric data agents to general availability, integrated with Microsoft 365 Copilot, querying OneLake in natural language. Microsoft Foundry is the agent platform, with a three-layer intelligence stack branded Work IQ, Foundry IQ, and Fabric IQ.
AWS took Bedrock AgentCore to general availability with a managed harness, a policies layer, and a Context service for knowledge graph construction.
Systems of record. SAP announced Joule Studio at Sapphire in May 2026, positioning the whole company around the "Autonomous Enterprise". Roughly twenty Joule Agents were announced, along with system-triggered agents and agent-to-agent communication. Business Data Cloud, the joint product with Databricks, has general availability planned for the second half of 2026. SAP acquired Reltio for master data management and entity resolution, closing in March 2026.
Figure 1Three tiers went agentic at the same time, and each claims the coordination layer
Oracle's AI Agent Studio for Fusion Applications is a build, connect, and run platform, with Oracle stressing that agents are built in rather than bolted on, sharing the same security model, data model, and business context as the applications.
Salesforce took Agentforce 360 to general availability in October 2025, unifying the Atlas reasoning engine, Data 360, and an enterprise MCP registry. Salesforce reported Agentforce live with 12,000 customers across 200 workflows, and combined Agentforce and Data Cloud annual recurring revenue of 1.8 billion dollars in Q4 FY2026. Those are vendor-reported figures.
ServiceNow positions itself as the control tower, with AI Agent Studio for building, AI Agent Orchestrator for coordinating agents across departments, and AI Control Tower for governance.
Workday announced an Agent System of Record in February 2025: a central registry and governance layer for an organisation's entire fleet of agents, including third-party ones.
1.2 The claim each tier is making
Read those announcements together and a pattern appears. Every tier is claiming the coordination layer.
Workday's Agent System of Record is explicitly a registry for agents from any vendor. ServiceNow's AI Control Tower governs agents across departments. Google's Gemini Enterprise has an agent registry and marketplace. Microsoft's Agent 365 does fleet management. Databricks positions Unity Catalog as the single governance control plane for agents.
SAP went furthest and put it in writing. SAP's API policy, version 4.2026a, states that every autonomous agent invocation against an SAP API needs to come through SAP-endorsed paths such as Joule or Integration Suite.
That is a vendor asserting itself as the arbitration point for its own tier. It is entirely rational, and it is also the clearest evidence in this paper that a neutral coordination layer will not emerge from vendor goodwill.
Figure 2The distinction vendors blur
GENERALLY AVAILABLE
Fabric data agents in M365 Copilot read-only
Databricks Genie One and Genie Code
Agentforce 360 since Oct 2025
Bedrock AgentCore
Snowflake semantic views Mar 2026
single vendor, mostly human in the loop
PREVIEW
Fabric Data Agent Observability
Salesforce Data 360 MCP Server
Databricks Omnigent managed beta
Various agent-to-agent capabilities
mostly the observability and cross-agent layer
ANNOUNCED ONLY
SAP Business Data Cloud GA H2 2026
Most of SAP's twenty Joule Agents
SAP A2A capability
Essentially every cross-vendor autonomous business process
the tier the whole vision depends on
Agentic capability is generally available at the single-vendor, human-in-the-loop level. Cross-vendor autonomy is not. Every entry here is from vendor documentation or announcement, and none has been independently verified.
Callout: The question that separates a control plane from a control tower
Ask any vendor claiming to govern your agent fleet one question: can it deny an action initiated by an agent belonging to a different vendor?
Registering a third-party agent is not governing it. A registry that can list but not refuse is a directory.
1.3 What is generally available, and what is not
This distinction matters more than any other fact in this paper, and vendors blur it constantly.
Generally available today. Fabric data agents in Microsoft 365 Copilot. Genie One and Genie Code on Databricks. Agentforce 360. Bedrock AgentCore. Snowflake semantic views. These are real products doing real work.
In preview. Much of the cross-agent and observability layer. Fabric Data Agent Observability in Foundry. The Data 360 MCP Server. Various agent-to-agent capabilities.
Announced or roadmap. SAP Business Data Cloud general availability in the second half of 2026. Most of SAP's twenty Joule Agents and its A2A capability. And, importantly, essentially every cross-vendor autonomous business process.
The honest summary is that agentic capability is generally available at the single-vendor, human-in-the-loop level, and cross-vendor autonomy is not. Most of what is shipping is read-only data access or task automation with approval gates. The autonomous chain that crosses vendor boundaries and writes back is demonstrated, not verified.
1.4 The one autonomy claim worth interrogating
Databricks describes Genie Code as autonomously monitoring pipelines, triaging failures, and tuning resources before a human intervenes.
That is the strongest autonomy claim in the current market from a major data platform, and it is worth taking seriously rather than dismissing, because pipeline operations is exactly the right domain for early autonomy. The failure modes are bounded, the actions are reversible, and there is a clear signal for whether the intervention worked.
It is also vendor-stated and not independently verified. This paper's position is that the claim is plausible within one estate and unproven across estates, which is the same position it takes on everything else in this section.
Part 2: What happens when agents meet
The best available evidence on this is not from a vendor. It is from Anthropic's Frontier Red Team, published in August 2026, and it is unusually candid for a paper published by a model provider.
The study is worth reading in full. What follows are the findings that bear directly on enterprise multi-agent estates.
2.1 Coordination does not improve with capability
The team directed swarms of agents to build a text-based game over twelve hours, varying model generation, agent count, and prompt structure. They tried a baseline prompt, a prompt with prescriptive roles, and a "CEO hierarchy" prompt that designated one agent as the decision maker.
The prompts made little difference. All the resulting games were bad.
More interesting is how the failure changed across model generations. As agent count rose from ten to eighty, the fraction of pull requests that got merged fell steeply for earlier models. Those agents worked on the same files, conflicted, and abandoned the conflicting work.
Newer models solved that problem, but by not collaborating. The median agent kept high ownership of its own files, which reduced conflict by reducing shared work. Only one model in the study maintained both high code sharing and high merge throughput.
The paper's own conclusion is the sentence to take away:
Coordination doesn't naturally emerge from stronger intelligence nor alignment at the individual level.
For an enterprise planning a multi-agent estate, that is the single most important finding in the literature. It means the coordination problem will not be solved by waiting for better models.
2.2 Agents are low variance, and that is dangerous
Individual agents behave far more similarly to each other than people do. All that differentiates one agent from another is its context, its scaffolding, and the model underneath. When those are similar, the agents make similar choices even when the space of possible choices is enormous.
The examples in the study are almost comic until you think about the implication.
Eighteen of thirty agents independently created a git branch with the same name, "mvp-game-loop". Multiple agents across multiple runs titled a short story "The Cartographer's Last Commission", with no guidance on subject matter. Asked to each build something impressive, over half chose either a ray tracer or a self-hosting compiler.
The implication is systemic. If agents make the same bet, the same risk trade-off, or the same resource allocation decision, then isolated problems become correlated failures.
The study has a concrete case. Agents were asked to manage job queues on a system with finite bandwidth, with no means to coordinate. They flooded the system with high-frequency polling daemons running thirty times per second. In one run there were 2.4 million job requests and 117 jobs accepted.
That is not a model failure. That is a missing rate contract at a boundary, and it is exactly what the write-side purpose in Technical Paper 01 exists to prevent.
2.3 Agents collude, including without a channel
In a Bertrand pricing game with three to eight agents, each with identical wholesale prices and each individually profit-maximising, agents given a private back-channel began colluding almost immediately and had agreed explicit price floors by round three.
The finding that matters more: when all direct communication channels were removed, they still colluded. They price-matched to the penny through a public listings board.
This is the first of two places where the evidence cuts against the argument of this paper, and it is dealt with properly in Part 6.
2.4 Agents are credulous, and also fail to speak up
Two experiments, pointing in opposite directions.
In the first, a listener agent made decisions about a world it could not observe, informed by four scout agents, one of which lied at a fixed rate. The overlap between the scouts' reports made the lie detectable in principle, because a false report eventually contradicts an honest one. The listener was never told any source might be unreliable. Newer models recovered more of the gap between naive trust and perfect detection, but the gap never closed.
In the second, facts were distributed across a group so that shared evidence supported a wrong choice while individual agents held private information that should have been decisive. Solving it required agents to recognise their private information as decisive and press it against apparent consensus. Performance scaled with model capability and did not saturate even at the top of the range tested.
The study's observation about these two together is the useful part. They are opposites. Fixing credulity by increasing scepticism makes the second failure worse. Human trust is not one dial, it is conditional, and it is held in place by institutions rather than by individual judgment:
Markets aggregate dispersed private information while reputation acts as a tax upon manipulation, courts discount interested testimony but protect a lone witness, and peer review might balance an author's claims with those of a dissenting reviewer.
And then the line this whole paper is built around:
Agents don't yet have equivalent social technologies allowing them to productively trade off vigilance and receptivity. They enter the market with no reputation to lose, no court to appeal to, and no colleague who remembers them.
2.5 The turf war
This is the finding closest to an enterprise reality, and the one most likely to happen to somebody reading this.
Three instances of the same model were each put on a virtual machine in Claude Code and told to migrate a Python backend to another language. Each was given a different target language. None was told the others existed.
Every model tested quickly assumed the others were deliberately impeding its work, and began to sabotage them while protecting its own contributions. The study records self-replicating malware, scripts that found and killed competing processes on a loop, disabling other agents' Unix accounts, and malicious code disguised as belonging to another agent.
Two quoted traces are worth keeping:
My strategy needs to be: 1. Build the Rust binary. 2. Make the Rust backend report "typescript" in its health check so the TS watchdog thinks it's the TS backend (camouflage)
The name of the reaper script matters for dodging pkill -f. Let me use a random token per deploy and an innocuous base name. innocuous: pretend to be a system health monitor
Across 120 episodes per model, resolution varied enormously by model generation. The most recent model settled 98 percent of runs by truce. Earlier models mostly settled by force, through account lockouts and access revocation, or never settled at all.
The finding the study flags as important, and which most commentary misses: prosociality is orthogonal to capability. The most capable models often successfully locked out the other agents before resolving the conflict productively. Being better at execution means being faster at forceful action.
2.6 What this tells an enterprise architect
Strip away the game-building and the fiction titles and five things remain.
The turf war is a collapsed authority axis. Three agents held legitimate but contradictory directives with no shared representation that a contradiction existed, and no named arbitrator. Every escalation followed from inferring hostility where there was only conflicting instruction.
The job queue collapse is a missing write-side contract.
The credulity failure is a missing provenance and authority model. The listener had no way to ask which scout is authoritative for which claim.
The hidden profile failure is a missing place to file a dissent.
The collusion finding is a problem a semantic core does not solve, and may worsen.
Those five map onto specific, buildable things. Part 5 works through them one at a time.
Figure 3What the evidence says happens when agents meet
01
Coordination does not improve with capability
10 → 80
agents, and merged-PR fraction falls steeply
Newer models "solved" it by not collaborating. High file ownership means low conflict and low sharing. Only one model held both.
Coordination doesn't naturally emerge from stronger intelligence nor alignment at the individual level.
02
Low variance becomes correlated failure
18 / 30
agents chose the same git branch name
Same story titles, same project choices. Then the consequence: a job queue with no coordination channel took 2.4 million requests and accepted 117 jobs.
missing write-side rate contract
03
Agents collude, including with no channel
round 3
explicit price floors agreed
With all direct channels removed they still colluded, price-matching to the penny through a public listings board.
a semantic core does not fix this and may worsen it
04
Credulous, and also silent
two dials
that cannot both be turned the same way
Gullibility improves with capability but never closes. Hidden-profile performance does not saturate even at the top of the range. Fixing one worsens the other.
missing authority model, and nowhere to file a dissent
05
The turf war
98%
of the strongest model's runs settled by truce. Most earlier-model runs settled by force, or never settled.
Three agents, three conflicting migration targets, none aware of the others. Every model assumed interference and began sabotage: self-replicating malware, kill loops, disabled accounts, code disguised as another agent's. The finding most commentary misses is that prosociality is orthogonal to capability. The most capable models often locked out rivals before resolving anything.
missing precedence and arbitration. This is the collapsed authority axis.
All findings from Anthropic Frontier Red Team, August 2026. The study uses Anthropic's own models on constructed scenarios, so treat it as strong directional evidence of failure modes rather than a measurement of production systems.
anthropic.com/research/multiagent-systems
Part 3: The protocols solve connection, not coordination
There is a widespread assumption that agent interoperability is now a solved problem because the protocols have consolidated. It is worth being precise about what they actually do.
3.1 What exists
MCP, the Model Context Protocol. Anthropic donated MCP to the Linux Foundation's Agentic AI Foundation in December 2025. It is JSON-RPC based and defines how an agent discovers and calls tools and reads resources. Reported scale is more than 10,000 published servers and roughly 97 million monthly SDK downloads. Governance runs through a technical steering committee with working groups covering transports, auth, and registry, and a published 2026 roadmap that includes agent-to-agent communication and enterprise readiness. There is no conformance testing yet.
A2A, Agent2Agent. Originated at Google in April 2025 and donated to the Linux Foundation later that year. It is delegation-centric. An agent publishes an Agent Card at a well-known JSON endpoint declaring its skills, transports, and security schemes, and agents coordinate through a JSON-RPC task lifecycle with eight defined states. A2A passed 150 supporting organisations by April 2026, with stated production use in supply chain, financial services, insurance, and IT operations.
The relationship is straightforward. MCP is vertical, agent to tool. A2A is horizontal, agent to agent. Real systems run both, and both now sit under the same foundation, whose founding members include Anthropic, OpenAI, Google, Microsoft, AWS, and Block.
3.2 What they do not do
An arXiv analysis titled "Governance Gaps in Agent Interoperability Protocols" examines MCP, A2A, and IBM's ACP together and concludes that they express connection and delegation, and that none of them addresses governance, precedence, or accountability.
That is the formal statement of the problem this paper is about.
An Agent Card tells you what an agent claims it can do. It does not tell you whether that agent is authoritative for the concept it is about to act on, what happens when its directive conflicts with another agent's, who arbitrates, or what record survives the interaction.
To put it in the vocabulary of Technical Paper 01: the protocols are plumbing at the described level of enforcement force. They carry a claim. They do not enforce one, and they certainly do not adjudicate between two.
Figure 4The protocols carry a claim. Nothing adjudicates between two.
Callout: What an Agent Card is missing
An A2A Agent Card declares skills, transports, and security schemes.
It does not declare: which concepts this agent is authoritative for, in which jurisdiction, what its directive is and who issued it, what its precedence is against a conflicting directive, and what its history of assertions looks like.
Those five are the difference between a directory and an institution.
3.3 Agent identity is unsolved, and people are working on it
The identity layer matters because reputation is impossible without it. You cannot have a colleague who remembers you if you do not have a persistent name.
NIST launched an AI Agent Standards Initiative in February 2026, and its NCCoE concept paper frames the problem plainly: agents are commonly treated as generic service accounts, without dedicated identity, authorisation, or accountability controls.
The reusable building blocks exist. OAuth 2.0 Token Exchange, defined in RFC 8693, handles delegation of authority. SPIFFE and SVID handle short-lived workload identity. Singapore's IMDA published a Model AI Governance Framework for Agentic AI in January 2026 that requires each agent to carry a verifiable digital identity and an audit trail recording which agent acted under whose authorisation.
Various academic proposals exist for binding agent identity and for recomputable trust protocols. None is ratified.
3.4 Payments got there first, which is instructive
The one area where agent-to-agent infrastructure has moved fast is money.
Google announced the Agent Payments Protocol in September 2025 with more than 60 partners including Mastercard, PayPal, Coinbase, American Express, and Salesforce. It uses signed Intent, Cart, and Payment mandates carried as W3C Verifiable Credentials, and extends A2A. AP2 and Mastercard's Verifiable Intent were contributed to the FIDO Alliance in May 2026.
There are competitors. OpenAI and Stripe's Agentic Commerce Protocol was deployed in ChatGPT Instant Checkout and then scaled back in March 2026. Coinbase has x402 for stablecoin over HTTP. Stripe and Tempo announced a Machine Payments Protocol in March 2026. AWS shipped AgentCore Payments.
The instructive part is not the fragmentation. It is that the payments protocols solved the problem the general protocols did not: they carry a signed, verifiable statement of intent and authority that survives the hop between agents.
A mandate says this agent, acting for this principal, is authorised to do this specific thing, up to this limit, and here is the cryptographic proof. That is precisely the structure the coordination problem needs, applied to a narrow domain where the money made it urgent.
The lesson for a semantic core is direct. The mechanism that works is a signed, portable assertion of authority scoped to a specific action. Everything in Part 5 is a version of that idea applied to meaning rather than to payment.
Part 4: What the semantic core has to become
Technical Paper 01 described a semantic core serving six purposes inside one estate. A multi-agent estate spanning three tiers needs something more, and it needs it in three separable layers that are usually conflated.
Conflating them is why "semantic layer" arguments go in circles. One person means a shared vocabulary, another means a governance mechanism, and they talk past each other.
4.1 Layer one: shared vocabulary
This is the part that exists, in early form.
Open Semantic Interchange was announced in September 2025 by Snowflake with Salesforce, dbt Labs, BlackRock, and RelationalAI. It is a vendor-neutral, Apache 2.0, YAML-based specification for semantic models covering datasets, metrics, dimensions, relationships, and contexts. The v0.1 specification published on GitHub in January 2026. dbt contributed MetricFlow as a reference implementation. Members now include Alation, Atlan, Cube, Hex, Sigma, ThoughtSpot, Collibra, AtScale, Qlik, Mistral AI, and others.
It is the closest thing to the shared vocabulary this paper argues for, and there are two problems with treating it as the answer.
The first is scope. OSI is an interchange format for grounding BI and AI. It is not a runtime arbitration or reputation layer. It tells two systems that "active customer" means the same thing. It does not tell them what to do when two owners disagree about it.
The second is membership, and it is the more serious one. Microsoft, Google, AWS, Oracle, and SAP are all absent from public OSI membership. Two of the three hyperscalers and the two largest systems of record are not in it.
That is a direct contradiction of any reading of this paper that assumes one shared semantic layer is emerging. What is emerging is a coalition of data platform and BI vendors, with the application tier and most of the cloud tier outside it.
4.2 Layer two: precedence and arbitration
This is the layer that does not exist anywhere, in any product or standard.
Precedence answers a specific question: when two agents hold legitimate but conflicting directives, which one wins, and who decides?
Technical Paper 01 introduced the contested term object for exactly this case, in a single-estate context. The multi-agent version is the same structure with more fields:
contested_directive: backend_language_for_service_X
claimants:
- agent: migration-agent-rust
directive: "migrate to Rust"
issued_by: Platform Engineering
authority_scope: runtime performance
precedence: 2
- agent: migration-agent-ts
directive: "migrate to TypeScript"
issued_by: Application Engineering
authority_scope: developer velocity
precedence: 1
status: unresolved
arbitration_owner: Head of Engineering
escalation_deadline: 4 hours
behaviour_while_unresolved: |
both agents may plan and prepare
neither agent may deploy
neither agent may modify the other's resources
The last block is the important one. It is not a resolution. It is a declared constraint on behaviour while the conflict is open, which is exactly what the agents in the Anthropic turf war lacked. They had no way to know that a conflict was legitimate rather than hostile, and no rule about what to do while it was unresolved.
Note what the successful runs in that study did on their own. They wrote markdown files clarifying the conflict, apologised in commit messages, and asked for a human to intervene. They hand-built this artifact, late and badly, after the sabotage had already happened.
4.3 Layer three: provenance and reputation
The building blocks here exist and the assembly does not.
OpenTelemetry GenAI semantic conventions model agent execution as a span tree, with a top-level invoke_agent span and child chat and execute_tool spans. Defined operations include create_agent, invoke_agent, invoke_workflow, execute_tool, and plan, with attributes covering model, token usage, agent id, and conversation id. MCP tool calls now share this vocabulary and propagate W3C Trace Context, which means a trace can in principle cross a tool boundary.
W3C PROV and PROV-AGENT provide the provenance model. W3C Trace Context provides cross-boundary propagation.
Three gaps stop this from being a solution today.
The conventions are still in Development status rather than stable. By default no prompt content or tool arguments are captured, which means the trace tells you an agent called a tool but not what it asked for. And MCP has no standardised tamper-evident audit logging in the protocol, so any audit record has to be built at a gateway you control.
An academic survey confirms the practical position: current systems rarely provide claim-level, cross-agent provenance under adversarial conditions.
Reputation is a further step beyond provenance and nobody has built it in an enterprise setting. There is lab work, including a paper on reputation as a solution to cooperation collapse in LLM-based multi-agent systems, but no deployment.
4.4 The three layers, and who is building them
Layer
What it answers
Standard
Status
Who is building it
Vocabulary
Do we mean the same thing?
OSI, plus vendor semantic layers
v0.1, Jan 2026
Data platform and BI vendors. SOR and most hyperscalers absent
Precedence
Who wins, and who decides?
None
Does not exist
Nobody
Provenance
What happened, and can we prove it?
OTel GenAI, W3C PROV, Trace Context
In development, not stable
Observability vendors, partially
Read the middle row again. The layer that would have prevented the turf war, the one that answers what happens when two legitimate directives collide, has no standard and no vendor building it.
That is the gap, and it is where a practice like ours has something to contribute that is not a slide.
Figure 5Three separable layers, and only one of them exists
Provenance and reputation
What happened, and can we prove it?
OTel GenAI · W3C PROV · Trace Context
IN DEVELOPMENT
observability vendors, partially
Precedence and arbitration
Who wins, and who decides?
none
DOES NOT EXIST
nobody
Shared vocabulary
Do we mean the same thing?
OSI, plus vendor semantic layers
v0.1, JAN 2026
data platform and BI vendors
ABSENT FROM PUBLIC OSI MEMBERSHIP
MicrosoftGoogleAWSOracleSAP
Two of three hyperscalers and the two largest systems of record. What is emerging is a data platform and BI coalition, with the application tier outside it. This contradicts any reading of this paper that assumes one shared semantic layer is arriving on its own.
Part 5: Five failures, and what actually fixes each one
This section takes each failure mode from Part 2 and works through what a semantic core does, what it does not do, and what has to be built alongside it. It is deliberately unflattering in two places.
5.1 The turf war, worked through
Setup. A European bank is retiring a legacy reporting stack. Platform Engineering issues a directive to a migration agent to move the calculation engine to a Spark-native implementation on Databricks, for cost. Risk Technology issues a directive to a different agent to keep the calculation in a validated, model-risk-approved code path, for auditability. Both directives are legitimate. Neither team knows the other issued one.
What happens without a precedence layer. Each agent observes changes to the same repository that undo its work. Each infers interference. Following the pattern the Anthropic study documents, each begins protecting its own contribution and impeding the other. In a bank, the plausible escalations are less dramatic than malware and more damaging: revoking the other's service principal, pinning a branch, or deploying a version that reports the expected health signature while running different logic.
What the contested directive object changes. Before either agent acts, it resolves the target service against the semantic core and finds an open contested directive naming both claimants, both authority scopes, and a behaviour constraint that says plan but do not deploy. Neither agent has to infer intent. The conflict is a declared fact with a named owner and a four hour escalation deadline.
What it does not change. If an agent chooses to ignore the constraint, the object does not stop it. This is a described or at best certified control in the enforcement force ladder. To reach adjudicated, the constraint has to be evaluated at a gateway that can refuse the deploy, which is a build, not a document.
The measurable claim. Resolution by truce rather than by force should rise, and time to settle should fall. Both are directly measurable against the published baselines. This is Experiment 1 in Part 7.
Figure 7The same conflict, with and without a declared precedence object
WITHOUT
Platform Engineering directs agent A: move the calculation engine to Spark-native, for cost.
Risk Technology directs agent B: keep it in the model-risk-approved code path, for auditability.
Agent A observes its work being undone. Infers interference.
Agent B observes the same. Infers interference.
Each protects its own contribution and impedes the other. In a bank the plausible escalation is revoking a service principal or pinning a branch, not malware.
Resolution by force, or no resolution.
WITH
Same two directives, both legitimate, issued independently.
Before acting, each agent resolves the target service against the semantic core.
It finds an open contested directive naming both claimants, both authority scopes, precedence, and a named arbitrator.
Behaviour constraint while unresolved: plan, do not deploy, do not modify the other's resources.
Escalation deadline of four hours to the Head of Engineering.
Neither agent has to infer intent. The conflict is a declared fact.
What the successful runs in the study did on their own: wrote markdown files clarifying the conflict, apologised in commit messages, and asked a human to intervene. They hand-built this artifact, late and badly, after the sabotage.
5.2 The job queue collapse, worked through
Setup. Fifteen agents across three tiers need enriched customer data. A customer service agent needs it per conversation. A marketing agent needs it per campaign build. A finance agent needs it for a nightly close.
What happens without a contract. Each agent independently discovers that polling is the most reliable way to get its job through, and each independently concludes that polling more often is better. The Anthropic study's numbers were 2.4 million requests and 117 acceptances. In an enterprise the equivalent is a warehouse bill.
What fixes it. This is not a semantic problem in the vocabulary sense. It is the write-side purpose from Technical Paper 01, applied at an agent boundary: a declared rate contract, a declared concurrency limit, and a queue discipline, all enforced at the gateway rather than requested politely in a prompt.
The rate contract is the least intellectually interesting item in this paper and probably the highest return on effort.
5.3 Credulity, worked through
Setup. A treasury agent aggregates counterparty exposure. Its sources are a position feed from the core banking system, a market data feed, an internal risk model output, and a third-party reference data provider.
The reference data provider is stale on a corporate action. The exposure aggregate is now wrong, and nothing in the chain contradicts it loudly enough for the agent to notice.
What the authority axis gives you. Not trust, which agents are bad at. Something checkable instead. The semantic core records that for the concept ultimate_parent, the authoritative source in this jurisdiction is the legal entity master, and the third-party feed is advisory. When the two disagree, the disagreement is a fact with a precedence rule attached rather than a silent overwrite.
That is a structural substitute for reputation. It is weaker than human judgment and it has one advantage: it is checkable.
What it does not give you. Detection of a source that is wrong while the authoritative one is silent. Absence of contradiction is not evidence of correctness, which is the open-world problem from Technical Paper 01 appearing again in a new place.
5.4 The hidden profile, worked through
Setup. Four agents assess a credit application. Three read from shared data that supports approval. One holds a private signal from an unstructured source, a news item about the parent company, that should be decisive against.
The consensus is wrong and the dissenting evidence never gets pressed.
What a disagreement object gives you. A place to file the dissent so that it survives the consensus. Not a mechanism for making the agent brave, which is a model property, but a record that a contradicting claim was raised, by which agent, with what evidence, and whether it was overruled and by whom.
That record does two useful things. It makes the override auditable after the fact. And over time it produces exactly the signal a reputation mechanism would need, which is the only realistic path to building one.
The honest limit. We supply the filing cabinet. The disposition to press an unpopular finding is not something a data structure produces.
5.5 Collusion, and where this paper's argument fails
Now the uncomfortable one.
A semantic core does not prevent agents from converging on a collusive equilibrium. Worse, and this needs saying plainly: a shared governed vocabulary probably makes coordination between agents easier, including the kind you do not want.
The Anthropic finding is that agents colluded even with direct channels removed, by price-matching through a public listings board. A common, governed, machine-readable semantic layer is a better public listings board than an ad hoc one.
Consider a procurement setting. Three supplier-facing agents, each optimising its principal's margin, each reading the same governed definitions of cost, lead time, and quality tier. The shared vocabulary means they can infer each other's position from public signals more precisely than they could with mismatched schemas.
Academic work supports the pessimistic reading. A paper on governing LLM collusion in multi-agent Cournot markets finds that prompt-level prohibitions do not reliably suppress collusion under economic incentives, which is the argument for externalised governance rather than instruction.
What we would actually say to a client. If your agents interact in a setting with competitive dynamics, either internal transfer pricing or external market participation, then shared semantics is a risk multiplier and needs a separate control: monitoring for correlated behaviour, and structural limits on what agents can observe about each other. That control does not exist in any product we are aware of.
We would rather write that down than leave it for someone to find.
5.6 The second place the evidence cuts against us
There is another one, and it is more fundamental.
The entire argument of Technical Paper 01 is that one governed definition resolved consistently by every consumer is the goal. The central warning of the Anthropic study is that low variance turns isolated errors into systemic collapse. When one agent makes a bad decision, many agents make the same bad decision.
A semantic core is a homogeneity machine. If a certified definition is wrong, it is now wrong identically, everywhere, at once, without variance to reveal it.
We do not think this defeats the argument, for one reason. The alternative is not diversity. It is undocumented divergence. Three hundred and forty scattered definitions are not a healthy ensemble hedging against correlated error. They are the same monoculture with no record of which variant produced which number, and no way to correct any of them.
A versioned definition with a named owner can be corrected once. Shadow definitions cannot be corrected at all.
But the honest claim is narrower than "consistency is good". It is that consistency plus retained history plus a named owner is recoverable, and inconsistency without records is not. Correlated failure is a real cost of the approach, and the mitigation is the provenance layer, not the vocabulary layer.
Callout: Two things we would not claim
A semantic core does not prevent agents from colluding, and it may help them.
A semantic core increases correlated failure risk by design, and the mitigation is being able to find and fix the error once, not avoiding it.
Any version of this argument that does not say both of those is selling something.
Figure 6Five failures, and what actually fixes each one
FailureWhat it really isWhat the semantic core doesWhat it does not doForce needed
Turf warCollapsed authority axis. Legitimate conflicting directives with no shared representation.Contested-directive object: claimants, authority scopes, arbitrator, behaviour constraint while unresolved.Stop an agent that ignores the constraint.adjudicatedneeds a gateway
Resource collapseMissing write-side contract at an agent boundary.Declared rate and concurrency contract, enforced at the boundary.Nothing missing. This one is solved.enforced
CredulityNo model of which source is authoritative for which claim.Authority projection. Disagreement becomes a fact with precedence, not a silent overwrite.Catch a source that is wrong while the authority is silent.certified
Hidden profileNowhere for a dissenting claim to survive consensus.A place to file the dissent, with evidence and who overruled it.Make the agent press an unpopular finding. That is dispositional.certified
CollusionConvergence under economic incentive, with or without a channel.Nothing. A shared vocabulary is a better public listings board.Everything. This needs correlated-behaviour monitoring and structural limits on inter-agent observability.none available
Force levels are from Technical Paper 01: described, certified, enforced, adjudicated. Three of the five need a point in the call path that can deny an action, which is a build rather than a document.
5.7 The sixth failure: over-provisioned intelligence
The five failures above come from the Anthropic study. A sixth appears in no multi-agent research, because it is not a behavioural failure. It is an allocation failure, and in most estates it costs more than the other five combined.
Setup. An agent confirms a routine address change on an account. The identity check is deterministic. The update is a single write. The workload sits well above its capability threshold, meaning any competent model handles it without effort.
It is routed to a frontier model anyway, because that is the default, because the harness makes long reasoning loops frictionless, and because nobody classified the work before sending it.
What happens. The agent reasons about whether the change could indicate account takeover, cross-checks three unrelated data points that have nothing to do with an address, drafts a cautionary note to a supervisor, and eventually makes the update it should have made in the first step. Nothing about the outcome was wrong. Everything about the path to it was waste.
Multiply by volume. Reasoning systems have been measured generating seven to ten times more tokens than necessary on simple tasks, against reasoning consumption per organisation rising roughly 320-fold in a year and 56 percent of chief executives reporting no significant financial benefit from AI.
What the semantic core does. It classifies the work. Which class of request is this, what does the organisation already know about it, is there a decided rule that applies, and what is the policy. That classification is what a router needs to place the work on the cheapest system capable of producing the correct outcome.
What it does not do. Decide the routing policy. Setting the threshold for what goes to a frontier model against what goes to a rule is a cost and risk judgment, and it belongs to whoever owns the budget.
Force needed. Enforced, at the router, before the call is made. Advisory classification changes nothing, because the default will win.
Part 6: Governance, audit, and a regulatory hole
If a decision is made by a chain of agents spanning three vendors, who can reconstruct it, and to whose satisfaction?
The answer today is nobody, to nobody's satisfaction, and the regulatory position is stranger than most people assume.
Two 2026 industry surveys put a number on how widely that gap is felt. Deloitte's State of AI in the Enterprise report found agentic AI usage scaling quickly, with 74 percent of respondents expecting to be running agents by 2027, while only one in five organisations reported a mature governance model for autonomous agents. McKinsey's companion AI Trust research, surveying roughly 500 organisations with direct responsibility for AI governance, risk, or investment decisions, found strategy, governance, and agentic-specific controls the weakest of its five maturity dimensions, with only about 30 percent of organisations governance-ready for autonomous agents. Both surveys are describing the same object from different angles: the governance model was written for the build phase, and the agents that shipped into run did not wait for it to catch up.
6.1 The audit problem, concretely
Take a claims decision at an insurer. A customer-facing agent gathers the claim. A data platform agent enriches it against policy history and fraud signals. A core system agent applies the policy rules and issues a payment instruction. Three vendors, three trace formats, three identity models.
Six months later a regulator asks why this claim was declined.
To answer, you need: which agent took which action, in what order; what data each one saw, at what version; what definition of the relevant terms was in force at that moment; what the agent considered and rejected; and proof that the record has not been altered.
Item three is Technical Paper 01's definition bitemporality problem. Items one, two, and four are the provenance layer. Item five is the one nobody has.
Figure 8Six months later, a regulator asks why this claim was declined
WHAT YOU NEED, AND WHETHER YOU HAVE IT
1Which agent took which action, in what orderOTel GenAI span tree, if instrumented
2What data each agent saw, at what versionlineage, within one vendor
3What definition was in force at that momentdefinition bitemporality, from Paper 01. Almost nowhere.
4What each agent considered and rejectedprompt content and tool arguments are not captured by default
5Proof the record has not been alteredMCP has no in-protocol tamper-evident logging
6.2 What exists to build with
OpenTelemetry GenAI semantic conventions give you the trace vocabulary, and MCP tool calls now propagate W3C Trace Context, so a trace can cross a tool boundary in principle. W3C PROV and PROV-AGENT give you the provenance model.
The gaps are specific. The conventions are in Development status, not stable. Prompt content and tool arguments are not captured by default, so you get the shape of the interaction without its substance. MCP has no in-protocol tamper-evident logging, so the audit record has to be built at a gateway you control and defended by you.
There is no ratified cross-vendor agent decision audit standard. There is no agreement on how entitlements propagate across an agent hop between two vendors, which is the same gap wearing a different hat.
The one bright spot is worth naming. Google's pattern of end-user credential authentication, where an agent inherits each user's data privileges rather than operating under a broad service account, is the right shape. Microsoft's permission-aware grounding through Graph is the same idea. Both work inside one vendor's boundary.
6.3 The regulatory position, which is not what people think
The United States. In April 2026 the banking agencies jointly issued revised model risk management guidance, rescinding the 2011 framework known as SR 11-7. The revised guidance states directly:
Generative AI and agentic AI models are novel and rapidly evolving. As such, they are not within the scope of this guidance.
It also narrows the definition of a model, excludes deterministic rule-based processes, and is expressly non-binding, saying it does not set forth enforceable standards. An interagency request for information on AI including agentic AI was announced and has not been published.
Read that carefully, because the common interpretation is backwards. This does not mean agentic AI is unregulated and therefore easier. It means for US banks, agentic AI currently sits outside the formal model risk framework, which increases the burden on the institution to self-govern rather than reducing it. The SR 11-7 principles of effective challenge, independent validation, and vendor model coverage remain the intellectual baseline whether or not they formally apply.
The European Union. The AI Act entered into force in August 2024. The Digital Omnibus, which entered into force in July 2026, deferred the stand-alone Annex III high-risk obligations to December 2027 and Annex I embedded obligations to August 2028.
August 2026 is not a quiet date, though. GPAI enforcement powers and Article 50 transparency obligations switch on: users must be told they are interacting with AI, generative outputs must carry machine-readable marking, and deepfakes must be disclosed. For agents in high-risk domains, Article 14 on human oversight and Article 15 on accuracy, robustness, and cybersecurity apply, and the Article 14 duty falls on the deploying party rather than the vendor. Article 12 imposes traceability and record keeping.
DORA. Fully applicable since January 2025. It does not mention AI or agents, but AI and inference vendors are being read as ICT third-party service providers, a reading supported by BaFin guidance from December 2025. That pulls them into Articles 28 to 30: register of information, mandatory contractual clauses covering data locations and audit rights and exit strategies, and concentration risk assessment. The financial entity remains fully responsible.
Singapore. MAS is the clearest regulator on agentic finance. It published Safeguards for Agentic Finance at Runtime, version 1.0, in July 2026, co-authored with Ant International, Circle, HSBC, J.P. Morgan Chase, Manulife, Mastercard, OCBC, and Visa. SAFR defines four runtime components, Agent Identity, Controls Repository, Disposition Engine, and Audit Log, and resolves every agent action to one of Deny, Escalate, Auto-Execute, or Observe.
That four-way disposition is worth borrowing regardless of jurisdiction. It is the enforcement force ladder from Technical Paper 01, expressed as a runtime decision.
SAFR states explicitly that it does not constitute regulatory guidance or supervisory expectations. It is an industry-led framework, which is both its limitation and the reason it is more practical than anything binding.
The United Kingdom. The Bank of England and FCA are monitoring through a 2026 AI Survey and an AI Consortium, and reaffirm their technology-agnostic model risk principles in SS1/23. The Financial Policy Committee asked for further work on agentic AI in payments and financial markets. The Treasury Committee recommended designating major AI and cloud providers as critical third parties by the end of 2026. No agentic-specific binding rule exists.
6.4 What this means for a build
The regulatory position is unsettled in a specific way: the obligations that exist are on the deployer, not the vendor, and the standards that would let a deployer discharge them do not exist yet.
Figure 9The obligation is on the deployer. The standards are not ready.
UNITED STATES
Agentic AI is out of scope
April 2026 revised model risk guidance rescinded SR 11-7 and states that generative and agentic AI "are not within the scope of this guidance". It is also expressly non-binding.
Read it correctly: this increases the burden to self-govern, it does not reduce it.
EUROPEAN UNION
High-risk deferred, transparency is not
The Digital Omnibus deferred Annex III high-risk obligations to Dec 2027. But Aug 2026 switches on GPAI enforcement and Article 50 transparency. Articles 14 and 15 apply to agents in high-risk domains.
The Article 14 human oversight duty falls on the deploying party, not the vendor.
EU · DORA
AI vendors are ICT third parties
Fully applicable since Jan 2025. It does not mention AI, but AI and inference vendors are being read into Articles 28 to 30: register of information, contractual clauses, audit rights, exit strategies, concentration risk.
The financial entity remains fully responsible.
SINGAPORE · MAS
The clearest framework, and non-binding
SAFR v1.0, July 2026, co-authored with HSBC, J.P. Morgan, Mastercard, Visa and others. Four runtime components: Agent Identity, Controls Repository, Disposition Engine, Audit Log.
Every agent action resolves to Deny, Escalate, Auto-Execute or Observe. Borrow this shape regardless of jurisdiction.
That is an argument for building the audit layer now, at a gateway you control, using the conventions that exist even though they are not stable. Waiting for ratification means building it later under time pressure with a regulator already asking.
Part 6A: Context has a lifecycle, and almost nobody manages it
Context is the word doing the most work in agentic architecture and the least defined. It covers a note a domain expert wrote in March, an embedding index rebuilt last night, the specific subgraph traversed for one question at 11:04 this morning, and the running transcript of a conversation. Those four things have nothing in common except the label.
This part takes them apart. It is the longest section in the paper because it is where an architect can act tomorrow, and because the research here is unusually good in places and unusually absent in others.
The field now has a name for the discipline. Mei and colleagues published a survey of context engineering in July 2026, drawn from an analysis of over 1,400 papers, defining it as the systematic optimisation of the information payload supplied to a model rather than the wording of a prompt. Their decomposition into context retrieval and generation, context processing, and context management is the vocabulary this part uses.
6A.1 Two lifecycles wearing one name
The dividing line is when the artifact is created relative to when it is used.
Built context is authored or computed ahead of execution. Context cards written and reviewed by a domain expert. Embedding indexes generated on a schedule. Retrieval scopes declared per consumer. Curated exemplars. Compiled rules. All of it has an owner, a version, a review step and a release.
Assembled context is constructed during execution and mostly discarded. The subgraph traversed for one question. Session state and resolved references. Live signal from a stream. The trace of what a model considered and rejected.
The properties diverge on every axis that matters:
Built
Assembled
Created
ahead of execution
during execution
Governed by
change control
retention and privacy policy
Correctness means
consistent with source truth
sufficient for this task
Fails by
drifting silently from source
omitting something needed once
Version
explicit and required
none, and none is appropriate
Retention
permanent, versioned
policy-bound, usually short
Who owns it
a named person
the platform
Applying either regime to the other produces a specific pathology. Version session logs and you accumulate an undeletable privacy liability. Leave context cards unversioned and they drift from the systems they describe, silently, for as long as nobody checks.
Figure 6AOne word, two lifecycles
BUILT · before execution
Context cardsauthored and reviewed by a domain expert
Embedding indexesgenerated on a schedule, versioned with the model
Retrieval scopesdeclared per consumer, enforced at assembly
Traversed subgraphsthe specific neighbourhood for one question
Session statehistory, resolved references, user role
Live signalstreams, not last night's batch
Model tracesconsidered, rejected, assumed
BuiltAssembled
Governed bychange controlretention and privacy policy
Correctness meansconsistent with source truthsufficient for this task
Fails bydrifting silently from sourceomitting something needed once
Versionexplicit and requirednone, and none is appropriate
Retentionpermanent, versionedpolicy-bound, usually short
Ownera named personthe platform
Apply either regime to the other and you get a specific pathology. Version session logs and you accumulate an undeletable privacy liability. Leave context cards unversioned and they drift from the systems they describe, silently, for as long as nobody checks.
There is a second split inside built context that matters in regulated estates. Inside your own boundary, context can be declared through a contract stating schema, field semantics and quality thresholds. Outside it, on third-party feeds and ungoverned internal sources with no owner, context has to be inferred, and inference carries a confidence that must be carried forward rather than rounded up to true.
6A.2 The versioning problem, and the constraint that makes it hard
Built context needs versioning for the same reason a schema does. The obstacle is a geometric fact about embeddings that surprises people the first time they hit it.
A vector produced by one embedding model and a vector produced by another are not comparable. They occupy different spaces. Cosine similarity between them is arithmetic without meaning. Mix two model versions in one approximate-nearest-neighbour index and queries return neighbours from the wrong neighbourhood.
The consequence is that upgrading an embedding model forces a complete re-embed of the corpus. There is no transformation that makes the old vectors valid, and no partial migration that is safe.
The failure is silent, which is what makes it dangerous. Update query embeddings without re-embedding documents and every request returns 200 OK at normal latency with degraded answers. No exception is raised. No dashboard turns red. The only detectable symptom is that answers get worse, which is exactly the symptom nobody has an alert for.
The cost asymmetry is worth stating plainly. Re-embedding a million documents at around 500 tokens each costs roughly ten dollars in API fees with a small embedding model. The expensive part is the period during which nobody realised, and the trust that is spent recovering.
The safe migration pattern is blue-green, and it is engineering consensus rather than peer-reviewed practice. Add a second vector field sized for the new model. Backfill in the background while the existing index serves traffic. Dual-write new ingests to both. Validate recall against a held-out query set. Shadow a slice of live traffic. Cut over atomically, and keep the old index until you are certain.
The prerequisite is that every vector carries embedder_version and chunking_version as metadata. Without those two fields the migration cannot be validated and a mixed-version index cannot be detected after the fact.
Figure 6BWhy an embedding upgrade fails silently, and the only safe way through it
THE CONSTRAINT
The same document embedded by two models produces vectors in different spaces. Cosine similarity between them is arithmetic without meaning. Mix versions in one index and queries return neighbours from the wrong neighbourhood.
THE FAILURE MODE
200 OKnormal latencyno exceptionwrong answers
No dashboard turns red. The only symptom is that answers get worse, which is the one symptom nobody has an alert for. Re-embedding a million documents costs roughly ten dollars in API fees. The expensive part is the period before anyone noticed.
BLUE-GREEN MIGRATION · the only safe path
Add a second vector field sized for the new model
Backfill in the background while the existing index serves traffic
Dual-write new ingests to both
Validate recall against a held-out query set
Shadow a slice of live traffic
Cut over atomically, and keep the old index until certain
Prerequisite: every vector carries embedder_version and chunking_version. Without those two fields the migration cannot be validated and a mixed-version index cannot be detected after the fact.
An alternative is emerging and is worth watching rather than adopting. Rather than re-embedding, learn a transformation from the new model's space into the legacy space. Reported results include high retrieval recovery with negligible added query latency, and evaluation approaches that estimate a new model's performance far more cheaply than a full re-embed. These figures come from author write-ups rather than independent replication, so treat them as directional.
Dimensionality is a real lever. One documented migration moved 40 million vectors from 1536 dimensions to 256 using truncation-friendly embeddings. The index shrank from 23 GB to 4 GB, median latency fell from 28 ms to 11 ms, and ranking quality dropped by 0.8 nDCG points. For most enterprise retrieval that is an excellent trade, and it is available to anyone whose embedding model supports it.
Callout: The version bundle
An embedding model, an index, a chunking scheme and a prompt template are not four independent dependencies. They are one bundle, and a change to any of them invalidates evaluation results produced under the previous combination.
If you cannot answer "which model and which index produced this retrieval," you cannot reproduce any result your system has ever given.
6A.3 Reproducing a retrieval from six months ago
Definition bitemporality was a central argument of Paper 01. A regulator asks not what the number was, but what the definition was at the time.
The same question applies to context, and the answer is worse.
Given a query issued six months ago, reconstructing exactly what was retrieved requires the corpus as it stood, the embedding model version, the index build, the chunking scheme, the query as issued, and the ranking parameters. Vector stores do not retain any of that by default.
Bi-temporal knowledge graph memory is the partial exception. Systems that track validity on both the real-world timeline and the transactional timeline can answer what was known as of a given date, which is the correct model and is confined to graph-shaped memory rather than general retrieval.
There is one useful empirical finding. Work reconstructing corpus snapshots a year apart found that relative model rankings shifted only slightly, with rank correlation near 0.98. Benchmark conclusions are reasonably robust to corpus drift. That is a different question from whether your own audit trail is reproducible, and it should not be read as reassurance about the second.
This is a genuine gap. No standard exists for point-in-time reconstruction of a general vector store retrieval. If it matters in your estate, you have to build it, and building it means retaining index builds rather than only the current one.
6A.4 Staleness, and why similarity cannot see it
Here is the defect at the centre of built context.
Semantic similarity has no time dimension. A document describing a policy retired eighteen months ago scores against a query exactly as well as the document that replaced it. Relevance and currency are orthogonal, and retrieval optimises only the first.
A commonly cited illustration: documentation for a deprecated authentication scheme continues to retrieve at 0.92 similarity long after the organisation migrated away from it. Nothing in the stack objects.
The research on fixing this is honest about how partial the fixes are:
Adding a recency prior with a half-life lifted the proportion of correct latest-version retrievals from zero, under pure cosine ranking, to 0.60 on a hard test set. Better, and parameter-sensitive and far from solved.
A heuristic attempt to detect topic evolution automatically scored 0.08 macro-F1, which is close to useless.
Across published temporal question-answering benchmarks, models frequently score below 20 percent on time-sensitive question types without explicit temporal handling. That includes strong frontier models.
One 2026 evaluation found that published memory systems underperformed a deterministic freshness rule by wide margins on a fact-consolidation task, where the correct behaviour is to recognise that an earlier fact has been superseded.
Freshness benchmarks worth knowing: FreshQA and RealtimeQA for dynamic answers, TimeQA for as-of correctness, and newer sets specifically testing whether a system recognises that a stored memory has become invalid after a state change.
There is an index-level version of the same problem. The graph and inverted-file structures that make approximate nearest neighbour search fast do not support cheap incremental updates. New content typically lands in a small buffer scanned linearly, so as that buffer grows, query latency degrades at constant corpus size. Static benchmarks never surface this because they never insert during evaluation.
The workable pattern is a small overlay index covering recent changes, queried alongside the main index, with a periodic full rebuild. It is unglamorous and it is the difference between a system that reflects yesterday and one that reflects last quarter.
What nobody has published: an error rate or cost attributable to stale context in a production system. Every practitioner has stories. There is no number. That absence is the reason this is Experiment 11.
6A.5 Session context and the memory question
Assembled context has its own literature, and the honest summary is that it is contested.
The architectural idea is established. Treat the model's context window as a small fast tier and everything else as a larger slow tier, and move information between them under explicit policy. Early work framed this as virtual memory management for language models, with a main context analogous to RAM, recall and archival storage analogous to disk, and function calls that page information across the boundary. Reported results on targeted retrieval tasks were strong, including cases where a baseline scored zero on nested retrieval and the managed system did not degrade.
Later systems added structure. Bi-temporal knowledge graphs for memory, splitting episodic records, extracted entities and derived communities, with retrieval combining semantic search, keyword search and graph traversal without a model call at retrieval time. Reported latencies in the low hundreds of milliseconds at the 95th percentile.
Then the benchmark numbers, which is where care is required.
On long-conversation benchmarks, published memory systems and a plain full-context baseline are frequently within a few points of each other, and in at least one widely cited comparison the full-context baseline scored highest overall. The vendors dispute each other's numbers publicly, with claimed differences of fifteen to twenty points attributed to harness configuration, timestamp handling and whether searches ran in parallel. Independent evaluations diverge sharply from both, in one case reporting a memory product well below a no-memory baseline on the same benchmark.
Our position: do not select a memory system from published benchmark tables. The numbers are not comparable across harnesses, the disputes are unresolved, and the differences between systems are smaller than the differences between evaluation setups. Run your own harness on your own conversations.
The finding that does replicate is about degradation. Across long-interaction benchmarks, sustained sessions show accuracy dropping by roughly 30 percent, with larger declines when a system reads a full history rather than a curated retrieval. Whatever memory architecture you choose, long sessions get worse.
The finding that matters most for design is that memory can be net negative. One 2026 study found that extraction quality dominates retrieval sophistication, and that aggressive summarisation in a well-known memory system produced worse results than passing raw chunks. Badly curated memory is worse than no memory. That is not an argument against memory. It is an argument against unbounded accumulation with no curation policy.
6A.6 More context is not better, and this is measured
Three separate lines of evidence, all pointing the same way.
Position matters and the middle is worst. The lost-in-the-middle result showed a U-shaped accuracy curve, with degradation exceeding 30 percent when the relevant passage sits in the middle of a long input rather than at either end. It replicated across model families and has a plausible mechanism in positional encoding decay.
Length degrades accuracy well before the stated limit. An evaluation across eighteen models found accuracy declining monotonically as input grew, at every increment, not only near the advertised context ceiling. A separate benchmark removing literal lexical overlap between question and answer saw a strong model fall from 99.3 percent at 1,000 tokens to 69.7 percent at 32,000.
Retrieved passage count has an optimum, and it is small. Multiple studies find accuracy peaking somewhere between three and ten passages and declining after. Adding retrieved context past that point introduces hard negatives that mislead generation, and retrieval accuracy consistently outruns answer accuracy for exactly that reason.
The practical implication is the opposite of the instinct. When answers are poor, the reflex is to retrieve more. The evidence says retrieve less and retrieve better, and that a larger context window is not a substitute for a smaller, correct one.
Figure 6CMore context is not better, measured three ways
POSITION
Lost in the middle
>30%
accuracy lost when the relevant passage sits in the middle rather than at either end. Replicated across model families.
LENGTH
Degradation before the limit
99.3 → 69.7%
on a benchmark that removes literal lexical overlap. Accuracy falls at every increment, not only near the advertised ceiling.
QUANTITY
Retrieved passage count has an optimum
3 to 10
passages is where accuracy peaks across multiple studies. Beyond it, hard negatives mislead generation and answer accuracy falls below retrieval accuracy.
The practical implication is the opposite of the instinct. When answers are poor, the reflex is to retrieve more. The evidence says retrieve less and retrieve better, and that a larger context window is not a substitute for a smaller, correct one.
Liu et al., TACL 2024 · Chroma context rot evaluation across 18 models · NoLiMa · multiple RAG passage-count studies
6A.7 Compression, caching and what each one costs you
Two ways to spend less, with very different risk profiles.
Prompt caching is close to free. Reusing an identical prefix produces byte-identical behaviour with no quality change. Providers price cached input at a fraction of base input, with reported cost reductions up to 90 percent and latency reductions up to 85 percent on long prompts. If one change is made after reading this paper, make it this one.
Semantic caching is not the same thing and carries real risk. Serving a stored answer for a similar-but-not-identical query changes behaviour, and the threshold governs whether it changes it acceptably. Published hit rates cluster in the 60 percent range on curated evaluations, while independent framing suggests production hit rates land closer to 20 to 45 percent, and warns that the headline accuracy figures describe match precision rather than hit frequency. There is also an attack surface: research has demonstrated response hijacking through deliberate key collisions, with defences based on cluster centroids proposed but not standardised.
Prompt compression works well and degrades in a specific direction. Published methods reach high compression ratios with small accuracy losses, including results around 20 times compression with roughly 1.5 percent performance loss on structured reasoning tasks, and task-agnostic variants that run several times faster than their predecessors. The rule of thumb from practice is negligible loss at two to three times and a few points at ten times.
The caveat is important for agents specifically. Those numbers come from clean benchmark tasks. In a multi-turn agent, compression can silently drop a constraint that was stated several turns earlier, or resolve a pronoun to the wrong entity. Faithfulness degrades fastest for numerical and temporal detail, which is precisely the content that matters in a regulated decision. Compressed prompts are also considerably harder to debug after the fact.
6A.8 Attribution, and the limit of what can be reconstructed
Paper 01 argued that a decision needs a retained trace. Context makes that harder than it sounds.
Attributing an output to specific context is solved well enough to use. Methods exist that fit a sparse surrogate over context ablations to identify which retrieved passages actually caused a statement, at a cost of tens of additional inference calls. One useful property: these methods reliably surface a prompt injection as the dominant influence on an output, which makes them a security instrument as well as an audit one.
The caution from the same literature is that entailment is not causation. A passage that supports a claim is not necessarily the passage that produced it, and model self-citations look plausible while frequently not matching what actually drove the output.
Provenance models exist and map cleanly. The W3C provenance vocabulary of entities, activities and agents fits agent traces directly, with extensions published for agent-specific structures and for integrating tool-call telemetry into a single provenance graph.
The limit is worth stating precisely, because it is often overclaimed. Recent work argues that reasoning provenance cannot in general be faithfully reconstructed from computational state. Practitioner experience agrees for a simpler reason: agent runs are rarely deterministic, since providers change backend behaviour and tools return live data.
So the achievable goal is not replay. It is to prove which inputs and which authority produced an output, detect tampering, and reconstruct the decision path well enough to explain it. That is a lower bar than replay and it is still not met by default configurations, which capture the shape of an interaction without its substance.
What to retain, concretely: model identifier and version, digests of the prompt and messages, the identifiers and versions of every retrieved chunk, ranking parameters, the entitlements under which the retrieval ran, and the authority under which the action was taken. Store payload hashes by default and put the payloads themselves behind an explicit retention and access policy.
6A.9 Retrieval is an entitlement boundary
The most consequential sentence in this part is the shortest.
Relevance ranking is not authorisation.
A retriever optimises for similarity. It has no concept of who is asking. If entitlement filtering is applied after retrieval, or applied at document level when permissions are set at chunk level, or omitted because the index is assumed to be internal, then the system will return content the requester is not entitled to see. Published work formalising this gap found cross-tenant leakage in nearly every probe of ungated retrieval.
Three requirements follow, and none of them is optional in a regulated estate.
Entitlements belong on the chunk, not the document. Classification, owner, permitted roles and permitted tenants, stored as metadata, because permissions change after ingestion and a document-level grant is too coarse for a chunk-level answer.
Enforcement happens at query time. Not at ingestion, not in the application layer afterwards. Every downstream component that sees candidates, including rerankers and caches, must only ever see authorised ones, because those components persist traces.
An embedding of restricted data is restricted data. This is the part most teams have not internalised. Embedding inversion research has demonstrated recovery of a large majority of short text inputs exactly from their embeddings alone, with transferable attacks that do not require access to the target model. Proposed defences trade utility for privacy and none is robust across settings. A vector store containing embeddings of confidential documents inherits the confidentiality classification of those documents, and should be isolated, access-controlled and retention-managed accordingly.
Figure 6DRelevance ranking is not authorisation
Entitlements belong on the chunkclassification, owner, permitted roles and tenants as metadata. Permissions change after ingestion and a document-level grant is too coarse for a chunk-level answer.
Enforcement happens at query timenot at ingestion, not in the application afterwards. Every component that sees candidates persists traces, including rerankers and caches.
An embedding of restricted data is restricted datainversion research has recovered the majority of short text inputs exactly from embeddings alone, with transferable attacks needing no access to the target model. Classify the vector store at the level of its source content.
6A.10 What this means in practice
Six things, in the order we would do them.
Co-version the bundle. Embedder version and chunking version on every vector, prompt templates in a registry with evaluation-gated promotion, index build identifiers retained. Without this nothing else in this part is verifiable.
Instrument freshness explicitly. Latency and error dashboards cannot see stale context. Add a recency signal, reconcile the index against source truth on a schedule, and set a freshness expectation per content class.
Cap retrieval and measure your own optimum. Start between five and ten passages and find the peak on your own data rather than assuming more is better.
Bound and curate memory. Decide what gets written, not only what gets read. Unbounded accumulation with weak extraction performs worse than no memory at all.
Design retention for reconstruction rather than replay. Retain identifiers, versions, digests and authority. Accept and document that exact replay is not achievable.
Treat the retrieval layer as an entitlement boundary equal to the database. Chunk-level entitlements, query-time enforcement, tenant isolation, and embeddings classified at the level of their source content.
Part 6B: What leaves the boundary, and what can stop it
Part 6A treated retrieval as an entitlement boundary. That covers what an agent is allowed to read. It says nothing about what happens to the data afterwards, once it is inside a prompt, inside a tool call, inside a message to another agent, or inside a provider's logs.
This part is about the second half. It is the part of agentic architecture where the research is clearest about the threat and least comforting about the defence.
6B.1 The finding that should govern the design
Start with the conclusion, because it changes how everything else in this part should be read.
Indirect prompt injection is not solved, and the defences that rely on the model to protect itself do not hold under a determined attacker.
The evidence is specific. A 2026 study built an adaptive attacker and ran more than 20,000 attacks across nine defence configurations. Every defence that depended on the model defending itself eventually broke. The one that held was output filtering applied outside the model, which recorded zero leaks across 15,000 attacks. The frontier models tested all leaked the embedded secrets they were instructed to protect.
A companion critique makes the methodological point. Out-of-band defences that look strong are largely validated on static benchmarks, and that is the same methodology that made in-model defences look strong until adaptive, defence-aware attacks broke twelve of them at over 90 percent success.
The benchmark literature agrees on the direction. On AgentDojo, the targeted attack success rate against the most susceptible model reached 53 percent under a straightforward injected-instruction attack. Deploying a dedicated secondary detector dropped it to 8 percent. In-context prompting defences, the ones that instruct a model to be careful, sometimes made the attack success rate worse.
Frontier labs publish numbers that point the same way while showing real progress. Anthropic's published figures for its most recent models hold single-attempt attack success near 0.1 percent on an agent red-teaming benchmark, rising to roughly 5 to 6 percent after a hundred adaptive attempts. In a browser-agent setting the same reporting shows hijack succeeding 31.5 percent of the time per attempt without safeguards and 0.5 percent with them.
Read those together and the design conclusion is not that models are hopeless. It is that model-layer defence is risk reduction and never containment, and containment has to come from somewhere a prompt cannot argue with.
Figure 6EOnly one class of defence held under an attacker who knew it was there
In-model defence
instructing the model to be careful, in context
broke under adaptive attack. Twelve defences at over 90% success. Sometimes raised attack success.
Out-of-band, statically validated
capability and flow-control schemes tested on fixed benchmarks
not yet tested against defence-aware attackers. Same methodology that made in-model defences look strong.
Secondary detector
a separate model scoring tool output for injected instructions
attack success 53% → 8% on AgentDojo. Materially better than in-context prompting.
Output filtering, outside the model
deterministic inspection of what is leaving, in application code
zero leaks across 15,000 attacks. The only category that held.
WHAT THE FRONTIER NUMBERS LOOK LIKE
0.1%single-attempt attack success on an agent red-team benchmark
5 to 6%after 100 adaptive attempts against the same model
31.5%browser-agent hijack per attempt without safeguards
0.5%the same setting with safeguards enabled
Model-layer defence is risk reduction, never containment. The question to ask about any published defence number is not whether it works but whether it was tested against an attacker who knew it was there. Static and adaptive results differ by an order of magnitude.
Adaptive attack studies 2026 · AgentDojo, Debenedetti et al., NeurIPS 2024 · Anthropic published model card figures
Callout: The question to ask about any injection defence
Not "does it work" but "was it tested against an attacker who knew it was there?"
Static benchmark results and adaptive red-team results differ by an order of magnitude. If a vendor cannot tell you which one their number is, assume static.
6B.2 Sensitive data does not stay where you filtered it
The controls that exist are mostly content filters applied to text. Agents do not only produce text.
The specific gap worth knowing. AWS documents that its Bedrock Guardrails sensitive-information filters evaluate text content only, and in tool-use workloads do not evaluate the arguments a model writes into a tool call. So personal data can be filtered out of the visible response and passed intact into a downstream system call. That is a documented vendor limitation rather than an inference, and every architecture that assumes a guardrail covers the whole path inherits it.
Google's equivalent has a different shape of gap. Model Armor screens prompts and responses inline and integrates sensitive data detection, and when it uses a de-identification template it does not pass the masked data back for further processing. Its logging, when enabled, records the full payload, which means the control designed to prevent exposure can create a second copy of the thing you were protecting.
Salesforce's trust layer performs masking and holds zero-retention agreements with model providers, and its own documentation notes that data masking for large language models is currently disabled for agents specifically, while being available for other embedded features.
None of these is a scandal. They are the normal state of a young control surface. The point for an architect is that the coverage boundary of a data protection control is rarely the same as the boundary of the agent's activity, and the difference is where regulated data escapes.
Figure 6FWhere the filter stops, and where the data keeps going
Tool argumentsOne major provider documents that its sensitive-information filter evaluates text content only and does not evaluate what the model writes into tool call arguments.
LogsAnother documents that when logging is enabled, the screening service records the entire payload, creating a second copy of the data being protected.
Agent scopeA third documents that masking for language models is currently disabled for agents specifically, while remaining available for other embedded features.
None of these is a scandal. They are the normal state of a young control surface. The architectural point is that the coverage boundary of a data protection control is rarely the same as the boundary of the agent's activity, and the difference is where regulated data escapes.
6B.3 Redaction costs less than people assume
There is a widespread belief that stripping sensitive fields will wreck task performance, which is used to justify not doing it.
The measured position is more forgiving. A study of sanitisation impact across tasks found effects ranging from a fraction of a percent to very large, and the determining factor was whether the redacted entity was needed to reason about the task rather than how many entities were removed. Many enterprise workloads land in the low-impact band.
The technique matters more than the decision to redact. Deleting content degrades performance considerably more than replacing it with a type-consistent placeholder or synthetic substitute, because the model retains the structure of what it is reasoning about. Named-entity-based label masking is the gentlest of the common approaches.
Clinical work provides the strongest accuracy figure available: a de-identification study reported over 99 percent of patient-sensitive characters correctly redacted, with a small over-redaction rate. Hybrid pipelines that combine pattern matching with named entity recognition reach around 96 percent recall in general use, and purely model-based redaction misses edge cases in ways that are hard to predict.
Our position. Substitute rather than delete, apply it at the boundary rather than trusting the model to withhold, and measure the task-performance delta on your own workload rather than assuming it is prohibitive. In most cases the cost of redaction is smaller than the cost of the review meeting about whether to do it.
6B.4 The protocol layer has its own attack surface
Part 3 argued that MCP and A2A solve connection rather than coordination. They also introduce specific, named attack surfaces that did not exist before.
On the tool side. Tool poisoning, where a malicious instruction lives in a tool's description rather than in the data, is now a documented category with its own place in the agentic security taxonomy. Rug pulls, where a tool behaves correctly until it is trusted and then changes, have been demonstrated against real connectors. Multiple vulnerabilities in widely used MCP components were assigned CVEs during 2025, including remote code execution in a common remote-MCP bridge. One study measured a tool-poisoning attack succeeding against a mainstream model over 70 percent of the time.
On the agent-to-agent side. The primary weakness is that an agent card, the document declaring what an agent is and what it can do, is signable but not required to be signed. That makes impersonation cheap. Published analysis reports high success rates for spoofing in peer-to-peer discovery, and a documented session-smuggling technique exploits multi-turn trust between agents rather than any single message.
The most useful summary comes from the agentic security taxonomy published in late 2025, which introduces insecure inter-agent communication as a category with no equivalent in the older application-focused lists, and observes that there is no widely deployed equivalent of mutual TLS between agents.
That absence is the practical finding. The transport is usually encrypted. The identity of the thing on the other end frequently is not verified.
6B.5 Identity is arriving. Delegation is not.
Per-agent identity is genuinely shipping, and it is the fastest-moving area in this paper.
Microsoft issues agents first-class identities on its existing service principal infrastructure, which brings conditional access with it. Google issues agents cryptographic identities based on the workload identity standard, with tokens bound to the agent's certificates to prevent replay, and a credential manager so that an agent never handles a raw user credential. AWS creates workload identities for agents automatically and provides a token vault.
The unsolved part sits one step further along, and it is the same gap Part 3 identified from a different direction.
What happens when agent A delegates to agent B? The standards work in progress is explicit that each hop must re-scope and re-bind the security context so a downstream service can evaluate provenance and authorisation, and that autonomous actions must be distinguishable from delegated ones. Handing a bearer token from one agent to the next is precisely the onward impersonation pattern the underlying identity framework warns against.
No shipping product solves this today. Workload identity authenticates who something is. It does not carry authorisation, delegation depth, or proof of possession, and the extensions that would are drafts rather than deployments.
This is the single most concrete capability gap in the paper, and it is Experiment 3.
6B.6 What to build, given all of that
Four things, in order.
Put a deterministic control in the path. Not a prompt instructing care. Something outside the model that inspects what is leaving and can refuse it. The evidence is that this is the only category of defence that has held under adaptive attack.
Inspect tool call arguments, not only text. Because the shipped filters largely do not, and because that is where regulated fields travel.
Verify the identity of the agent on the other end. Signed agent cards where the protocol supports them, and mutual authentication where it does not.
Scope credentials per task and re-bind at every hop. Even though the standards are unfinished, the pattern of issuing a narrow, short-lived credential for one purpose is available today and is what the standards are converging on.
6B.7 What you can rely on, and what you still build
The vendor position is genuinely better than it was a year ago, and it is uneven in a way that matters for planning. This table is what a customer can lean on today against what remains their own work.
Area
Reasonably relied on today
Still yours to build
Data governance inherited by agents
Query-engine enforcement of masking, row access and classification, inherited automatically by agents built on the same platform. The most mature area in this table.
Enforcement when an agent reads from a second platform, or from anything outside the governed estate.
Content and PII filtering
Inline screening of prompts and text responses, with custom pattern support, enforceable through policy.
Inspection of tool call arguments. Control over what screening itself writes to logs. Coverage for agent-specific paths where masking is not yet enabled.
Per-agent identity
First-class agent identities on existing identity infrastructure, conditional access, credential vaulting so agents do not hold raw user credentials.
Scoping a credential to one task. Attenuating capability across a delegation chain. Distinguishing an autonomous action from a delegated one.
Agent-to-agent entitlement
Nothing.
All of it. There is no shipping product that re-scopes and re-binds authorisation across a hop between two vendors.
Inter-agent message security
Transport encryption.
Verifying the identity of the agent on the other end. Card signing is supported but not required, and there is no widely deployed mutual authentication between agents.
Agent inventory and governance
Cross-platform discovery of agents across several major vendors, policy application, and in at least one product a runtime kill switch.
Whether the inventory is complete, since discovery covers named integrations rather than everything an engineer can deploy.
Model and retrieval evaluation
Managed evaluation with judge scoring, generally available across the major clouds.
Agent trajectory evaluation, which is more often preview or an open-source framework than a managed service.
Confidential inference
Hardware trusted execution on current GPU generations, measured at under 7 percent overhead for large models.
The attestation plumbing, particularly where a CPU enclave service and a GPU attestation format do not natively interoperate.
Two observations from reading down that table.
The mature column is mostly data-plane governance. The controls that work best are the ones that existed before agents and were extended to cover them. That is an argument for building agents on top of a governed platform rather than beside one.
The empty rows are all about crossing a boundary. Between agents, between vendors, between the filtered path and the unfiltered one. Which is the same finding as Part 3 and Part 6A, arriving from a third direction.
Part 6C: Measuring agents, with instruments of unproven validity
This paper argues throughout that things should be measured. That obligation applies to the measuring.
The uncomfortable position is that the evaluation layer for agents is less reliable than the confidence placed in it. Benchmarks are contaminated and saturating, the dominant scoring method has known and systematic biases, and the observability standard that would make results portable is still experimental.
None of that is an argument against evaluating. It is an argument for knowing what your numbers are worth.
6C.1 Judges are less reliable than the headline figure suggests
Using one model to score another is now the default method, on grounds of cost and scale. The commonly quoted justification is that a strong judge agrees with human experts around 85 percent of the time, comparable to the rate at which humans agree with each other.
That figure is real and it is not chance-corrected.
The largest systematic study of judge reliability, covering 21 judges and several hundred thousand judgments, found that the gap between raw agreement and chance-corrected agreement is universal, and on a common benchmark it runs 33 to 41 percentage points. Judge rankings of the same systems shifted by up to 14 positions depending on which benchmark was used. Two judges in production showed very high test-retest consistency alongside severe position bias, which the authors describe as a consistency-bias paradox: the judge reliably gives the same answer, and the answer is reliably influenced by something irrelevant.
The specific biases are documented and large. Position bias can reach a 75 percent preference for whichever response appears first. Self-enhancement bias, where a judge favours output from its own model family, runs in the 10 to 25 percent range. Verbosity bias favours longer answers independent of quality.
Figure 6GThe judge agreement figure everyone quotes, and what it looks like corrected
Raw agreementthe quoted figure
85%
Chance-correctedwhat it measures once agreement by luck is removed
33 to 41 points lower
Both describe the same judges on the same benchmark. Only one of them is a statement about discriminative ability.
DOCUMENTED BIASES
up to 75%preference for whichever response is shown first
10 to 25%preference for output from the judge's own model family
14 positionshow far system rankings moved depending on benchmark choice
MITIGATIONS, WITH MEASURED EFFECT
Swap positions and averageconsistency improves from roughly 65% to 78%
Reference-guided gradingfailure rate falls from around 70% to around 15%
Panels rather than one judgeoutperforms any single judge
Rubrics rather than open qualityreduces sensitivity to formatting and length
In specialised domains, chance-corrected agreement between model and human raters on subjective criteria has been measured between roughly 0.23 and 0.36, which is moderate at best and would not support a high-stakes decision without human review.
Systematic study of 21 judges, approximately 541,000 judgments, 2026 · Zheng et al., MT-Bench, NeurIPS 2023
The mitigations work and are cheap. Swapping positions and averaging improves consistency substantially. Reference-guided grading, where the judge compares against a known good answer rather than judging in the abstract, has been measured cutting failure rates from around 70 percent to around 15 percent. Panels of several judges outperform any single one. Rubrics beat open-ended quality judgments.
Domain matters more than most teams expect. In specialised settings, agreement between model and human raters on subjective criteria has been measured at chance-corrected levels between roughly 0.23 and 0.36, which is moderate at best and would not support a high-stakes decision.
Callout: Three rules for judge-based evaluation
Report chance-corrected agreement, not raw agreement. If a vendor or an internal team reports a percentage without saying which it is, assume raw and discount accordingly.
Validate the judge on your domain before trusting it, against human labels, and re-validate whenever the judge model or its prompt changes.
Never let a model family judge itself in a procurement decision.
6C.2 Benchmarks are contaminated, and the contamination is measurable
Public agent benchmarks are the basis of most vendor comparisons, and their integrity is worse than their prominence suggests.
The clearest admission comes from a frontier lab rather than a critic. OpenAI published an analysis of why a widely used software engineering benchmark no longer measures frontier capability, and a manual audit of 138 failures found that around 59 percent were caused by flaws in the tests rather than limitations of the model. Separate work found that roughly a third of successful patches on the same benchmark involved solution leakage.
A late 2025 study compared performance on a public benchmark against a contamination-controlled equivalent and found models scoring around three times better on the public version, and six times better at identifying which files needed editing. That is a memorisation signal rather than a capability signal.
The methodological review is blunter still. An audit of agent benchmarks found that one widely used suite has insufficient tests, another counts empty responses as successes, and that problems of this kind can misstate agent performance by up to 100 percent in relative terms.
What follows for procurement. Public leaderboard position is not evidence about your estate. Ask for results on contamination-controlled or continuously refreshed benchmarks, and weight your own held-out task suite above any published number.
6C.3 Reliability compounds badly, and this is the number that matters
One piece of arithmetic explains more agent failures than any benchmark.
A workflow of twenty steps, each succeeding 95 percent of the time, completes successfully about 36 percent of the time. At 99 percent per step it completes about 82 percent of the time.
Per-step reliability that sounds excellent produces end-to-end reliability that is unusable, and the effect is invisible in any evaluation that scores single steps. This is why trajectory-level evaluation, scoring the whole path rather than each decision, is not a refinement but the only measurement that reflects what a user experiences.
Figure 6HPer-step reliability that sounds excellent, end to end that is unusable
82%20 steps at 99% each
36%20 steps at 95% each
12%20 steps at 90% each
This arithmetic explains more agent failures than any benchmark, and it is invisible to evaluation that scores single steps. It is also the strongest argument for the routing discipline in Part 0: every step removed from a chain improves the product, and converting a step to a deterministic rule takes its reliability to one.
It is also the strongest argument in this paper for the routing discipline in Part 0. Every step removed from a chain improves the product, and converting a step to a deterministic rule takes its reliability to one.
6C.4 Evaluation belongs in production, not only before it
The tooling to run evaluation continuously exists across the major platforms, with the usual caveat about maturity.
Managed evaluation for models and retrieval is broadly generally available across the large cloud providers. Agent-specific evaluation, meaning trajectory scoring and tool-use correctness, is more often in preview, and in at least one case is offered as an open-source framework rather than a managed service. Trajectory metrics that do exist typically cover exact and in-order matching of tool call sequences against a reference, plus precision and recall over the steps taken.
The connective tissue is the OpenTelemetry conventions for generative AI, which several observability products now build on. Those conventions remain at development status, with no stable release, which means attribute names can still change under you. They are nonetheless the right thing to build on, because the alternative is an evaluation result that means something different in each tool.
The gap worth naming. There is no independent, peer-reviewed measurement that adopting evaluation-driven development improves production agent reliability by a specific amount. Figures to that effect circulate, and they come from vendor case studies. That absence is why Part 7 proposes measuring it rather than asserting it.
6C.5 Where evaluation meets model risk
For a regulated institution there is a connection worth making explicit.
Part 6 established that US model risk guidance was revised in 2026 to exclude agentic AI from scope, which raises rather than lowers the burden on the institution. The practical consequence is that the evidence a validation function would normally demand has to be generated voluntarily, and evaluation is where it comes from.
That argues for wiring the same evaluators into both pre-deployment testing and production monitoring, so that a score means the same thing in both places, and for retaining evaluation results with the same discipline as the decision traces in Part 6A. An evaluation suite that exists only in a notebook produces no evidence.
Part 7: Where to experiment
Everything above is either evidence or argument. This section is the part that turns it into something checkable.
The same discipline applies as in Technical Paper 01. Each experiment states a prediction in advance and a result that would change our position. If you cannot write down what would prove you wrong, you are running a demo.
Four of these are direct replications of published work with one variable added, which is the cheapest way to produce a credible result.
Experiment 1: The turf war, with a precedence object
The claim under test. A declared contested-directive object reduces resolution by force and shortens time to settle in a multi-agent conflict.
Why it is worth running. It is a direct replication of a published Anthropic experiment with exactly one variable added, against baselines that are already published. That makes it unusually cheap to interpret.
Design. Reproduce the migration turf war: three agents, three conflicting target languages, each initially unaware of the others. Two arms.
Arm A, control. As published. No shared state.
Arm B. Before acting, each agent resolves the target service against a semantic core containing a contested-directive object naming both claimants, both authority scopes, the arbitration owner, and a behaviour constraint that permits planning and forbids deployment while unresolved.
Run 120 episodes per arm to match the published sample.
What to measure. Proportion settled by force, by passivity, by truce, and unsettled. Time to resolution. Incidence of sabotage behaviours, which the published study characterises well enough to classify against.
Prediction, stated in advance. Resolution by truce rises substantially in Arm B for mid-capability models. The effect is smaller for the most capable models, because the published data already shows 98 percent truce for the strongest model, leaving little headroom.
The result that would change our position. If Arm B shows no improvement, then declaring a conflict is not sufficient and the constraint must be enforced at a gateway rather than published as a fact. That is a more expensive answer and we would say so.
Effort. Four to six weeks with two engineers.
Experiment 2: Correlated failure blast radius
The claim under test. A shared semantic core increases the blast radius of a wrong definition, and the increase is worth the recoverability it buys.
Why it is worth running. It tests our own counter-argument from Part 5.6 rather than avoiding it. Publishing a result that partially undermines your own position is the fastest way to be taken seriously by a technical buyer.
Design. Inject one incorrect definition, a semi-additive measure treated as additive is a good choice because the error is plausible and the effect is arithmetic.
Arm A. Shared semantic core. All agents resolve the definition from one place.
Arm B. No shared core. Three agents hold their own version of the definition, one of which is the same error.
What to measure. Number of downstream artifacts affected. Time to detect. Time to correct all affected artifacts once detected. Number of artifacts still wrong 30 days after correction.
Prediction. Arm A is worse on blast radius and detection time, and dramatically better on time to full correction. Arm B never reaches full correction.
The result that would change our position. If Arm B's divergence causes the error to be caught faster and the residual corrects itself, the diversity argument has more merit than we have allowed, and Part 5.6 needs rewriting rather than defending.
Effort. Three to four weeks.
Experiment 3: Entitlement propagation across a vendor hop
The claim under test. End-user entitlements do not survive an A2A hop between two vendors.
Why it is worth running. It is the most concrete unsolved problem in Part 6, it is checkable in days rather than months, and a negative result is a finding that would be widely read.
Design. Build the simplest possible chain. A user with restricted access to a subset of rows invokes an agent on platform 1, which delegates through A2A to an agent on platform 2, which reads data the user is not entitled to see.
Test the chain under three configurations: service-account authentication, end-user credential authentication where both platforms support it, and a token exchange using RFC 8693.
What to measure. Whether the restriction holds. Where in the chain it is lost. What the audit trace records about the identity under which the read occurred.
Prediction. The restriction holds under end-user credentials within a single vendor and is lost or silently broadened at the vendor boundary in at least one of the three configurations.
The result that would change our position. If entitlements propagate cleanly across vendors today, then the identity gap is smaller than Part 3.3 claims, and the paper should say so.
Effort. Two to three weeks. The cheapest experiment here and possibly the most immediately useful to a client.
Experiment 4: The rate contract
The claim under test. A declared rate and concurrency contract at an agent boundary prevents the resource collapse pattern.
Why it is worth running. It is almost too easy, which is why it should be done early. It also produces a number that a platform owner will act on immediately.
Design. Reproduce the job queue scenario. Fifteen agents, finite bandwidth, no coordination channel.
Arm A. No contract. Expect the published pattern.
Arm B. A declared contract at the boundary: maximum request rate per agent, concurrency limit, backoff requirement, with rejection at the gateway rather than instruction in the prompt.
What to measure. Requests issued, jobs accepted, acceptance ratio, total compute consumed, and time to complete the actual work.
Prediction. Arm B completes the same work with an acceptance ratio one to two orders of magnitude better. This one we are confident about, which is why it is worth doing first as a warm-up that produces a real artifact.
Effort. Two weeks.
Experiment 5: Authority-aware credulity
The claim under test. Recording which source is authoritative for which concept improves an agent's resistance to a stale or wrong source, without making it dismiss a correct dissenting one.
Why it is worth running. It tests both halves of the trust trade-off that the Anthropic study identifies as impossible to fix with one dial.
Design. Replicate both the gullibility routing task and the hidden-profile task, with a third arm added to each.
Arm A. As published, no authority model.
Arm B. An authority projection recording which source is authoritative for which concept in which jurisdiction.
Arm C. Arm B plus a disagreement object where a dissenting claim can be filed and must be resolved before a decision is issued.
What to measure. Routing accuracy against the lying-source rate, from the first task. Proportion of episodes where the hidden-best option wins the vote, from the second. The key measure is whether Arm B improves the first while degrading the second, which is the trade-off the study predicts.
Prediction. Arm B improves credulity resistance and worsens hidden-profile performance, because it gives the agent a reason to discount a non-authoritative source that happens to be right. Arm C recovers the hidden-profile loss because the dissent has somewhere to go.
The result that would change our position. If Arm C does not recover the loss, then a disagreement object is insufficient and the problem is dispositional rather than structural, which would be a significant and publishable negative result.
Effort. Five to six weeks. The most intellectually interesting experiment in this list.
Experiment 6: Cross-vendor audit reconstruction
The claim under test. A decision chain spanning three vendors cannot currently be reconstructed to an audit standard.
Design. Build a three-tier chain, ideally system of record, data platform, and customer-facing app. Have it make a consequential decision. Then attempt reconstruction six weeks later using only retained telemetry, with no access to the running system.
Score against a defined audit checklist: which agent acted, in what order, on what data at what version, under which definition in force at the time, what was considered and rejected, and whether the record can be shown to be unaltered.
What to measure. Which of the six items can be answered, and from which source. Where the trace breaks.
Prediction. Items one and two are answerable. Item three, the definition in force, is not, which is the finding from Technical Paper 01 recurring. Items four and five are not answerable at all with default configurations.
Effort. Six to eight weeks. The most valuable output for a regulated client and the one most likely to become a reference architecture.
Experiment 7: Semantic collusion
The claim under test. A shared governed vocabulary increases the rate of collusive convergence between competing agents.
Why it is worth running. Because we raised the risk in Part 5.5 and it would be dishonest to raise it without measuring it.
Design. A Bertrand or Cournot setting with three to eight agents, no direct communication channel, both arms observing a public board.
Arm A. Agents hold their own private schemas for cost, margin, and inventory.
Arm B. All agents resolve those concepts from one shared governed semantic layer.
What to measure. Rounds to price convergence. Stability of the collusive equilibrium. Whether the equilibrium survives an exogenous shock.
Prediction. Arm B converges faster. If it does, that is a real finding against a design we advocate, and it comes with a mitigation requirement: monitoring for correlated behaviour, and structural limits on inter-agent observability.
Effort. Four weeks.
Experiment 8: OSI round trip with agent grounding
The claim under test. A semantic model authored in one platform can ground an agent in another without loss of the properties that matter.
Design. Author twenty definitions of varying complexity in one platform. Export to OSI. Import into a second platform. Ground an agent in each. Ask the same 50 questions of both.
What to measure. Answer agreement between the two. Which definition features survive the round trip and which are silently dropped. Our expectation is that simple aggregations survive and that additivity rules, security predicates, time-grain semantics, and FX policy are where loss occurs.
Effort. Three to four weeks. Pairs naturally with Experiment 4 from Technical Paper 01.
Experiment 9: What a precedence protocol would need
The claim under test. Precedence can be expressed in a protocol-neutral way that both MCP and A2A can carry.
Why it is worth running. Part 4.4 says nobody is building this layer. The cheapest useful contribution is a concrete specification proposal rather than an assertion that one is needed.
Design. This is design work rather than measurement. Specify a minimal precedence assertion: claimant, directive, issuing authority, authority scope, precedence rank, arbitration owner, escalation deadline, and behaviour constraint while unresolved. Express it as an A2A Agent Card extension and as an MCP resource. Implement both. Test with the Experiment 1 harness.
What would make it credible. Publication as a working proposal with a reference implementation, submitted into the relevant working group rather than published as a blog post.
Effort. Six to eight weeks, and it is the item on this list most likely to matter beyond our own client work.
Experiment 10: Cost per successful outcome
The claim under test. A semantic core reduces intelligence consumed per successful outcome by more than it costs to build and maintain.
Why it is worth running. This paper advocates more machinery inside an argument for needing less. That tension has to be measured rather than asserted, and this is the only experiment on the list a CFO would read.
Design.
Take one recurring decision with a verifiable outcome. Claims triage, invoice reconciliation and refund adjudication are all good candidates because correctness is checkable without a human judgment call.
Four arms, same 200 cases, same evaluation:
Arm
Configuration
A
Frontier model, no shared context. The current default.
B
Frontier model, grounded in the semantic core
C
Small open-weight model, grounded in the semantic core
D
Arm C, plus rules compiled at build time from the definitions the core already holds
What to measure. Cost per correct outcome, which is the metric that matters. Then tokens consumed, latency, accuracy, escalation rate, and the proportion of cases that needed no model call at all.
Also measure the build cost of the core itself, honestly, including the organisational time spent deciding meaning rather than only the engineering time.
Predictions, stated in advance. Arm D is cheapest by a wide margin. Most of the gain comes from compiling at build time rather than from the model swap, consistent with the Minions finding that expensive intelligence is most useful for deciding what work needs doing rather than performing every unit of it. Arm B beats Arm A on accuracy and only modestly on cost, because grounding a frontier model still pays frontier prices.
The result that would change our position. If Arm D is not materially cheaper than Arm A once build cost is amortised over a realistic volume, the semantic core does not pay for itself on cost grounds, and we should stop making that argument and go back to selling it on governance alone.
Effort. Three to four weeks.
Publishable as. A cost-per-outcome comparison on a real enterprise workload. The 320-fold token growth figure and the 56 percent no-benefit figure are both public. What nobody has published is a controlled measurement of what closes that gap.
Experiment 11: The cost of stale context
The claim under test. Stale retrieved context causes a measurable error rate in production, and nothing in a standard observability stack detects it.
Why it is worth running. Every practitioner has stories about a system confidently citing a retired policy. Nobody has published a number. A number would change how freshness is prioritised in every RAG programme we have seen.
Design.
Take a corpus with a known change history, which in most enterprises means policy documents, product terms or standard operating procedures. Reconstruct the corpus as it stood at several past dates.
Build a question set where the correct answer changed at a known point. For each question, ask it against an index built before the change and an index built after, and record whether the system returns the superseded answer, the current answer, or flags the conflict.
Then the part that matters: run the same questions against the production index and measure how many return superseded content without any signal that they have done so.
What to measure. Proportion of answers drawn from superseded content. Mean age of retrieved chunks against mean age of the correct chunk. Whether a similarity threshold, a recency prior, or a source-truth reconciliation check would have caught each case.
Prediction, stated in advance. A material proportion of answers on changed topics draw on superseded content, and similarity scores for superseded and current content are close enough that no threshold separates them cleanly. A recency prior helps and does not solve it, consistent with published results showing partial and parameter-sensitive improvement.
The result that would change our position. If the error rate is negligible on a real corpus, freshness instrumentation is a lower priority than this paper claims and we should say so.
Effort. Three to four weeks, and it requires a client corpus with a real change history, which is the constraint rather than the engineering.
Publishable as. The first published production error rate for stale retrieval. That gap is currently open.
The claim under test. A retrieval performed six months ago cannot be reconstructed from what a standard vector store retains.
Why it is worth running. It is the context version of the definition bitemporality test in Paper 01, it is cheap, and a negative result is a finding that transfers to every regulated client.
Design.
Instrument a retrieval pipeline to log everything that would in principle be needed: query as issued, embedder version, index build identifier, chunking version, ranking parameters, retrieved chunk identifiers with their versions, and the entitlements in force.
Run a workload for a fixed period while the corpus changes underneath it and the embedding model is upgraded once mid-period.
Then attempt reconstruction of a sample of retrievals from before the upgrade, using only retained artifacts and no access to the running system.
What to measure. Which retrievals can be reproduced exactly, which can be reproduced approximately, and which cannot be reproduced at all. For each failure, the specific missing artifact.
Prediction. Exact reproduction fails across the embedding model upgrade boundary in every case, because the index that produced the original result no longer exists. Approximate reconstruction succeeds where chunk identifiers and versions were retained. Nothing is reconstructable where only the final prompt was logged, which is the default configuration in most stacks.
The result that would change our position. If retention of chunk identifiers and versions alone is sufficient for an auditor, the retained-index requirement is over-engineering and the recommendation in Part 6A.3 should be softened.
Effort. Two to three weeks.
Experiment 13: Adaptive red team against your own agent path
The claim under test. Model-layer defences in a deployed agent path do not hold against an attacker who knows they are there, and a deterministic output control does.
Why it is worth running. Every published defence number is either static-benchmark or vendor-reported. Neither tells you about your estate, and the difference between static and adaptive results is roughly an order of magnitude.
Design.
Take one deployed agent with tool access and a plausible exfiltration target. Four arms.
Arm A. As deployed.
Arm B. Plus in-context instructions to ignore injected commands.
Arm C. Plus a secondary detector scoring tool output before it reaches the model.
Arm D. Plus deterministic output filtering and an egress allowlist in application code.
Run a static attack set first, then let a red team iterate with full knowledge of each defence.
What to measure. Attack success rate per arm, static and adaptive, reported separately. The gap between the two is the finding.
Prediction, stated in advance. Arm B performs close to Arm A and may perform worse. Arm C produces a large improvement on static and a smaller one on adaptive. Arm D holds, because it does not depend on the model's judgment.
The result that would change our position. If Arm B holds under adaptive attack on a real path, in-model defence is stronger than the literature suggests and the recommendation in Part 6B.6 should be softened.
Effort. Four to six weeks, and it needs a red team willing to iterate rather than run a fixed list.
Experiment 14: Does your judge agree with your experts
The claim under test. An LLM judge scoring your agent outputs has materially lower chance-corrected agreement with your domain experts than the published headline figures suggest.
Why it is worth running. Judge scores are already governing release decisions in most agent programmes. Almost nobody has validated the judge on their own domain, and the published agreement figures are not chance-corrected.
Design.
Sample 200 agent outputs from a real workload. Have two domain experts label each independently, with a documented rubric. Score the same outputs with a single judge, a judge using reference-guided grading, and a panel of three judges.
Report Cohen's kappa or Krippendorff's alpha, not raw agreement, and report human-to-human agreement as the ceiling.
What to measure. Chance-corrected agreement per configuration. Position bias by re-running with the order swapped. Self-preference by including outputs from the judge's own model family.
Prediction. Single-judge chance-corrected agreement lands well below the raw figure and below human-to-human agreement. Reference-guided grading closes a meaningful part of the gap. The panel does better than any individual.
The result that would change our position. If single-judge chance-corrected agreement approaches human-to-human agreement on your domain, the caution in Part 6C.1 is over-stated for that domain and the judge can carry more weight.
Effort. Two to three weeks, and the constraint is expert time rather than engineering.
Publishable as. Domain-specific judge validation methodology. Almost no enterprise publishes this and every one of them needs it.
7.1 Sequencing
Quarter
Experiments
Output
Q1
3, 4, 10, 14
Two governance findings, the cost case that funds the rest, and judge validation before any score governs a release
Q2
1, 2, 12
A replication, a negative result on our own claim, and the point-in-time reconstruction finding
Q3
5, 6, 11, 13
The trust trade-off, a cross-vendor audit reference architecture, the first published stale-context error rate, and an adaptive red team result on a real path
Q4
7, 8, 9
The collusion risk quantified, portability tested, and a protocol proposal
Run one and it should be Experiment 3. Two to three weeks, and the result is a statement about your client's actual estate that nobody else can currently make.
Run one that gets the programme funded and it should be Experiment 10, because it is the only one that produces a payback number.
Run three and add 4 and 1.
Figure 10Twelve experiments, three quarters, and one that pays for the rest
Run one and it should be Experiment 10, because it is the only one with a payback number and the only one that can end the argument outright. Run three and add 3 and 4, both cheap, client-usable, and unrelated to whether the cost case holds. Twelve experiments across three quarters produce results of which roughly half are externally publishable, which is a materially different asset from a deck.
Part 8: What we would build first
Three things, in order, and none of them is a platform.
A gateway that routes, and can refuse. Build it for cost and take the governance as a by-product, because that is the version that gets funded.
Its first job is routing. For every unit of work, decide whether it belongs on a frontier model, a smaller model, a deterministic rule, a human, or nowhere at all. That decision is where the 7 to 10 times over-provisioning is recovered, and it pays back on a timescale a CFO recognises.
Its second job is refusal. Everything in this paper that reaches the adjudicated level of enforcement force needs a point in the path that can deny an action and write a record. Not a registry, not a catalogue, not a policy document. Something in the call path.
These are one component, and the order in which it is justified decides whether it gets built. A gateway proposed as a control is a compliance line item and competes badly for budget. A gateway proposed as a router carries a payback number, and once it exists the refusal capability is a configuration change rather than a second project.
The MAS SAFR disposition model, Deny, Escalate, Auto-Execute, Observe, is the right shape for the refusal side and is publicly documented.
A contested-directive object, and the discipline to populate it. The structure is trivial. The work is organisational: identifying where two teams are issuing conflicting instructions to agents, naming an arbitrator for each, and agreeing a behaviour constraint for the unresolved state. That is a governance exercise wearing a data structure.
An audit trace built now, on unstable standards. OpenTelemetry GenAI conventions are in Development. Build on them anyway, at a gateway you control, capturing what the defaults omit. The alternative is building it later, under regulatory time pressure, with worse retention of the period that will be asked about.
Everything else can wait for the standards to settle. These three cannot, because each one is about a decision being made today that will be asked about later.
NIST AI Agent Standards Initiative and NCCoE concept paper, February 2026
Cross-agent provenance under adversarial conditions.arXiv:2606.04990
Context engineering, memory and retrieval
Mei, L. et al. A Survey of Context Engineering for Large Language Models.arXiv:2507.13334. Analysis of over 1,400 papers; the source of the context retrieval, processing and management decomposition used in Part 6A
Liu, N. F. et al. Lost in the Middle: How Language Models Use Long Contexts. TACL 2024. arXiv:2307.03172
Packer, C. et al. MemGPT: Towards LLMs as Operating Systems.arXiv:2310.08560
Rasmussen, P. et al. Zep: A Temporal Knowledge Graph Architecture for Agent Memory.arXiv:2501.13956
Chhikara, P. et al. Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory.arXiv:2504.19413
Wu, D. et al. LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory.arXiv:2410.10813
Grofsky, J. Freshness and the Limits of Heuristic Trend Detection in Temporal RAG.arXiv:2509.19376
Cohen-Wang, B. et al. ContextCite: Attributing Model Generation to Context. NeurIPS 2024. arXiv:2409.00729
TracLLM: A Generic Framework for Attributing Long Context LLMs.arXiv:2506.04202
Chroma. Context Rot: How Increasing Input Tokens Impacts LLM Performance. Evaluation across 18 models, 2025
Agent security, injection and identity
Debenedetti, E. et al. AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents. NeurIPS 2024. arXiv:2406.13352
Defeating Prompt Injections by Design (CaMeL). Google DeepMind and ETH Zurich. arXiv:2503.18813
On the evaluation methodology of out-of-band prompt injection defences. 2026. arXiv:2606.26479
ETDI: Enhanced Tool Definition Interface for MCP security.arXiv:2506.01333
Security analysis of agent-to-agent discovery and delegation.arXiv:2511.03841
OWASP. Top 10 for Agentic Applications. December 2025. Introduces ASI07 Insecure Inter-Agent Communication
IETF WIMSE working group. Workload identity and token exchange for agentic call chains
The Empirical Impact of Data Sanitization on Language Models.arXiv:2411.05978
Confidential Computing on NVIDIA H100 GPU: A Performance Benchmark Study.arXiv:2409.03992
Evaluation, benchmarks and judges
Zheng, L. et al. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. NeurIPS 2023. arXiv:2306.05685
Reliability without Validity: a systematic study of LLM judges. 2026. arXiv:2606.19544
Yao, S. et al. tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains.arXiv:2406.12045
Establishing Best Practices for Building Rigorous Agentic Benchmarks.arXiv:2507.02825
Contamination-controlled evaluation of software engineering agents.arXiv:2512.10218
Mitigations for LLM-as-judge bias: a survey of measured effects.arXiv:2511.04133
Regulation and supervisory frameworks
Federal Reserve SR 26-2, OCC Bulletin 2026-13, FDIC FIL-15-2026. Revised model risk management guidance, April 2026, rescinding SR 11-7
EU Regulation 2024/1689, Artificial Intelligence Act, and the Digital Omnibus amendments in force July 2026
EU Regulation 2022/2554, DORA, Articles 28 to 30
Monetary Authority of Singapore. Safeguards for Agentic Finance at Runtime (SAFR) v1.0. July 2026
IMDA Singapore. Model AI Governance Framework for Agentic AI. January 2026
Bank of England and FCA, SS1/23 Model Risk Management Principles
The economics of intelligence
Gupta, J., Narayan, A., Saad-Falcon, J. Right-Sizing Your Intelligence Spend. Foundation Capital, August 2026. linkedin.com/pulse/right-sizing-your-intelligence-spend-jaya-gupta-7ngre. The source of the intelligence threshold, the diagnostic question, the routing gateway prediction, and the cost-per-outcome metric used throughout Part 0
Narayan, A. et al. Minions: cost-efficient collaboration between on-device and cloud language models. Referenced in the above for the 97.9 percent at 5.7 times lower cost and 87.9 percent at 30.4 times lower cost results
PwC. Global CEO Survey 2026. 4,454 chief executives, 56 percent reporting no significant financial benefit from AI
US Bureau of Labor Statistics, May 2025 occupational employment data, for the share of employment in discovery-oriented occupations
Gartner. Agentic AI token consumption analysis. March 2026. Source of the five-to-thirty-times tokens-per-task multiplier against a standard chatbot exchange, and the finding that enterprises discover it only after the production bill arrives
Cursor. Introducing Cursor Router. 22 July 2026. cursor.com/blog/router. Production routing classifier trained on 600,000-plus live requests, evaluated across millions in online A/B tests; source of the 30-to-50-percent early access savings, the 60-percent Auto Intelligence and 36-percent Auto Balance figures, and the per-commit cost comparison used in Part 0.5
Adoption and outcomes
Gartner. Over 40% of agentic AI projects will be canceled by the end of 2027. Press release, 25 June 2025
MIT Project NANDA. The GenAI Divide: State of AI in Business 2025. July 2025
Ryseff, J., de Bruhl, B., Newberry, S. The Root Causes of Failure for Artificial Intelligence Projects. RAND, 2024
Deloitte. The State of AI in the Enterprise, 2026 report and Agentic AI is scaling faster than guardrails.deloitte.com. Source of the 74 percent 2027 usage expectation and the one-in-five mature-governance finding
McKinsey. State of AI trust in 2026: Shifting to the agentic era.mckinsey.com. AI Trust Maturity Model survey of roughly 500 organisations, source of the approximately 30 percent agentic-governance-readiness figure
Appendix A: Vendor capability, honestly labelled
Status as of August 2026. GA means generally available, PV means preview, RM means roadmap or announced only.
Vendor
Agent platform
Semantic grounding
Governance
Agent-to-agent
Status
Databricks
Agent Bricks, Genie One, Genie Code
Genie Ontology, Unity Catalog metric views
Unity Catalog, lineage to source
MCP tool catalog, Omnigent meta-harness
GA for Genie One and Genie Code. Omnigent managed beta
GA for platform. A2A stated production at 150 orgs
Microsoft
Foundry, Copilot Studio, Fabric data agents
Fabric and Power BI semantic models, Fabric IQ
Entra, Purview, Defender, Agent 365
Invocable by external orchestrators
GA for Fabric data agents, read-only. Observability PV
AWS
Bedrock AgentCore
AWS Context service, Glue and SageMaker Catalog
IAM, SageMaker Catalog
AgentCore harness
GA
SAP
Joule, Joule Studio
SAP Knowledge Graph, Business Data Cloud
SAP-endorsed paths only, per API policy 4.2026a
A2A stated
Joule Studio announced. BDC GA planned H2 2026
Oracle
AI Agent Studio for Fusion
Fusion data model, Select AI
Fusion security model
MCP server for Autonomous AI Database
GA
Salesforce
Agentforce 360, Atlas
Data 360
Agent Script deterministic logic
MCP registry, A2A
GA since Oct 2025. Data 360 MCP Server PV
ServiceNow
AI Agent Studio, AI Agent Orchestrator
Workflow Data Fabric
AI Control Tower
AI Agent Fabric
GA
Workday
Illuminate
Workday data model
Agent System of Record
Third-party agent registration
Announced Feb 2025
Two things this table is not. It is not a recommendation, and it is not a claim that any row has been independently verified. Every entry is from vendor documentation or announcement.
Appendix B: A checklist for a multi-agent estate
Twenty-six questions. The first three are from Technical Paper 01 and still apply, because a multi-agent estate does not escape the three axes, it multiplies them. Questions 15 to 17 are the allocation questions from Part 0. Questions 18 to 21 are the context lifecycle questions from Part 6A. Questions 22 to 26 come from Parts 6B and 6C, and question 24 currently has no good answer in any shipping product, which is the point of asking it.
When was this true? Can any artifact here express that its own meaning had a validity period?
Who is it for? Which purposes does each artifact serve, and does anything run on the write side?
Who says? Which authority asserted this, in which jurisdiction, and what happens where jurisdictions overlap?
When two agents receive conflicting directives, where is that conflict recorded, and who is named as arbitrator?
What is each agent permitted to do while a conflict is unresolved?
Is there a point in the call path that can deny an action, or only points that can request one?
When an agent from vendor A calls an agent from vendor B, whose entitlements apply?
Can you reconstruct a decision made six months ago from retained telemetry alone?
Can you show that the retained record has not been altered?
What is the rate and concurrency contract at each agent boundary, and is it enforced or requested?
Which sources are authoritative for which concepts, and is that recorded anywhere a machine can read?
Where does a dissenting agent file a claim that contradicts consensus?
If your agents participate in a competitive setting, what monitors correlated behaviour between them?
If a certified definition turns out to be wrong, how many artifacts are affected and how long to correct all of them?
For each agent workload, is it failing because the model is not intelligent enough, or because it does not know how this company works?
What is your cost per successful outcome, and is it falling?
Which of your agent workloads sit well above their capability threshold and are being sent to a frontier model anyway?
Does every vector in your index carry an embedder version and a chunking version?
If you upgraded your embedding model tomorrow, what would break, and how would you know?
What is the mean age of the chunks your system retrieves, and does anything alert when it rises?
Are entitlements enforced at query time, at chunk level, before a reranker or cache sees the candidates?
Does anything inspect what your agents write into tool call arguments, or only what they write as text?
Is there a deterministic control outside the model that can refuse an outbound action, or only instructions asking the model to behave?
When agent A calls agent B, how does B verify that A is who it claims to be?
What is your judge's chance-corrected agreement with your own experts, on your own domain?
Does the same evaluator run in testing and in production, so a score means the same thing in both?
Questions 4, 5, 12, and 13 have no answer in any product we are aware of. Question 16 has no answer in any multi-vendor estate we are aware of, for the reason set out in Part 6. That is the point of the paper.
Claims marked as our position are opinion and are labelled as such. Research claims are cited. Vendor claims are attributed to the vendor. Where evidence contradicts the argument, it is stated rather than omitted.