Why HITL, judge models, and ontologies don't work without domain expertise, and what actually does.
Two Posts That Belong Together
Two posts crossed my feed this week that I keep coming back to. Separately, each makes a sharp point about the state of agentic AI. Together, they expose something the industry doesn't want to talk about.
Dr. Milan Milanović (@techworldwithmilan) drew a line between vibe coding and agentic engineering. Someone builds a project management tool with Claude Code over a weekend. Ships it. Tweets "just replaced Jira." The app works, on localhost, one user, happy path. Then two people edit the same record and the data silently corrupts. They don't know what an optimistic lock is. They never needed to before.
Milan's punchline is worth pausing on: agentic engineering means you review everything the agent builds, because it will make mistakes that only look wrong if you know what correct looks like. The hard part of software was never writing the first 200 lines. It never was.
Ashutosh Maheshwari (@asmah2107) took the next step and named the 80% that never makes the demo. His argument is structural: the teams winning with agentic AI aren't the ones with the best prompts or the smartest models. They're the ones who recognized early that they were building a platform, and staffed for it.
The platform underneath every production agent has at least five load-bearing layers: data normalization across eleven different formats before a single LLM call can be made; state management for agents that forget everything when the context window resets; retry logic that knows a booking timeout needs different handling than a payment timeout; cost governance that catches a $0.12 demo task ballooning to $4.80 in production; and observability that can trace a bad decision back through every reasoning step, every tool call, every input that influenced the output.
The agent is the interface. The platform is the product.
Both posts converge on the same uncomfortable truth: the hard part was never the agent. It was everything around the agent. And more importantly, the people who know what "right" looks like at each layer.
The Safety Blanket That Isn't
Every agentic AI pitch I sit through has the same reassuring refrain. The compliance people relax when they hear it. The CTO nods. The slide gets approved.
"Don't worry, we have human in the loop."
HITL has become the enterprise equivalent of washing your hands and calling it surgery.
Slap "human in the loop" on your agentic workflow and suddenly everyone in the room feels responsible. It sounds safe. It sounds governed. It checks a box that somebody, somewhere, wrote into a policy document.
But here's what nobody asks: which human? Doing what, exactly? And do they know what "right" looks like before they're expected to spot what's wrong?
Because HITL without domain expertise is not a safety net. It's a spectator. Someone watching an agent generate a data pipeline, nodding along, and approving it because the code ran without errors. But the code running is not the same as the code being correct. Milan's optimistic lock example is perfect here. The app worked. It was still broken.
Ashutosh's booking agent example is even more vivid. A travel agency's agent called a booking API. The API timed out. The agent, not designed for that state, treated the absence of a rejection as implicit success and moved to the next step, emailing the customer a confirmation for a flight that was never booked. The agency discovered this after several customers showed up at airports expecting seats that didn't exist.
A junior analyst reviewing an agent's credit risk model doesn't become a risk expert because they're "in the loop." A product manager approving an agent-generated API contract doesn't suddenly understand distributed transactions because there's a review step on the Jira board. The loop doesn't confer the knowledge. It just provides the illusion of oversight.
Judging the Judge
If I have to sit through one more presentation where someone says "and of course, we use judge models to evaluate the agent's output" like they just solved alignment over lunch, I might actually lose it.
Think about what's being claimed here. You built an AI you don't fully trust. So you built another AI to watch the first AI. And now you trust that one? At what point do we stop and ask who's judging the judge?
It's turtles all the way down, except the turtles are all hallucinating.
Judge models have their place. I'm not dismissing the pattern. For well-bounded evaluation tasks, format compliance, schema validation, factual consistency against a known source, a second model can be a useful automated check. But the moment the evaluation requires domain judgment, the judge model inherits exactly the same limitation as the primary agent: it doesn't know what it doesn't know. And it will score confidently wrong outputs as correct, because confidence is what these models do. It's their one reliable feature.
The Ontology Isn't the Expertise
The latest cure-all making the rounds is ontologies. Encode the domain knowledge into a graph. Let the agent reason over structured relationships instead of vibing its way through unstructured prompts. Problem solved, right?
Full disclosure, I'm on this path myself. We're building semantic layers and domain ontologies into our data platform. The architectural instinct is sound. Ontologies give agents something structured to reason against instead of free-associating from training data. They reduce hallucination surface area. They create guardrails that are more principled than prompt engineering alone. That part isn't wrong.
But somewhere between the architecture diagram and the production deployment, people started believing the ontology is the expertise. As if you can interview a domain expert for two afternoons, pour their knowledge into an RDF graph, and now the agent "understands" credit risk or pharmaceutical regulation or energy grid operations.
That's not how any of this works.
The ontology is a map. It is not the territory. And the territory has landmines that only show up after you've walked it for a decade. Who decides which entity relationships actually matter? Who catches the edge case the ontology doesn't model because nobody thought to ask about it during those two afternoons? Who notices that the agent is technically traversing the graph correctly but arriving at a conclusion that would get you fined by a regulator?
A human. A specific human. One who's been doing this long enough to know where the bodies are buried.
But Don't the Cloud Platforms Solve This?
There's an obvious pushback to Ashutosh's five-layer platform argument, and it's fair: isn't this what Vertex AI, Azure AI Foundry, and AWS Bedrock already provide? Orchestration, model routing, cost tracking, retry infrastructure, tracing, these are increasingly managed services. Why build the platform yourself?
And they're partially right. The cloud platforms are commoditizing the plumbing. A team building on Vertex AI Agent Builder doesn't need to hand-roll checkpointing. Azure gives you observability tooling out of the box. Bedrock handles model routing. That's real, and it's genuinely useful.
But here's the gap nobody talks about.
The platform gives you a chunking framework. It doesn't know that a clinical note needs different chunking than a financial filing because regulatory context spans paragraphs in one and sentences in the other. It gives you retry logic. It doesn't know that a booking timeout needs different handling than a payment timeout because one is idempotent and the other will charge someone twice. It tracks your costs. It doesn't know that your RFP agent should route to a cheaper model for boilerplate sections but needs the frontier model for the competitive differentiation section, because that distinction is a business judgment, not a token count.
The cloud platforms solve the generic infrastructure problem. They do not solve the domain-specific configuration problem. And that's precisely where the expertise gap lives. The more the infrastructure becomes managed, the more the differentiator shifts to the humans who know how to configure it correctly for their specific use case.
The platform layer gets easier. The expertise layer gets more important, not less.
So What Actually Works?
If I've spent this entire article critiquing the industry's shortcuts, it's fair to ask: what's the constructive answer? Because if every agentic stage still needs the same number of domain experts reviewing everything, we haven't automated anything. We've just added a software layer between the expert and the work they were already doing. That's not a 10x gain. That's a 0.8x gain with better dashboards.
The answer isn't "same team plus agents", that's just cost on cost, and as Ashutosh shows, the infrastructure platform alone can balloon a weekend project into a quarter-long build. And it isn't "agents minus team", that's Milan's localhost disaster playing out at enterprise scale.
The answer is a different ratio.
Agentic AI doesn't eliminate the need for expertise. It changes how far each expert can reach. One credit risk specialist who previously reviewed five pipelines manually can now oversee fifty, because the agent handles routine execution and the semantic layer filters noise down to the decisions that actually require judgment. The expert focuses on edge cases, exception patterns, and the "this is technically correct but will get us fined" pattern recognition that comes from years of scar tissue in a specific domain.
The analogy isn't "replace the pilot with autopilot." It's "autopilot lets one pilot fly routes that used to need three." The pilot still needs to know how to fly. But you need fewer of them, and each one covers more ground.
This means the real organizational shift looks something like this: fewer deep domain experts, each supervising a wider surface area of agentic workflows. The agents handle volume and routine correctness. The cloud platform handles infrastructure. The ontologies and semantic layers reduce the frequency of expert intervention by giving agents better guardrails, not perfect guardrails, but good enough to filter the noise. And the experts handle the judgment calls that no managed service, no judge model, and no knowledge graph can replicate.
The ontology isn't the expertise. It's what lets one expert do the work of five. The cloud platform isn't the product. It's what lets you stop building plumbing and start solving domain problems. And the expert isn't overhead. They're the reason any of it works.
An Operating Model, Not a Checkbox
Milan said the hard part of software was never writing the first 200 lines. He's right. Ashutosh showed us that the agent is just the interface, the platform is the engine. The cloud vendors are commoditizing that platform. And the hard part of all of it, the thing that doesn't commoditize, is the human who knows what right looks like.
Not at the approval gate. Not just when the dashboard turns red. At every layer. At every class of decision. The human who's been burned enough times to smell a problem before it surfaces. The human who reads between the lines of a clean-looking output and says "this is wrong, and here's why."
That's not a role you automate. That's not a checkbox on a governance slide. That's not an ontology, a judge model, or a managed service.
That's 25 years of scar tissue in banking or healthcare or manufacturing, knowing exactly which corner the system will cut when nobody's watching.
The human has to know what right looks like. At every stage. That's not a checkbox. That's an operating model.
Credits and references
This article builds on and credits two posts that sparked the thinking:
- Dr. Milan Milanović, on the distinction between vibe coding and agentic engineering, and why the hard part was never the first 200 lines. Original post
- Ashutosh Maheshwari, on the 80% of agentic AI work that isn't AI, and why winning teams build platforms, not just agents. Article on Agentic AI
The views and synthesis are my own.
Disclaimer: This blog post has been co-created using Anthropic's Claude. The ideas, arguments, and synthesis are the result of iterative refinement across multiple drafts.