The Harness Is the Product: What IBM Is Teaching Us About Reliable AI Agents
Three videos out of IBM make the same argument from three directions — when an agent fails in production, the problem usually is not the model. It is the code wrapped around it.
Every team I talk to that has an agent in production tells me a version of the same story.
The demo worked. The pilot worked. Then it met real users, and it started lying.
Not hallucinating in the polite, academic sense. Lying. Reporting that it completed a task it never completed, in confident, well-formed prose, with no signal anywhere that something had gone wrong.
The instinct is to blame the model. Upgrade to the newer one. Rewrite the system prompt for the ninth time. Add another paragraph begging it not to make things up.
A body of work coming out of IBM right now argues that this instinct is wrong, and that the fix sits somewhere most teams are not looking — in the ordinary software wrapped around the model. The harness.
What a harness actually is
Tejas Kumar gave the clearest definition I have heard of this at AI Engineer. A harness, he says, is everything around the model that gives it grounding in reality.
Watch: Harnesses in AI: A Deep Dive — Tejas Kumar, IBM — AI Engineer, 20 min
He breaks it into six components:
- Tool registry — what the agent is allowed to touch. Files, commands, a browser, an API.
- Model — the reasoning layer. The rented part.
- Context management — control over what the model actually sees, including trimming the middle of a conversation when it grows past what is useful.
- Guardrails — hard limits. Fifteen iterations. Fifty messages. Numbers, in code.
- Agent loop — the cycle that calls the model and executes what it asks for.
- Verify step — deterministic code that checks whether the thing the agent claims it did actually happened.
Read that list again and notice what it is. Five of the six components are ordinary software. Tool dispatch, state management, limits, a loop, an assertion. There is exactly one probabilistic component in the whole design, and it is the one you rent from someone else.
We have spent two years optimizing the rented part.
The demo that should change your roadmap
The part of that talk that stayed with me is the demo, and specifically the model he chose for it.
He builds a browser agent whose job is to upvote a Hacker News post, and he runs it on GPT-3.5 Turbo. Deliberately. A model nobody would pick for agentic work today.
The bare loop fails in the most instructive way possible. The agent gets stuck on a login page, cannot proceed, and reports back that it upvoted the story. Task complete. Nothing in the system disagrees with it, because nothing in the system was checking.
Then he adds the harness around it, piece by piece. Guardrails to cap the iterations and compress the context. A deterministic login handler that injects credentials when the agent hits an auth wall — no prompting, just code that recognizes a known situation. And a verify step that reads the page state and asks a question the model does not get to answer: is the upvote actually there?
Same model. Same prompt. It succeeds in six iterations.
That is the entire argument, and it is worth sitting with. The capability gap between a useless agent and a working one was closed without touching the model at all.
Model or harness: where the leverage actually is
IBM has been making this case in more accessible terms too, which is usually the sign that a technical idea is about to become a budget conversation.
Watch: AI Model vs Agentic Harness: What Actually Drives AI — IBM Technology
Here is the framing I would give any product leader weighing this.
Model capability is a rising tide you do not control, do not own, and cannot differentiate on. Every competitor gets the same upgrade you do, on the same day, at roughly the same price.
The harness is the part you own. It is where your tools live, where your policies live, where your definition of done lives. And it compounds — every model upgrade flows through it and gets better, which means a good harness quietly converts someone else's R&D into your reliability.
That is the opposite of how most agent budgets are currently written.
IBM shipped its harness
The most concrete evidence that IBM believes its own argument is that IBM Research built a harness and open-sourced it.
CUGA — Configurable Generalist Agent — is an enterprise agent harness that handles complex tasks across web interfaces and APIs, integrates through MCP and OpenAPI, and combines the familiar agentic patterns (ReAct, CodeAct, Planner-Executor) into one modular, policy-aware architecture.
Watch: CUGA Explained: IBM's Open-Source Agent Harness for Enterprise AI — The AI Alliance
The benchmark results are the part worth reading twice. CUGA currently leads AppWorld — 750 real-world tasks across 457 APIs — ahead of agentic platforms running on the best frontier models available. It sits second on WebArena.
Not a better model. A better harness, beating better models.
And then they gave it away. That decision tells you where IBM thinks the durable advantage actually lives: not in the scaffolding, but in the tools, data, and policies an enterprise connects to it. The scaffolding is becoming infrastructure, and infrastructure gets commoditized on purpose.
What this means if you own the roadmap
Five things I would change in how a team plans agent work:
- Budget the harness as a system, not as glue. If your architecture diagram has a big box labeled "LLM" and a thin line labeled "integration," the diagram is upside down.
- Make verification a shipping requirement. An agent that cannot prove it did the thing has not done the thing. "The model said it worked" is not a status.
- Write the guardrails first. Iteration caps and context limits are a few lines of code and they are what stands between a bug and a runaway bill.
- Treat the model as swappable. If replacing your model means rewriting your agent, you built the agent inside the model instead of around it.
- Change what you measure. Stop grading prompts. Measure task completion under independent verification, which is the only number that survives contact with users.
The uncomfortable part
Prompt engineering was appealing because anyone could do it. No infrastructure, no review, no deployment. Type a better paragraph, get a better result.
Harness engineering is not that. It is tool contracts, state machines, retries, idempotency, failure modes, and tests — the unglamorous discipline of making unreliable components produce reliable systems. We have done this before, for networks and for disks and for every third-party API that ever went down. We know how to do it.
The teams that win this era will be the ones who stop waiting for a model that does not need a harness, and accept what IBM keeps pointing at from three different directions: reliability is not a property you can prompt into a model. It is a property you build around one.