Why 'Done' Is Dead: How Probabilistic Systems Are Rewriting Enterprise Ways Of Working

Direct Source Verification: This story is aggregated from Forbes (forbes.com). Full reporting rights and copyright belong to the primary publisher.
gettyGautam Nadkarni, Chief Technologist, Distinguished Member of Technical Staff (DMTS) - Wipro Technologies.

gettyGautam Nadkarni, Chief Technologist, Distinguished Member of Technical Staff (DMTS) - Wipro Technologies.

​An insurance client called me this year with a problem that fit no playbook they had. Their customer-service agent passed the preproduction evaluation at 94% accuracy and went live, yet within three weeks, complaint volumes climbed, even though nobody had touched the code. What had changed was the mix of customer queries after renewal season, and on that new mix, the agent had slipped to 81%.

The team’s first instinct was to log it as a defect, but nothing was defective. The agent was behaving exactly as a probabilistic system behaves, and although it had been declared done, the real work had only just begun.

Most enterprises I’ve worked with this year have faced a problem like this one. The AI we’re putting into production operates on a different principle from the software our delivery models assume, and that difference is undoing how we plan, ship, pay for and own outcomes.

A deterministic system gives the same output for the same input every time. You write a specification, build to it, test against it and ship, and at the end, the work is observably done. That idea of “done” sits underneath sprint planning, RACI matrices and vendor statements of work.

A probabilistic system offers no such certainty. The same input can produce different outputs, accuracy depends on context, data freshness and whichever queries arrive that week, and all of those drift. Performance degrades without anyone modifying the code, and when it does, nothing is broken—the system is working as designed.

KPMG’s June 2026 AI Quarterly Pulse Survey found that 53% of large U.S. firms have agents deployed, and the share chaining multiple agents across a workflow doubled in a quarter, from 9% to 18%. But only 26% of them can see what their AI systems cost to run, and Deloitte researchers report that just one organization in four has put at least 40% of its pilots into production. Agents are multiplying faster than anyone can see, cost or govern them.​

My read on why so many stall is that the models are usually fine, but the ways of working around them aren’t.

In a deterministic world, shipping is a finish line; in a probabilistic one, it’s where the clock starts. The agent that scored 94% in preproduction isn’t the same agent at week 12 or after the next upstream model upgrade because drift is the default rather than the exception.

The AI technologies are also adapting accordingly. Microsoft’s Build 2026 pitch says it plainly: “Their behavior shifts as models update, tools change, and traffic patterns evolve and most of that drift happens silently, long after the demo.” The tooling has arrived, but the operating model that would put it to use mostly hasn’t.

An agent launch should be treated as a handover from build to operate, not a milestone. Define “good” as something measured continuously, run nightly evaluations against curated input sets and watch for when the numbers move. One client summed it up: “We used to ship and forget. Now we ship and watch.”

Deterministic work can be scoped into stories and committed to a sprint. Probabilistic work resists that because you don’t know whether a use case will land at 95% accuracy or plateau at 70% until you’ve tested it on real data, and even then the results vary.

Cost has the same problem, and it bit hard this year. KPMG’s global survey found that 49% of executives had scaled back agent deployments because operating costs outran the benefits. That’s what happens when a budget built around a one-time build cost meets software that meters every run, indefinitely. “Done” is dead in finance too.

Leaders who have adapted ask for thresholds rather than point estimates. “What accuracy are we prepared to ship at, what does each run cost and what is the playbook when we drop below the line?” are better steering questions than “When will it be done?”

If “done” is dead, someone has to own what comes after it, which is why titles such as agent supervisor, agent QA lead and AIOps manager are appearing in enterprise org charts. Somebody needs to own the evaluation suite, be on call when accuracy wanes and decide when behavior has crossed from variation into incident.

1. Retire “done” from your AI vocabulary. Replace it with “in operation, against these thresholds, at this run cost, with this owner.” If your steering decks still treat AI delivery as a project plan, you’re approaching it wrong.

2. Fund the operating layer, not just the build. Evaluation, observability, cost telemetry and on-call rotations separate a pilot from a production capability. Most stalled pilots I see failed not on model quality but because nobody funded the operating layer.

3. Name the owner. Every AI system in production needs one accountable human rather than a committee or a vendor SLA. Until ownership is named, drift is everyone’s problem, yet nobody takes responsibility.

My insurance client didn’t solve their problem with a better model. They hired an agent operations lead, built a nightly evaluation pipeline and redefined “live” to include continuous monitoring of accuracy and cost. The agent is still probabilistic, but the operating model has stopped pretending otherwise, and that, more than any model upgrade, is what production-grade AI looks like.

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?

Original Source
https://www.forbes.com/councils/forbestechcouncil/2026/11/20/why-done-is-dead-how-probabilistic-systems-are-rewriting-enterprise-ways-of-working/
Visit Forbes ↗
SHARE STORY:
𝕏 f in

Related Coverage in Technology