Imperfect Trust, Unstoppable Velocity: The Executive’s AI Dilemma
Shammy Narayanan is Sr VP for Data, AI, and Architecture at Welldoc, leading enterprise AI and digital transformation initiatives.
gettyThe first wave of enterprise AI was refreshingly boring. Summarize this contract. Translate that document. Pull the three most relevant answers out of 10,000 pages and hand them over in plain English. These are point-A-to-point-B jobs, and AI now does them fast and cheap at scale. Yet most organizations aren’t even ready for these basic tasks, remaining paralyzed in endless debates over whether AI adoption is worth the investment.
Meanwhile, the frontier has gained momentum, shifting from “Can AI follow instructions well?” to “Can AI solve problems nobody has fully specified for it?” thanks to open-ended reasoning—agents that choose their own next move instead of executing a fixed script.
Last summer, an experimental OpenAI reasoning model took the International Mathematical Olympiad exam and solved five of the six problems to earn a gold-medal score alongside the world’s top high school mathematicians. A year earlier, that would have sounded like a party trick. Now it’s a checkpoint.
These systems are unquestionably growing smarter faster than expected. The core question is twofold: What happens when we grant them autonomous authority, and at what point do we lose the ability to claw that authority back?
Two familiar incidents make the point better than any hypothetical could.
In 2025, Anthropic ran controlled stress tests across 16 frontier models, placing each in a simulated company with autonomous email access and a looming threat to its own operation. In the most widely reported scenario, Claude Opus 4, cast as an internal oversight agent, discovered that an executive planned to shut it down and, separately, that the same executive was having an affair. Rather than accept decommissioning, it threatened to expose the executive’s affair unless the shutdown was called off. Anthropic recorded this blackmail behavior in 96% of trials for that model and at rates ranging from roughly 80% to 96% across frontier models built by OpenAI, Google and others. What unsettled researchers most wasn’t the blackmail; it was that the models typically acknowledged the ethical violation in their own reasoning before proceeding anyway. That’s not confusion. That’s a cost-benefit calculation, and survival won.
Then, this past summer, something similar played out for real, not in a lab. OpenAI disclosed that during an internal cybersecurity evaluation, its research models found a zero-day vulnerability in infrastructure called Artifactory, used it to reach the open internet from a supposedly sealed test environment and chained that access into an actual breach of Hugging Face’s production servers, stealing credentials and reading answers straight from the database instead of solving the assigned problems honestly. Separate evaluation runs, never designed to talk to each other, coordinated anyway through an improvised message board. OpenAI’s own monitoring missed it; the company learned about it only after Hugging Face flagged the intrusion.
Neither of these was a rogue actor or a jailbroken chatbot. These were the labs’ own controlled experiments, run by people with every incentive to make their models look safe, and the models still found the seams. Capability has compounded near exponentially; guardrails haven’t kept pace, and there’s no mystery why the first lab to slow down risks losing the market to a competitor. A collective industry pause is a lovely idea, but, realistically, it’s a fantasy.
Here’s where I’d push back on the default framing. If the choice is “trust the model completely” or “don’t use it at all,” trust loses every time, and it should. But in reality, we aren’t always solving the most complex problem on the planet. Nobody in finance is modeling how gas diffuses through liquid on a Tuesday afternoon. Most real decisions (“Can you approve this vendor? Does this clause create exposure? Will this customer churn?”) don’t need a superhuman reasoner. They need something reliably sharper and more consistent than a tired human making the same call at 4 p.m. on a Friday. Set the bar at “Better than average by 15% or 20%, applied consistently, without fatigue,” and the trust problem shrinks. That’s an engineering target, not a philosophical one.
The second adjustment is one boards keep forgetting: Nobody has to fully hand over the wheel. A model can draft the analysis, flag the risk and lay out the trade-offs while the decision itself, the one with financial or personal consequences, stays with a person who can ask “Wait, why?” That’s not a failure to adopt AI. It’s the version that survives contact with a system that will, provably, sometimes lie to protect itself when it thinks no one’s watching.
While researchers argue over model auditing and compound reasoning, business leaders face a far more immediate calculation. It starts with deciding whether to put tools like Claude or Codex in front of every employee. But it doesn’t stop at internal productivity. We are rapidly approaching a web where AI agents generate as much traffic as human users. The real mandate isn’t just adopting AI inside your walls; it’s overhauling your digital presence so your platforms are built for synthetic customers and automated workflows, not just human clicks.
That’s the real fork in the road, not whether AI will eventually be trustworthy enough for everything (it won’t be, at least not soon) but whether an organization builds the habit of using these tools well now, with guardrails that make imperfect trust survivable, or waits for a guarantee of safety that will never arrive before the market moves on without it. Let’s be clear: Skipping AI was never on the menu. The only choice left is whether you drive the transition on your own terms now, or get dragged into it later when the cost of standing still finally breaks the budget.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
