Why Physical AI Works In A Demo And Struggles In Your Business
Maria Zhang is the CEO and Co-Founder of Palona.
gettyPhysical AI must understand the operation and keep that understanding accurate as conditions change.
A physical AI demo can make a difficult problem look solved. But it establishes what the system can do under the conditions shown. A business needs that capability to hold as workloads, layouts and routines change.
The gap between AI adoption and business value is familiar. McKinsey’s 2026 survey (login required) found nearly nine in 10 respondents reported AI use in at least one business function within their organizations. Only about 6% reported both significant impact and at least 5% of earnings before interest and taxes attributable to AI.
In my previous article, I explored a coding agent for physical businesses. Delivering that promise requires a reliable, continuously updated understanding of the operation.
In many knowledge-intensive applications, a foundation model needs specialized knowledge: a law firm’s precedents, internal policies or proprietary history.
Much of that context already exists in documents or structured records. Retrieval supplies relevant material, fine-tuning adapts models to specialized tasks and workflow integration connects them to the business.
Physical businesses need these capabilities, too. They also contain important information that has never been recorded.
An order system may record when work began and ended without showing where an employee waited or which handoff stalled. A language model cannot retrieve events the system never captured.
Physical AI must observe that activity and interpret its business meaning, drawing on knowledge of the operation. This is part of perception.
Consider measuring how quickly a restaurant line moves. Detecting people does not distinguish someone waiting to order from someone collecting food or waiting for a friend.
An experienced operator reads relationships and context. The AI must understand what people are doing, how interactions change and when the evidence is incomplete.
A staffing recommendation can be logically consistent with the reported queue while responding to a queue that does not exist. Strong reasoning can still produce the wrong decision from a mistaken account of the operation.
Vision-language-action models, or VLAs, connect visual observations and language instructions to actions. World models learn how environments evolve to support prediction and planning. Simulation and digital twins provide virtual environments for training and testing; their usefulness depends on fidelity to relevant conditions.
These capabilities can support different architectures. An end-to-end system that observes, plans and acts may be appropriate for some applications.
A modular approach connects foundation models, domain knowledge, business rules and workflow systems through explicit interfaces.
I believe a useful boundary is the translation from physical observations into meaningful operational states.
The system might estimate that a customer is waiting, an order is ready or a task is blocked. Recorded events, such as an observed handoff, help track state changes.
For business decisions, this is more useful than object detections or generic captions.
Much of the business problem can then be expressed symbolically: as people, objects, events, relationships and rules. Language models, planners and workflow systems can use these representations to evaluate alternatives and coordinate action.
The representation must preserve timing, evidence and uncertainty. A robot may still require detailed geometry; an order-status decision can often use a more abstract description.
This boundary makes the system more modular. Changing an acceptable wait-time threshold need not require retraining the model that recognizes when service begins. It also helps teams distinguish observation or interpretation errors from failures in reasoning, planning or execution.
For operators, “The order is ready, but no handoff has been observed” is something they can check. If the handoff area was out of view, missing evidence cannot establish that no handoff occurred.
The goal is meaningful points of inspection, without requiring every internal neural representation to be explainable.
Camera placement, resolution and viewing angle affect what can be observed. Lighting, obstructions, connectivity and packaging changes affect whether the complete system keeps working. A strong model cannot reliably recover information its installation never captures. Even when the information is available, conditions that differ from training and testing can expose weaknesses.
Starbucks’s inventory-counting rollout illustrates the challenge. In September 2025, NomadGo announced deployment to more than 11,000 North American locations, advertising 99% accuracy.
Fast Company reported refrigerator reflections doubling milk counts, Wi-Fi failures erasing scan progress and packaging changes that could require weeks of adaptation. Reuters reported that Starbucks retired the tool in May 2026, citing standardization and consistency.
Reliable operation depends on the model, its inputs, software and workflow together.
A model can work, and the deployment can still fail.
Reliability requires good models, systems engineering and sustained work with customers. Operators reveal exceptions and environmental changes that should inform model development, evaluation and installation practices.
This is why we are developing Interaction Tracking as a primary modeling objective. Many meaningful changes in an operation are expressed through interactions among people and objects, within particular spaces and workflows. Tracking those interactions can estimate the business states that downstream reasoning needs.
For buyers, the important questions extend beyond what the demo can do: What does the system actually understand, how can that understanding be checked and how will the vendor keep it reliable as conditions change? For investors, the company’s technical advantage deserves examination at the level of the complete deployed system.
A system can reason convincingly about the wrong situation and execute the wrong action precisely. Closing physical AI’s deployment gap means maintaining a reliable understanding of the operation as it changes.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?