I have been debating under what conditions AI will have a lasting impact on the enterprise. This is a strategic question for enterprise leaders, as it dictates where and how they allocate their AI budget. Approaching AI purely as a utility would imply that the bulk of the investments for the lasting impact will be borne by others, e.g., governments and hyperscalers, rather than the enterprise. Waiting for the AI utility could delay how quickly enterprises scale their pilot AI efforts. Approaching it as infrastructure would imply that the enterprise builds upon whatever backbone exists at decision-time, no matter how complete, reliable, or expensive it is.
Today, many business leaders, from CEOs to CFOs, business unit executives, CIOs, and Chief AI Officers, individually or jointly, are waiting for AI to become a utility. They are waiting for AI to become cheap, ubiquitous, and deterministic, very much like getting electricity from a wall socket. My argument is that this is a strategic mistake. Moreover, this moment will never arrive in the way they expect.
The Utility Trap
Treating AI as a utility assumes that “better models” from Google, OpenAI, and others, coupled with the right hardware systems, will easily solve an enterprise’s specific business problems, while the cost of intelligence races to zero.
This is a fallacy. While the cost of raw reasoning (public tokens) is dropping, the cost of contextual accuracy (the enterprise’s business logic) is rising. Hyperscalers will build the “intelligence grid,” but they will never build the “last mile” connection to the enterprise’s proprietary data. Recent research found that up to 42 percent of enterprise AI pilots will be abandoned in 2025, largely because organizations are waiting for models to magically work without investing in the proprietary infrastructure required to ground them.
The Emerging Enterprise AI Stack
To understand why the “utility” assumption fails, we must look at the stack AI-first enterprises are actually deploying. It is a sophisticated, multi-layered architecture. An enterprise cannot just subscribe to this stack; it must architect it. The stack consists of:
The Compute and Storage Layer: This layer incorporates computing for model training and inference, storage, network interconnect, and energy. Today, this layer is offered by corporate hyperscalers and government-controlled data centers (sovereign clouds and here). Companies offering the Compute Layer compete on energy efficiency (Joules per Token), making compute a differentiated asset.
The Public Model Layer: A mix of massive proprietary Foundation Models (GPT-5, Gemini 3), and smaller general-purpose, task-specific, or modality-specific models, some of which are open-source, e.g., Llama 3, DeepSeek 3, Nano Banana, etc.
The Proprietary Knowledge Layer: This is the first layer where the enterprise’s unique value lives. It is a dual-store system comprising:
The Latent Space: Unstructured data (emails, documents) converted into embeddings (vectors) so neural AI models can understand context and semantic meaning.
The Symbolic Space: Structured data (customer records, business rules, various types of relations) typically represented as Knowledge Graphs so AI models can adhere to deterministic reasoning and constraints.
The Application/Agent Layer: The execution layer that incorporates the business logic, another piece of business value, that implements the enterprise’s business processes. The logic is incorporated into enterprise agentic applications.
The Orchestration Layer: The “switchboard” that coordinates and oversees interdependent agentic applications so that complex workflows can be performed, prioritizes and parallelizes application execution, and routes prompts and API calls to the right model(s) based on cost, trust, latency, performance, privacy, and cybersecurity requirements.
The PARK stack, LanceDB (Latent Space management), and NebulaGraph (Symbolic Space management) are relevant open-source projects.
AI Breaks the “Utility” Analogy
Electricity, water, and telephony are typically associated with essential services, not utilities. As such, they offer a specific economic contract: commoditized pricing (you pay for volume, not quality) and deterministic performance (it’s always there when you want it).
AI violates this contract:
Price Differentiation: A token is not a token. An inference from GPT-5.2 is a fundamentally different product than an inference from a quantized 7B model running on a laptop.
Probabilistic Performance: Neural AI models, including foundation models and their variants, make best guesses. They hallucinate. A system that varies its output based on a parameter such as temperature, as LLMs do, cannot be regulated like a power grid.
Fragmented Access: Unlike a country’s power grid, the AI landscape is fragmenting. Sovereign AI initiatives in China, the Middle East, Singapore, and other places are creating distinct “sovereign AI zones” with regional rules governing the compute, model, and orchestration layers, shattering the idea of a single, universal utility.
Better Analogies
If AI shouldn’t be compared to a pure utility, what should it be compared to? Under certain conditions, a more apt analog is search. Consider how the web works today. Google organizes the world’s public information. It provides a massive, standardized “Public Index” that benefits everyone and acts as a public infrastructure.
However, to function correctly, business processes may need more than public search results. For this reason, enterprises also develop their private search infrastructure that works in tandem with a search engine’s Public Index.
The enterprise needs a system that combines the proprietary knowledge layer, the application layer, and its orchestration layer with the public layer that brings together the foundation models, the diverse ecosystem of publicly available general and specialized models, and the compute and storage layer.
As we move from business processes that utilize exclusively digital agents (like Walmart’s shopping assistant) to processes that incorporate embodied AI and adaptive robots (like Tesla’s Optimus), the public layer’s models are enhanced with World Models. These models provide the common sense (gravity, object recognition). The proprietary layer provides the specific skill (folding a shirt, assembling a part).
The Strategic Imperative
Ultimately, the debate between “Utility” and “Infrastructure” is a question of business value.
The enterprise that treats AI as a utility becomes a tenant. It pays rent to hyperscalers and AI specialists for intelligence. When they raise prices, the enterprise pays more. Also, when they deprecate models, the enterprise scrambles. Most importantly, the enterprise’s “intelligence” is identical to that of its competitors because they are both pulling from the same public socket.
The enterprise that treats AI as infrastructure becomes a landlord. It builds a proprietary asset that grows in value over time. Every entry into the Latent Space and the Symbolic Space, every enhancement of the Orchestrator, increases the equity of the enterprise’s intelligence. Building infrastructure does not mean smelting your own steel. It means owning the factory. This becomes the enterprise’s “AI Factory“ that scales intelligent applications and generates enduring value.
This approach dictates the enterprise’s budget strategy. The AI spend shouldn’t just be OpEx for API calls (Utility). It must include CapEx for building this Factory (Infrastructure).
