Post Detail ImagePost Detail Image
He Built Siri at Apple. Now His Private AI Runs Inside the FBI, Google, and Netflix
Contents
AI Industry

He Built Siri at Apple. Now His Private AI Runs Inside the FBI, Google, and Netflix

AI has spent years getting better at conversation. The harder question is what happens when it is expected to do work.
by
Datasaur
on
November 11, 2026

AI has spent years getting better at conversation. The harder question is what happens when it is expected to do work.

I recently joined Sam Nadler and Jordan Metzner on Built This Week to discuss that shift and why the enterprise has become the more compelling place to look for AI agents. We also talked about open-weight models, the changing economics of hardware, and what happens to software budgets when an agent workflow uses millions of tokens in one run.

These topics are connected. As AI moves from answering a prompt to carrying out work, organizations have to think about more than model quality. They also have to consider where the system runs, how it fits into their environment, and how to understand the resources it consumes.

When an AI agent becomes more than a chatbot

For a long time, personal assistants promised a new way to interact with technology, but the experience often remained close to a simple exchange: ask a question and receive an answer. In our conversation, we discussed the inflection point when agents began moving beyond that pattern and started doing useful work.

That distinction matters. A chatbot is primarily judged by the response it produces. An agent invites a different question: can it take on a task and move it forward?

The value is no longer only in the quality of a sentence. It is in whether the system can contribute to an outcome that matters to a person or a team.

This also helps explain why consumer personal assistants and enterprise agents have not necessarily followed the same path. Personal assistants have been part of the conversation for years, yet the strongest market fit for agents today may be in the enterprise. Organizations have work that can be described, repeated, and evaluated. That gives teams a concrete way to ask whether an agent is useful, rather than measuring it only by how natural a conversation feels.

The shift from answering to doing also changes what it means to evaluate a model. Benchmark scores on reasoning tasks are useful, but they do not tell you whether a system can navigate a real workflow, handle an unexpected state, or recover from a partial failure. Those are the questions that matter when an agent is expected to carry out work rather than produce a response.

Open-weight models and the question of where AI runs

One of the more significant shifts in the past year has been the availability of capable open-weight models. These are models whose weights are publicly released, which means organizations can run them on their own infrastructure rather than sending data to an external API.

For many enterprises, this matters a great deal. Regulated industries, organizations with strict data residency requirements, and teams working with sensitive information have often found it difficult to adopt AI tools that require data to leave their environment. Open-weight models change that calculation.

The tradeoff is real. Running a model on your own infrastructure requires hardware, engineering time, and ongoing maintenance. The economics depend on scale: at high volumes, the cost of running your own model can be lower than paying per token to an external provider. At lower volumes, the overhead may not be worth it.

What has changed is that the gap in capability between the best proprietary models and the best open-weight models has narrowed considerably. A year ago, the frontier models were clearly ahead. Today, the best open-weight models are competitive on many tasks, and the choice between them is no longer simply a question of quality.

This creates a more interesting set of options for organizations thinking about where their AI systems should run. The answer is not always the same, and it depends on the specific requirements of the use case, the sensitivity of the data involved, and the resources available to operate the system.

Hardware economics and the cost of inference

The economics of running AI have also shifted. GPU availability has improved, and the cost of inference has come down significantly over the past two years. What was expensive to run at scale eighteen months ago is considerably cheaper today.

This matters for how organizations think about agent workflows. An agent that completes a complex task might use many more tokens than a simple question-and-answer exchange. If the cost per token is high, that limits what is practical to automate. As costs come down, the range of tasks that make economic sense to hand to an agent expands.

There is also a hardware dimension to this. The availability of specialized inference hardware, and the emergence of more efficient model architectures, has changed what is possible at a given cost point. Organizations that were priced out of certain use cases a year ago may find them accessible today.

None of this means cost has stopped being a consideration. A workflow that uses millions of tokens in a single run is still expensive, and understanding that cost is part of designing the system well. But the trend has been toward lower costs and better hardware, and that trend has opened up use cases that were not practical before.

What the enterprise actually needs from AI agents

The conversation kept returning to a practical question: what does it actually take for an AI agent to be useful in an enterprise context?

Part of the answer is reliability. A system that works most of the time is not good enough when it is handling work that matters. Organizations need to be able to trust that the agent will behave consistently, handle edge cases gracefully, and fail in ways that are recoverable rather than silent.

Part of the answer is integration. An agent that cannot connect to the systems an organization already uses is limited in what it can do. The most useful agents are the ones that can read from and write to the tools and data sources that are already part of the workflow.

Part of the answer is observability. When an agent takes an action, the people responsible for that system need to be able to understand what happened and why. This is not just a debugging concern. It is a governance concern. Organizations need to be able to audit what their AI systems are doing, especially when those systems are making decisions or taking actions that have real consequences.

And part of the answer is fit. The best use cases for AI agents are not always the most obvious ones. They tend to be tasks that are well-defined enough to be described clearly, repeated often enough to justify the investment, and structured enough that the agent can make meaningful progress without constant human intervention.

Finding those use cases, and building systems that handle them reliably, is where most of the interesting work is happening right now. The technology has moved faster than most organizations' ability to deploy it well. The gap between what is possible and what is actually working in production is still significant, and closing that gap is the real challenge.

If you are thinking about where AI agents fit in your organization, I am happy to talk through it. The questions are specific to each context, and the answers depend on what you are trying to accomplish.

No items found.
Related post