Why Enterprise AI Agents Lag Outside Engineering:
Box CEO Aaron Levie

Aaron Levie argues that AI agents have taken off in engineering and are only beginning to work in the rest of knowledge work, because knowledge work has less of what let coding agents succeed: fewer technical users, less verifiable output, and harder data access. Levie is co-founder and CEO of Box, which counts 68 percent of the Fortune 500 as customers. He called the split "sort of a tale of 2 cities," with "engineering, obviously, kind of complete vertical takeoff of AI agents" in one and "the real messy environments of knowledge work, where things are just quite a bit different" in the other.

Levie attributes coding agents’ takeoff to engineering’s built-in advantages

Levie pointed to what already existed in engineering. The work is "mostly in a text-based medium," and the users are engineers who "can keep up with all of the updates that are happening in AI." The output is, in his words, "more or less verifiable": "If I build a bunch of code and I can go and do a regression test on it, I can see, you know, did I break anything or does it still work?"

Knowledge work has less of the two Levie compared. Its users are a "generally less technical audience just by definition," and "the work is sort of less verifiable by definition, so a contract, you can't compute whether the contract is sort of correct or not," he said. "It has to experience the red lines from the other party."

Data access is the other difference. An engineer generally already has the code base for the project, so "by definition, the agent that you're deploying also has access to all of that data." In the enterprise, "we're constantly asking for permission to other systems and other resources and other data environments, and so an agent is only as good as the data that it has access to." Levie's rule of thumb: "If you show me your IT stack, I'll show you what you're going to be able to get from agents." His timeline for closing the gap is long: "That is just going to be a very big project that will take years and years of sort of diffusion into organizations."

AI psychosis comes back when new models arrive

Host Michael Krigsman read back Levie's remark that CEOs are uniquely prone to AI psychosis because they are distant from the last mile of work, and asked him to explain. Levie described his own first weeks with the tools and his reaction: "Oh my God, this thing is going to just automate everything." He thinks he got through it in maybe "a couple weeks," once the bugs and the review work appeared, and said it returns when a new model comes out.

"A CEO, by definition, is the furthest away from the real work that's happening in the company," he said. "You couldn't get further from, you know, in any other role other than maybe the board of directors." Levie's advice: "Use the technology actually so much that you get to the other end of that psychosis, and you can actually see in a much more practical and pragmatic way all the places where humans are still necessary to really get the ultimate gains from this technology."

Push hard, then retry a failed project six months later

Levie's answer to a viewer's question about ambition was to aim high. "Anytime you hear stories about people not getting real ROI from agents," he said, "I do think it often approximates, you know, not pushing them hard enough."

When an ambitious project fails, "you almost have to try it again 6 months later like almost every single time no matter what it is." Box had tested Fable against Opus 4.8 the day before the show, and "the thing that Opus 4.8 couldn't do, Fable just finally actually did." Asked whether advantage will come from the best models or from proprietary data and workflows, Levie said he would bet on companies that combine both: "the frontier models that they're leveraging and the best ability to get those models the right context to be able to work with."

A two- to five percent IT budget caps what AI can do

Levie put IT budgets at roughly two to five percent of revenue, an artificial cap, he said, on AI's potential: "you ultimately need the line of business to own the budget and own the sort of deployment of where do they want, you know, AI being used in the organization."

Subsidized tokens are over, and now "you're more or less going to be paying for the real underlying costs that it takes to deliver this," Levie said. At Box, provided the roadmap is sound, he said, "If the token spend goes up exponentially that should actually be correlated with a good thing, which is we can deliver more software to our customers faster." When asked about the value, he said, "It is no harder or easier to do this with AI than at any other point in history." The one difference, he allowed, might be this: "With the wrong prompt and the wrong limits, you could probably go and spend $50,000 and wake up the next day and have that bill."

The most exciting moment in history to be a CIO

Asked what advice he had for CIOs, Levie said, "I think it's the most exciting moment in history to be a CIO." The CIO role's importance "rises dramatically," he said, because "agents are maybe the most technical solution that has ever been deployed to non-technical people," and what they produce is "all determined by your technology architecture. It's all determined by how you deploy these agents. It's all determined by how you trained your users to use them." "You're effectively providing work to your organization," he said. "That's the first time in history where IT was responsible for deploying the actual sort of real output of the organization, not just the tools that enable the output."

Watch the full conversation with Aaron Levie and read the complete transcript on the episode page.

CXOTalk prepared this article with AI assistance from the verbatim transcript of episode 921. Quotations are unedited from the broadcast.