Redis CEO:
Scaling AI Agents from Pilot to Production

Rowan Trollope: Turning dollars into tokens is easy, and we're seeing that with this massive growth of the models in terms of their revenue. But turning those tokens into P and L benefits and back into dollars is actually much harder.

Michael Krigsman: Rowan Trollope is CEO of Redis, which feeds AI agents real-time data.

Agents fail at the edge cases

Rowan Trollope: It's primarily a data problem. The obvious data you can connect to an agent, your order system, your customer service, you know, backend, et cetera. But it's the edge cases where these agents fail. It's the discount policy that perhaps is sitting in a Slack channel somewhere or in an email that got emailed out that isn't accessible by the agent. So agents need to be able to access all that data that sits inside the heads and the Slack channels and the emails of a complex enterprise system.

When you are running agents at scale, now you're talking about not just one agent or a handful of agents, but as you said earlier, thousands, tens of thousands, hundreds of thousands. That's real load on your infrastructure that you have to manage and you have to rate limit and you have to control and orchestrate. So that's the other part of it. So it's both accessing the right context at the right time that is often hard to get at.

Managing that at scale is a kind of a different classical computer science scaling problem. And those problems still have to be solved even in an agentic world.

Turning tokens back into business value

Michael Krigsman: Why is it so difficult to balance cost, context, and scale?

Rowan Trollope: Turning dollars into tokens is easy. Right. And we're seeing that with this massive growth of the models in terms of their revenue, but turning those tokens into P and L benefits and back into dollars is actually much harder. So how do you take raw intelligence and turn that back into business benefit?

And I think that's really the sharp edge of what customers are dealing with right now, and dealing with the shift of deterministic compute to probabilistic systems, which are AI systems, is really something that's not been presented to enterprises as a challenge before, so it's new.

Context, not intelligence, limits agents

Michael Krigsman: You have said that AI agents don't have an intelligence problem, but it is this set of context issues. Can you talk about that in business terms?

Rowan Trollope: The latest large language models are very, very smart. Imagine if you had just read the whole internet, you know, last year, and you could recall it with alarming accuracy. That's a large language model. But put them into an enterprise and expect them to do relatively simple things, and you start to get somewhat unexpected results. I mean, you would assume if I put a PhD, you know, in 100 subjects and a Fields Medal winner in mathematics and the best orator you've ever seen into any job, they would do fairly well.

But that's only true to the degree that you have educated them on the job that they're doing. And that's a context problem. It's like that education and the feeding of all the right stuff into the context window at the right time. And some folks early on in this transformation took the approaches of, let's just put everything in there. Let's put all of our company knowledge base, let's put all the training manuals, let's put all the, you know, this and all the, you know, Slack messages that we can find.

The problem is then you're just overloading it with unnecessary information. And no matter how smart these models are, they really do start to fall down when they're overwhelmed with irrelevant information in that context window. So the context problem that I speak about is choosing just the right context at just the right time. And that is a hard problem.

Michael Krigsman: But Rowan, as models become smarter, won't this entire set of context problems automatically just get better and eventually dissipate, go away?

Rowan Trollope: No, the model getting smarter, what that is implying is that it's getting a higher IQ. Just getting a higher IQ does not make an agent better at knowing, you know, how to handle a refund request from a VIP. That's a very specific piece of company policy that needs to be understood by the agent. And it doesn't matter how smart it is, if it doesn't know the company's policy, it's not going to be good at doing that job. The problem is more a context problem at this point.

It's not so much an intelligence problem. So I think that's been a big innovation in the last few years is tool-calling agents. And most of the context now can be provided through tools. And you just have to basically tell the agent, here's where you should go look for this or that or the other thing. Here are your memories. Here is company policy. Here is the product manual. It doesn't have to all be loaded on all at once.

It can then go search and query and find the relevant information in real time.

Pricing still depends on human judgment

Michael Krigsman: You've spoken about pricing agents as an example of context.

Rowan Trollope: Pricing is one of those things that agents have famously gotten wrong over the last few years. You know, we've seen customer stories in the news about customers calling an airline and, you know, demanding refunds and where the airline would normally not give out a refund, it turns out if you cajoled and badgered the AI agent enough, it would just offer up full refunds. And so, you know, getting that right is quite difficult.

And companies have very evolved policies when it comes to pricing and discounting that often involve, you know, escalations and talking to other people and all kinds of, you know, sort of stuff. And they rely a lot on judgment. You know, it's a lot of stuff you cannot document in a paper policy about when to apply discounts and when not to apply discounts. That old sort of thing that humans have been doing for millennia, haggling. Very, very hard to see any modern examples of a computer that is really great at haggling.

There's been a few attempts at this. In the context of a business, and we talk about things like, can I get a discount on my, you know, my mobile phone plan? Well, they have, for sure they've got a standard offer that they make to you. If you've ever called any subscription company and you've threatened to cancel, the first thing they do is offer you a discount. They're not making that up. It's in a book. It says, the customer threatens to cancel, give them this discount.

That's pretty easy for an agent to do. But it's hard for an agent to deal with the customer says, Absolutely not, because this thing happened and that thing happened, and you know, I was going to spend more money, but now I'm not, and blah blah blah. And at that point, you know, maybe revenue for that quarter is pretty important, or maybe you just need to hit one more customer retention to hit your quota. All of those facts, all of those pieces of information are pretty useful in making a decision.

And so that relies heavily on human judgment. Now, one could argue that eventually we can document all of that at such a rigorous level, perhaps using AI, that the AIs will become even better than humans at doing pricing.

Memory and communication for agent fleets

Michael Krigsman: How do Redis and your products address this kind of exception management situation that you were describing?

Rowan Trollope: So the first thing that we did is we built an agent memory product, literally a thing that extracts memories from a text conversation with a customer, figures out what's important to remember, stores that, figures out over time which facts are relevant and should be kept in memory and which ones should be forgotten, right, over time, et cetera, or compressed. We also created a context gateway that really helps enterprises take all the stuff I talked about earlier around FAQs and documents and processes and all that stuff.

And basically provides it to the language model in a very easy way for it to consume those tools. In a business environment, as you start to scale up, you start to go from, you know, one agent to 10 agents to 100 agents to 1,000 agents. And as you mentioned at the beginning of the show here, there's going to be hundreds of thousands of agents in the average business. And I would argue it's going to be even higher than that if you think about agentic processes. But they're hierarchical.

You might have the agents that are actually talking to customers at any one time. Maybe you have a couple thousand of those for your business, you know, 2,000 or 3,000 dealing with active customer, you know, chats or phone calls. But behind the scenes, each of those agents may be communicating to, you know, a dozen or more worker agents that are off answering hard questions, reasoning about things, figuring things out, searching backend knowledge bases, you know, entering things into CRM systems, et cetera.

And so the communication between agents, there's a new protocol for that called A2A, agent-to-agent communications, all runs on Redis. Redis, our product called Agent Mesh. So we are really helping enterprises and some of the leading enterprises in the world build out fleets of agents at scale for their enterprise.

Evals and governance get agents into production

Michael Krigsman: Among Redis customers that have successfully put AI agents into live production, what are the common threads or patterns that you've seen?

Rowan Trollope: One is having good evals. Probably the number one thing is knowing, being able to answer in a mechanical way, is this agent doing the right thing? So if I ask a question, I might have a corpus of test questions, for example, and I want to make sure that you answer them in a way that I find acceptable. And you know, you don't want to do that with a human being typing and getting the answers and judging it. You want to build a test suite. So that's called an evaluation.

So you want to have rigorous evaluations for your agents. That's kind of part one. Step one is just making sure you can tell, is this agent good? The second is tracking how the agents do against those evals over time. This is all in the category of governance, which is, why did my agent make that decision? So maybe you find that the agents are degrading in terms of the quality against your evaluations.

In the contact center, there's a whole sub-industry called workforce management that deals with quality and you know, agent capability, yada yada, that whole sub-segment of an industry is being replicated in agents around automated evals and governance. So you want to be testing them, you want to be taking the cases where they didn't do the right thing, and you want to have enough data traces to understand why did they not do the right thing. So governance and evaluations and observability and traces.

These are all the things that go into making a production agentic system work very, very well. And I think it's something that a lot of companies are now discovering and implementing and it's starting to actually work because we're now starting to see enterprises deploy agents at scale. They're being helped by leading technology companies in Silicon Valley like Decagon, Sierra. You know, there's a lot of tech companies that are working hard on these problems.

Michael Krigsman: Inherent in what you're describing is having clear KPIs or metrics early on in that eval process so you have something to measure against as you go downstream towards going live.

Rowan Trollope: That's exactly right. Getting all of this stuff written down, documented, formatted well, that's all part of your, I would call your enterprise alpha. It's what makes your company your company. Your policies, your procedures, how you operate, how do you train your people, how do they act, you know, how do you enforce, how do you test that they're acting in the right way and then validate and then correct. That's all, when you think about building a company, that's what you're doing.

As we see the agents deployed in the enterprise, the companies that get out in front of this and can really corral all of that information within their enterprise and then serve it up effectively to their agents, they're the ones that are going to get the most benefits over time.

Agent workloads outgrow human-scale infrastructure

Michael Krigsman: You mentioned Decagon as one of your customers.

Rowan Trollope: So Decagon is a customer service company. They're one of the leading tech companies, frankly, in the Valley that's driving agentic customer service products to customers doing a great job. They're actually using us for agent communications. I don't know, hundreds of thousands, millions of agents simultaneously. And they use us as that communication substrate, the technology that in real time allows them to, you know, deal with complex work processes. Those work processes often happen in parallel.

So an agent decides it needs to go do something, and it might fire off a bunch of different processes at the same time. All that work has to be orchestrated and managed, and that's a communication system. So that's the Agent Mesh I was talking about earlier. It's a way of handling all that communications, and it uses Redis under the covers. Other companies are using us for, you know, directly connecting in context and memory. So I think a good example would be Asurion. Asurion is using us in their customer service environment.

They're using our vector search and our semantic caching. They're using the underlying tools that we provide to take, I think it's about 70% of the queries from users, they actually want sent through deterministic answer paths. They don't want to give the agent too much freedom. They have a lot of training material and a lot of data that tells them in these particular types of queries, we know exactly what we want you to do.

And so they don't pass those queries on to the language model and kind of let it figure it out. They route those to deterministic answer paths where a guaranteed answer will always be provided. The product they're using from us there is called Intent Router, so they can map out, here are all the queries and here are the answers. And when you get anything like this, I just always want you to say this. You know, you say that. I don't want you making stuff up on the fly.

The current infrastructure that businesses have built up today was built for human scale and human access patterns. So first of all, humans sleep every day. You know, we work from 9 to 5 or 9 to 10 or whatever the hours are, but we're not working 24 hours a day. Agents don't have these constraints. Agents don't have bandwidth, I/O. They can type much faster than we can. And they can, you know, output instructions much faster than we can. They can read much faster than we can.

And they don't sleep and take breaks. So the access patterns against your backend infrastructure as a business are very, very different than the access patterns of human beings. So one consequence of that is scale. As you deploy, let's say you have a company with 1,000 employees, but every employee is using agents to one degree or another in their daily work.

You know, you can have times when an employee might be trying to solve a problem and may fire off a fleet, a fan out of agents to go do a research project that creates 1,000 agents, all doing internet searches and doing this and that, or I should call them agentic processes. And so that's going to hit your infrastructure in a way that an individual human never could. So you have to redesign and re-architect your data infrastructure and your application infrastructure to be able to handle these wildly different access patterns.

We have done that at Redis. We've been thinking about this for a while. So we've built interfaces directly for agents to interface with the architecture, and we've deployed agents in the backend of the architecture to operate the infrastructure. You can't have human beings servicing infrastructure and agents on the other side. The scale factor doesn't work. So for example, we've been operating Redis for thousands and tens of thousands of customers for many years.

All of the knowledge we've collected over the last, you know, 15 years of doing that at the highest scale, the biggest companies in the world rely on us to do that. We've taken all that knowledge and trained our own set of operations agents that sit in the background and basically keep your Redis infrastructure working all the time. No humans involved.

Agents need guardrails, just like employees

Michael Krigsman: What about security and compliance concerns as organizations share that additional context with their AI agents?

Rowan Trollope: Security is one of the top concerns of every executive team, every manager deploying AI. I think this is really right up there. And there's a few different angles to it. You know, security, compliance, governance, these things all are interrelated. But purely from a security perspective, if you're building new applications that have access to customer data, you have to put security and governance rules in front of those tools.

And so, you know, there have been lots of cases of, you know, large language models being tricked into exposing, you know, personal details that they shouldn't expose. You have to reimagine, well, what happens if I have a probabilistic agent that's fairly clever that, you know, may go off the rails and do bad things. By the way, we have those probabilistic entities within businesses today. They're called employees. And we have guardrails and systems and processes to make sure that the employees don't make mistakes.

In fact, the common refrain in the cybersecurity industry today is that the weakest link in your enterprise security is your employees. They can be socially hacked and engineered into, you know, providing information that they shouldn't, their passwords can be stolen. All those problems that we have, those problems we have today, they already exist in the agent world. They're just sort of turned 90 degrees on their side. It's a lot cheaper for a bad actor to come after your company now with an incredibly powerful toolset in the large language models.

And so the only answer to that is to fight fire with fire. Enterprises and their cybersecurity groups have got to be deploying large language models and agents to do the forensics and to evaluate the cybersecurity risks and holes and so on.

Pick a few top-down AI initiatives

Michael Krigsman: Rowan, can you talk about the cost issues as agents proliferate? The cost of tokens has just skyrocketed.

Rowan Trollope: The value equation is not yet proven. There have been two broad approaches that companies have taken. The first approach I think most companies probably took was kind of let a thousand flowers bloom, which was open up the tools to your employees, let them have at it, and sort of see what comes of it. And I think in that scenario, you know, it's hard to put your finger on where the value is. You know, you can see the bills, but you can't see the benefits.

Leadership teams are now pivoting towards a tops-down approach. They're saying, okay, it's great that employees can use these tools and they're getting experience with it and figuring out where it can be applied. But once you've done that, you can take two or three initiatives where you think there's going to be real benefit to the business and go invest hard in those two or three things and do a tops-down model. So you could sort of think step one was bottoms-up. Now step two is very much more tops-down.

I'm seeing a lot of companies do that and get great, great benefits. Financial services companies, for example, are taking KYC and AML, you know, know your customer and anti-money laundering. Larger banks can have thousands and thousands of employees, often offshore, who do this work, you know, at significant expense. And it's very tightly controlled and regulated in terms of the steps and the processes. And so a lot of the tasks in those teams can be automated. So that's a top-down approach to resolving that.

And I think when you do the top-down approach, you can rigorously measure the outcomes in terms of the value and the benefits.

Leaders must learn AI hands-on

Michael Krigsman: Rowan, Redis occupies a unique spot in agentic AI. And you talk with so many business leaders. What advice do you have for CEOs and other senior business and technology leaders as they scale AI agents?

Rowan Trollope: Number one is, it's really important that you use it yourself. You know, whatever line of business or whatever type of business you're in, everybody can benefit at this point from being a power user of large language models. So the first thing is, and look, in my business, AI has changed the entire, I mean, we make software. It was the first disruption was software development. So I had to go back to the basics and sit with language models over the last two to three years. Well, since ChatGPT came out.

You know, painstakingly relearning the craft of software development with a large language model at my side. And that changed pretty dramatically from three years ago till today, and it's still changing. But I had to personally get involved as a CEO to understand, you know, look, the way I learned software was I wrote software and I built software companies over the last 30 years.

But if you're starting a software company today, you wouldn't do what I did 30 years ago or 20 years ago or 10 years ago or even two years ago. You would do it totally differently. There's a lot of buzzwords being thrown around. There's a lot of, you know, it seems like every day you wake up and there's some radical innovation. And I think a lot of people, especially executives, are feeling anxious about this amount of change that's happening, which is of course normal.

But what I would suggest is carve out some amount of time out of your schedule, 10, 20, 30%. You know, maybe take a couple weeks of dedicated deep diving to really learn the new technology and what are its limits and what are its capabilities so that you can figure out how to reinvent your business. And this is tops down, right? The executives of the company have to do this.

If you rely on all the organization and the people lower down in the company kind of coming up with a new, you know, figuring all that out for you, it's going to be a fail. So that's the first thing is, and I think the most important, personal hands-on experience in your craft or your business or your industry is absolutely essential and irreplaceable.

Michael Krigsman: I have had other guests on CXOTalk describe the same thing, and it seems like this is essential advice.

Rowan Trollope: It is. And by the way, it's fun. It was a little hard for me to juggle my schedule, kind of having to run the business for today and go relearn software engineering with AI at my side. But man, I've had a lot of fun doing it. You know, and that's one way to offset the anxiety of like, oh my God, what changed today? It's just go get hands-on.

Just go start doing things, start building, start getting, you know, start using the technology in all the different ways and try to see where its limits are. That's really, it's just really, really fun. I talk to a lot of people who are having a blast. And so it's a fun time. It's never been a better time to be really in business or in technology.

Michael Krigsman: Rowan Trollope is CEO of Redis. Rowan, thank you so much for taking time to chat with us today.

Rowan Trollope: Thank you, Michael.