Inside AI Strategy with Google Cloud's CTO

Google's AI Strategy and Agent Development

Michael Krigsman: Google is betting billions on AI agents and multimodal models. But what's really behind the strategy? Today, on CXOTalk number 897, Will Grannis, Chief Technology Officer of Google Cloud, takes us inside what you need to know now. I'm your host, Michael Krigsman. Let's get into it.

Will Grannis: My team and I work with our top roughly 150 customers around the world to make sure that as this technology factory that we're spinning up all the time and releasing new products, that they know how best to leverage that technology for their own specific business use cases and needs.

And then on the other side, we also work on kind of new and emergent problems in technology to try to solve them. An example of this could be right now as you build tens, hundreds, even thousands of agents, agent alignment against tasks is a really difficult problem. So that's one of the things that more of R&D mode we work to make easier for our customers.

Michael Krigsman: What's happening in the agent world, and then we can talk about what you're doing specifically with agents.

Will Grannis: Well, agents have quickly become kind of the surface through which many consumers and businesses are starting to experience the power of AI. And if you think about it in historical context, we've always been looking to automate tasks that human beings, it isn't necessarily the highest expression of our value.

So for example, you know, in spreadsheet software, we have been running formulas for many years to try to automate the calculation of certain numbers and to automate a task of calculation. That's something computers do really well, and humans don't need to necessarily spend their time on. If you've ever worked on a laptop, and you've ever tried to put a macro or an automation on your laptop, that was, you know, an early form of task automation. Because, you know, if you're continuously clicking or doing the same thing over and over again, you wanna automate that so it gets done a little bit more quickly and more efficiently.

So agents are kind of like, almost like this third wave of automation and intelligence in that they can take intent, and they can take it in a variety of ways. It can be typed, it can be spoken, and it can execute tasks on your behalf. And this obviously has profound potential impact in, today, real world impact, across a number of consumer use cases, as well as business to business use cases.

And it's not unusual in my, you know, I mentioned earlier I work with, you know, 150-ish customers around the world, top customers, and across all industries. It's not unusual for me to run into an organization these days like Highmark Health, which has an agent that they've deployed, an agent platform they've deployed, where their employees who have routine questions about benefits, processes, procedures within the organization. Before, they'd have to go to websites and scroll through it. They'd have to ask somebody if they know how to book travel, or how to expense something, or how to provide, or how to set up a conference room.

All of those kind of basic everyday tasks an organization has to do, all that knowledge now has been indexed and is now available through a very simple interface that, today, I think roughly 60,000 users at Highmark Health are interacting with agents to get the knowledge that they need to execute the work that they need to execute, almost immediately.

Michael Krigsman: Now let's take a moment to learn about Emeritus, which is making CXOTalk possible. If you're a business leader navigating change or driving growth, explore executive education programs with Emeritus, a global leader. They offer programs developed in collaboration with top universities designed for decision-makers like you. There's a program tailored to your goals, whether that's AI and digital transformation or strategy and leadership. Find your program at www.emeritus.org.

Gemini Enterprise: Google's Unified AI Platform

Michael Krigsman: What are the complexities of creating these agents, and what are you doing, what Google doing to make it easier?

Will Grannis: We are consolidating our own approach to agent development and AI development. And just yesterday we launched something called Gemini Enterprise, which brings together all the different pieces of AI and agent creation and deployment and management into one singular stack and package. And what this has done is it's created what we like to refer to as the new front door for AI in the workplace, and it consists of six core components.

So first, the chat interface or the AI interface. This is something that people are now used to using. Three years ago was a brand new concept. Now it's, you know, it's very commonplace. So this is the first piece of this Gemini Enterprise is the chat interface.

Now, underneath that are all the models that feed the intelligence. So this could be Gemini 2.5 Pro, it can be Gemini Flash, it could even be multimodal models, such as Veo and Imagen. And those are the models that help people execute their tasks, and that's the brains and the execution engine behind agents.

Underneath that is the agent platform. So individuals within an organization can build this kind of task automation level with agents, but also multiple organizations within one company or firm can come together and build multi-agent workflows using this third tier, this agent platform. Underneath that, you have out of the box agents. So we have data science agents. We have deep research agents. We even have coding agents that are available to get value from the platform out of the box and then create scaffolding and enrich them over time.

There's, anybody who works on agents knows you need access to data and context, so there's also a set of connectors to third-party data sources, like your ServiceNow implementations, workflows, Oracle, Salesforce, as well as even Jira, Confluence, and databases like Google BigQuery. So those connectors are a key part of this Gemini Enterprise Stack.

And then finally, and maybe most importantly, as agents proliferate governance and security, so making sure that the agents are created by the right people with the right policies, have the ability to execute their tasks safely and securely within an enterprise context. And so that's job number one for us in making agents more useful and more available, was bringing together all of the ways that we help organizations with AI and agents.

There are a couple of really important things I think translate beyond even Google's approach to this for any organization in general when they're building and deploying agents. So number one, I mentioned this earlier, is context. Think of agents as a very explicit task execution machines, and within an organization, first step is to make sure that you actually have, written down or available to, you know, this task automation, your business process or your business workflow.

One interesting quirk right now, Michael, is that in regulated industries such as financial services, healthcare, public sector, those organizations for many years have been highly documented in terms of their business process and workflow, their standard operating procedures, and so they actually have a jump start into the agent world.

Whereas we tend to think of technology as, you know, coming to regulated industries second or a little bit later, in the world of agents, they're seeing time to value a little faster because they already have all of those key business processes and data documented that could then feed the agent execution and the agent rules through a workflow. So the context and the data is critical.

Michael Krigsman: I just wanna remind everybody that right now, you can ask your questions. If you're on Twitter X, use the hashtag #cxotalk. If you're watching on LinkedIn, just pop your questions into the LinkedIn chat.

Best Practices for Agent Deployment and Future Trends

Michael Krigsman: And this is from Anthony Scriffignano, who's a very prominent data scientist, and he says, "One of the biggest challenges with RPA, robotic process automation, was automating previous tasks that were not designed for automation," and so how do you advise clients to be agentic ready before they rush to deploy agents in the enterprise?

Will Grannis: Every customer that I've worked with that has been successful in these early agent implementations, they've all picked a problem or a business process for which they had data and context, they had standard operating procedures, it was a known problem in a workflow. And so to your point around trying to automate the wrong thing, they take the time and the diligence and it's kind of, in the old world, we would call this application rationalization. I guess in the new world we might call this, you know, agentic workflow rationalization, and that's taking inventory of all the things that you might choose to do and being very methodical about picking one that has the data, the context, the documentation. That's absolutely critical.

Then once you, you know, you've identified that very specific workflow that you're trying to automate and execute, it's also extremely important to have a sense of how you're going to evaluate completion and success. And evals actually, and evaluations, tend to be one of the stickiest parts of the kind of agentic development workflow today because, think about it as a manager at a desk. And in the past, the work that that would come to that manager would be gated by how fast humans could bring decisions or bring execution tasks to that manager. And so, you know, it was a steady stream of them.

Today, agents and automation can bring many, many tasks for approval, and it overwhelms a human's ability to make decisions fast enough. So the only way out of that trap is to design evaluations and AI as a judge or AI as a critic or AI as an evaluator so that the AI itself, and the agents themselves, can evaluate task completion, whether it's good enough or not, and either send agents back to do a better job or move on to the next step in the workflow. And that is a unique problem that kind of transcends RPA, which is RPA being kind of single-task, single-automation. These are multi-task, multi-step automation concerns now.

And, you know, looking ahead a little bit, one of the things that we see is that most of the value from agents will be derived from multi-step, multi-task, multi-agent workflows. And an example of this could be, you know, there's a really interesting problem in telco and in customer service in telco, and that is that when your wifi goes out, you know, that's a very, it's, in our hyper-connected world, wifi going out is a very emotional moment for human beings, myself included.

Modernizing Customer Support with Multimodal Experiences

Will Grannis: And so when the wifi goes out, the first thing you wanna do is you want to get help. And the typical workflow for getting help is you go to the website and you try to scroll through, you know, whoever your provider is or the mobile app, you try to find how to debug the problem. It's a very chat kind of 2010 interaction model.

Well, in the world of agents, and there's a telco in the UK that's doing this right now that we're working with, what they do is they have an app experience, and if someone is experiencing an outage of the wifi and they still have access to cellular, they pop up a live multimodal experience where the customer support agent will come up, they will have the individual point their camera at the equipment that they have in the house, and they're able to figure out how to do troubleshooting based on the equipment that's in the location, you know, of the apartment or the house that person is through a completely native multimodal experience.

And so that is an example of, you know, you have an orchestrator agent, you have sub-agents underneath the hood that are characterizing, you know, a video and orchestrating the chat, and orchestrating the text. So all these different agents are coming together to create this very seamless support experience. And that is a very modern multi, multi-step, multi-work, multi-agent workflow.

Michael Krigsman: Now, let's quickly hear from Emeritus, which is making CXOTalk possible. If you're a business leader navigating change or driving growth, explore executive education programs with Emeritus, a global leader. They offer programs developed in collaboration with top universities, designed for decision-makers like you. There's a program tailored to your goals, whether that's AI and digital transformation, or strategy and leadership. Find your program at www.emeritus.org.

AI as a Judge: Evaluating and Automating Decision Processes

Michael Krigsman: You made a very interesting point several times, which is this idea of intelligence or judgment, I don't think you used those terms, as being a key distinguishing factor. It seems to me that's the point that actually changes all of this.

Will Grannis: That's absolutely true. And we call it evals, or AI as judge, or AI as a critic. But it is the step at which companies themselves and organizations themselves also have to, in some cases for the first time, actually document what their decision rules are.

You know, I come from the enterprise. I've been working in enterprise technology for a few decades now. Longer than I would like to specify very specifically. But one of the things that happens in a lot of companies, in most companies that I work with and I've worked in, is there's the process, there's the documentation, and then there's the human judgment, unspoken norms and things that sit underneath the surface.

And so in order to give, you know, the software is very explicit. If you say, you know, "I want," we're working with a company right now, and part of what we're working on is people want to buy home goods, but they want to be able to see it in their, in their kind of living space before they choose. And so if you're gonna create an AI as kind of a designer to automate these processes and to create this kind of, "Before you buy it, let's put it in your space." That's a very complex problem because you're dealing with physics. You're dealing with inventory, real-time inventory. There's a lot of very complicated pieces to making this design work really well.

And so what we've had to do is we've had to architect evaluation at multiple steps in the process. The first step is, okay, so someone, the AI has come back and said, you know, "I wanna create this type of setup in this room." Well, first off, does it defy the laws of physics? A funny little quirk is one thing that we found when we were first starting to do this is that, you know, manipulating objects in three-dimensional space, sometimes we'd have a couch sitting on its side. And so we had to actually instruct, you know, give explicit instructions to the evaluating agent to say, you know, "It has to sit with, you know, the bottom oriented towards the bottom of the space." We actually had to orient the physics. So that's step one.

Step two was, is this item actually, you know, an item that's in our inventory? You know, a very specific item. And so that was the second evaluation and critique.

A third evaluation and critique was there are certain clusters. If you ever go to a home design showroom, you'll see that they purposely put an end table and a chair and a lamp and a appliance, they put them all together in clusters. And one of the things we learned is that these clusters should meet certain brand criteria or brand sensibilities. And so depending on which company you're working with, they may have different clusters. So the clusters had to match the design sensibilities and brand sensibilities of that.

So we're already into four different evaluation steps, and that, it's that methodical definition and methodical evaluation and instrumenting all of that that, to your point, that really unlocks the scale. Once you get that right, then you can send lots of different requests and lots of different users through that same pipeline, and it meets the brand guidelines, it matches the laws of physics, and it actually matches inventory. It can create some pretty magical experiences.

Continuous Improvement Through Small Adjustments

Michael Krigsman: It's really interesting the way you describe these eval points, putting them through the entire process frequently enough. Would it be correct to say that you are making many, many, many, many small course corrections? Is that an accurate way to say it?

The Evolution and Challenges of AI Agents

Will Grannis: Yeah, that's absolutely true. And one of the most interesting things to me about agents, and I've been around AI both, you know, kind of pre-generative AI and now, you know, generative AI and agents for about 20 years. And one of things that's really interesting to me is once you get into agents and agentic workflows, you're actually creating a trail of behaviors, documents, log files, telemetry that is at a scale that is, that far surpasses kind of the business intelligence, the operations logs of the business today are all gated on manual workflows. There's a human in every, in every loop.

Well, the more that you get to agents and automation and these automated workflows, you're actually creating more data and exhaust to analyze the behaviors of the agents themselves. So it's kind of a recursion, which is the more agents you have, the more interactions you have, the more telemetry you have, which then you can analyze their behaviors, and you can actually refine them. So to your point, Michael, it's an iterative cycle. Nothing works the first time. Nothing works the first time.

When we, you know, from the most simple, single flow agent to handling, you know, an inbound issue in customer service, to the most complex workflows of trying to do financial reconciliation at the end of a quarter, you know, that has to pull off of treasury. It has to pull off of systems of record. It has to be, you know, match SEC rules for disclosures. You know, that is, you know, this volume, this analysis and being very methodical about capturing what the agents are doing is both one of the greatest challenges and one of the greatest opportunities. Debugging multi-agent workflows right now is very, very complicated.

Task-Specific vs. Role-Based AI Agents

Michael Krigsman: We have a very interesting question on LinkedIn from Stephanie Tsatsos, who is an account executive with Workday. And she and she says, "Are agents role or skills-based?" Which is to say one agent equals one job to be done, which I think is so interesting, Will, given the lengthy discussion you were just describing around multi-agent technologies working together.

Will Grannis: What we see today are kind of two ends of the spectrum. On one end are kind of single-purpose, single-task agents that individuals themselves might build within their company to, let's say you've got meetings coming up this week, and you're like, "You know what? I wanna prep for these meetings."

Well, today it's, you know, it's kind of a singular agent to multi-data source and back through a singular agent where you say, "Let's get me prepped for these meetings. Could you tell me who I'm meeting with, what I ought to know about these folks?" And so that single agent workflow is pretty easy to compose as long as you have the connectors to, you know, or the data access and context access for, your calendar, for, you know, a form of CRM, for even email, then you can gather the context.

And agents are, and they are, pretty good at returning, "Hey, here are the people you're meeting with. Here's when you're meeting with them. Based on what interactions you had in the past, what we know about them, here are some things you might want to prep." So on one end of the spectrum, they're these kind of task-specific, they're not really role-specific. It's more task-specific execution, you know, very quickly against known data sources.

On the other end of the spectrum, are these more what you're describing, which are these role-based agents which have many, many functions that occur within a role. It's just like if you're a financial analyst at a bank, you have multiple tasks and multiple jobs that you execute every single day. And once you start getting into that end of the spectrum, you're actually composing multiple agents to execute multiple tasks in concurrency or sometimes in a serial form.

So one example of this is that, for example, in at Harvey.ai, it's a legal AI company. Right now, they have AIs, there's kind of, they have a role-based AI. You can think about this as you're kind of a paralegal where this paralegal AI goes out, and it'll do, it'll analyze contracts. It'll do due diligence. It'll, you know, it'll do all of this work that before, a paralegal, you know, might have to go and search through and index all this information and bring it back. The AI now can go and index that information, synthesize it, bring it back. And now a paralegal, you know, the human paralegal is more like steering and the AI paralegal is more consolidating and analyzing. That's much more of a role-based agent.

Michael Krigsman: Arsalan Khan on Twitter X, who's a regular listener, asks this. He says, "When it comes to AI agents, how many low-hanging fruits should folks go after until they address," quote, what he says, "the entire forest?" And also, who decides what is holistic AI agent deployment?

Will Grannis: It depends what your objectives are. The best advice I can give you is if you personally and/or your organization aren't already in the building and construction of early agentic workflows, you should get started right away. There is the, so that's capacity building.

I've been working in advanced technology for a really long time, and I can tell you, it always takes longer than you think to get used to using a new technology. And so just to even start the learning curve, just to get comfortable with the tools, you know, it's different. In the past, maybe you worked with APIs. Well, MCP servers and the flow of agents are a little bit different. You know, we even now have computer use which can bypass some of the, you know, need to construct APIs in front of data. And now you can just interact with, you know, the screen in front of you without having to, you know, drop a bunch of semantic ties between, you know, data sources and intent.

So there's all these emerging technologies that if you haven't, rolled up your literally, rolled up your sleeves like I have, and if you're not experimenting, then you're kind of falling behind a little bit. So on that case, I would say, you know, I talked about Gemini Enterprise earlier. One of the great things about Gemini Enterprise or, you know, these platforms is that you can go right to the user interface, the main page, and you can start building an agent right away. You can, and in Gemini Enterprise, I know 'cause I was just building one this morning, you know, you can go in. You can specify what type of agent it is. You can specify the data you want it to connect to, that you have access to based on your organizational policies. And you can get up and started right away with building agents. So do that right away.

Now, the other question, which is go tackle the, you know, the big, nasty, multi-agent, multi-workflow, kind of those role-based agents that Stephanie had referred to earlier. That's something that requires multiple stakeholders usually within an organization and usually comes from a top-down business imperative or urgent business imperative. This could be, you know, we need efficiency in our software engineering. Or, "We need lift in revenue and so we need more engagement at our, you know, on kind of our storefront from a digital consumer journey in retail," for example.

And so I would say, depending on what your role is and the nature of your company, and where you're at in the lifecycle, you know, it might be a good idea to just start building. And it, or you might be in the place where it's time to bring the finance department together, central IT together, the line of business, and actually start to construct and build these multi-agent workflows. But it's all gonna be kind of where you sit, where your organization sits in a period of time.

Organizational Challenges in AI Agent Implementation

Michael Krigsman: Let's talk for a moment about the organizational challenges associated with multi-agent orchestration. In a way it's kind of easier to talk about the technology. The technology is hard, but people are harder. So tell us your thoughts about that.

Will Grannis: The most complex problems that we deal with when it comes to implementation of technology at scale are human and organizational issues. So number one, I mentioned this in the previous question, willingness to use. If you think about every great technology wave, mobile, you know, big data, you know, AI, cloud AI, they were all started both from an organization's understanding that they need to do something differently, but usually it's by people within their own organization who are experimenting with new technology.

We, you know, we call this shadow IT or, you know, kind of consumerization of IT. People discover these technologies. They see how it could be leveraged for their business and they're bringing it in from the bottom up.

And so one of the things that, you know, I have been really appreciative of, at least in Google's approach to technology, is that we really encourage folks to experiment with new technology, and we're always watching for where the hotspots show up. So for example, we figured out that AI and, you know, this kind of new generative AI wave, you could access it through the command line interface if it, there was a really, you know, a smart integration done. So Gemini CLI, you know, is this thing that we just released. It was a small team that worked on it, and, you know, exploded immediately.

But it was our organization's willingness to allow a small number of engineers to try something, and to, you know, allow it the space and time. Not a lot of resources, by the way. That's a misnomer. It's actually very scrappy, very bootstrap-y. But just allowing and fostering and being willing to encourage experimentation is super, super important.

Second organizational thing that I highly recommend is, when you're getting into agents, again, they're going to take instructions, and they need rules of engagement, and they need explicit guardrails. And if you don't have standard operating procedures or decision frameworks within your organization for how you wanna, you know, how you want to guide your own operations, you can't guide agents, because they can't interpret intent very well. They're very explicit right now.

And probably the last one is, you know, we've mentioned this at the beginning in a kind of a different lens, but there are, in every organization there are hidden rules, agendas, norms. You know, it's important that, to understand that if you ask software to do something, it will do it. If you don't ask software to do something, it won't do it.

And as you're constructing, you know, agents within a large organization, a lot of the, a lot of the outcomes that people are disappointed with, you can trace back to, they just assumed that it would be able to make a logic leap that wasn't explicitly given to it, and that's a different way. You know, today, a lot of what, the way that, managers and leaders and organizations work is they trust the humans that they hire and they put in these positions to bridge the gap between what is stated and what is unstated and what's needed for the business. And so being really aware of that and getting in that culture of experimentation will often surface hidden rules and norms that the organization didn't even know that they had, because the software will break.

Michael Krigsman: You're the CTO of Google Cloud, and we want the AI to intuitively know and understand our implicit rules of engagement and our implicit culture, so we can just let the software do its thing and help us.

Customizing AI for Industry-Specific Applications

Will Grannis: Think about it this way. If you receive, look, Gemini, for example. Gemini has been trained on, you know, an enormous amount of information that is accessible to everyone. So if you're looking for outcomes that are very specific to your industry and/or your company, then it also requires pairing this AI with grounding or other data sources that will allow it to understand the unique context of your business, your industry, or your use case. There are no shortcuts.

So, you know, these frontier models need, and I think the biggest opportunity for organizations is to bring their data, their context, and their understanding, partner with these frontier models and this amazing AI that's being provided, and create something that is specific to their organization and their industry. And that's where differentiation and that's where competitive advantage lies. It doesn't lie in accessing the same model everybody has access to. It lies in combining the power, the incredible power of these, you know, AI models that have a general understanding of intent based on them, the data they've been trained on, but you might have very domain-specific language within your industry.

So for example, I spent a bunch of time in manufacturing and industrials. You wanna talk about a place that has its own language. You know? There are terms, very specific terms that mean very specific things, and AI isn't necessarily gonna be trained on what those terms mean, and so it won't have the ability to bridge you know, that semantic layer, you know, going from general interpretation of how of what a term might mean to a very domain-specific term.

This also shows up in code. A lot of organizations have built their proprietary software languages. You see this a lot in financial services and healthcare companies. And the AI wasn't trained on that data out of the box. So as an organization, it's that combination of bringing your domain-specific data and language and understanding intellectual property, and combining that with this AI in a privacy-safe way that creates the real, the really big outcome.

So Michael, I hate to disappoint you, but, you know, it requires these organizations to do both, to leverage that AI but also, you know, using a platform like Gemini Enterprise, bring in and ground it in the realities and the specific knowledge of their industry.

Practical AI Integration for Businesses and Public Services

Michael Krigsman: This is a question again from Anthony Scriffignano, and then we're going to move to some new folks. But he's asking about the role that active inference can play in all of this to make agentic AI better, more useful, and so forth.

Will Grannis: The more inference and the more, this kind of, this loop that needs to be created, is gonna be an opportunity for organizations to continuously improve the quality of their agents and, you know, hill climb on performance. And, you know, one thing, having worked in AI for a while too, is it's a never-ending journey. Training AI isn't the end, it's the beginning. And having the sensibility about continuous improvement and continuous loops of training and feedback are really important to making agents improve their outcomes over time.

Michael Krigsman: This is from Justin Kavanaugh, and he's asking, "What are the most practical ways that small businesses can start integrating AI at the infrastructure or data level, not just to save time but to future-proof how they attract customers and compete with enterprise organizations so that they are not left behind?" It's a really important question, actually.

Will Grannis: The most practical method is to just leverage the native capabilities of the cloud. So for example, today in BigQuery and Google Cloud, we have native AI integrations and native AI inference in the data system itself. And I think trying to bolt it on is a costly and time-intensive proposition. And so for the smaller to medium-sized organizations, I would say leverage the native capabilities of the vendors and the cloud partners that you have. And in Google Cloud's case, you know, if you need intelligence baked into the data systems, you know, that already comes standard in what we provide.

Michael Krigsman: I just have to amplify, Will, a comment that you made earlier, which is just get started. If you're a small business, the more you can gain familiarity with the kind of services, for example, that Will was just describing, the better off you're gonna be. And you'll, then you'll learn and you'll know how to, how to take those tools and apply them to your specific business.

Will Grannis: And so much now, in our platform, for example, AI is everywhere and embedded into the services themselves at every core component. So it doesn't matter whether, you know, you're talking about a storage system, a database tier. If you're talking about even compute and how we optimize compute for specific jobs, you should really just leverage the cloud provider's integrations natively of AI into these, into every tier that might be supporting your business applications, your website, and what have you.

Michael Krigsman: We have a question on LinkedIn from Kenroy Benedict who says, "Do you see an increase in public services using AI outside of simple chatbots?" And I'm not sure whether he means public cloud services or in the public sector.

Will Grannis: So as, you know, the founder of Google Public Sector and a board member still, which I'm very proud of, the explosion of AI in public services is starting to happen. And I'll give you, you know, one example that is, I think, really, really cool, and that is one of the most important functions that public sector organizations fulfill is that they provide help when people need it. And so for example, unemployment benefits at the state level can often be, or at the, and, or at the federal level. But unemployment benefits are a really big deal because this is a person's most vulnerable time.

And one of the things that AI has enabled states to do with the state of Wisconsin and others, is instead of having someone submit an application for benefits and going back and forth with a bunch of paper, and that taking, you know, potentially months. Meanwhile, this person is suffering. They don't have access to the resources they need. It's very, it's, I mean, it can be a very, very significantly, you know, painful time for an individual and their families.

AI has enabled interactions and the processing of claims, for example, to go from, you know, weeks to hours or days, through, you know, the initial submission of information, to AI triaging the severity inherently of the cases, pushing, you know, top cases to the top. Providing initial recommendations, and, you know, all of those kind of automation speed up steps are the difference between, you know, thriving and, you know, or at least getting back on your feet or not. And we're gonna, you're gonna see a lot more in public sector services at the federal state, and even internationally.

Michael Krigsman: Greg Walters on LinkedIn says, he sees a world where all applications and functions are contained within the LLM and the AI. What say you? And I'm gonna ask you to answer that pretty quickly.

Will Grannis: Wow! That's my answer. I love the vision. I do think that one of the more exciting capabilities of these models is their ability to provide tailored user interfaces. We call them ephemeral apps or ephemeral UI. And that means that the AI becomes the UI, and that the application UI is less important. And so in that way, I do agree with the vision that, so for example, if you have, if the LLM has access to the data, has access to the user intent, we've seen behaviors where AI now can spin up the user interface ad hoc, and create what is essentially an ephemeral AI app immediately.

Michael Krigsman: We have crazy question, a good question from Arsalan Khan on Twitter, X. He says, and very quickly, please. He says, "These AI agents are like soldiers who will follow orders. Who will be the generals, colonels, or even military police to provide leadership and guidance to these soldiers? Should that be high level AI agents too?"

Will Grannis: Well, it would depend on the stakes of the workflow that you're dealing with. So for example, if you're dealing with transportation safety, those are always gonna be human in the loop, human monitored workflows. But if you're dealing with, I'm trying to create 3,000 short form videos for certain brand outcomes to go create an advertising campaign, you don't necessarily need to have a human in the loop because of the stakes of getting something wrong. So think about it in terms of stakes and what happens if you get something wrong. And you can probably back into which workflows are gonna be more human in the loop and which are gonna be more agent driven.

Challenges and Success Factors in AI Deployment

Michael Krigsman: Let's talk about deployment of agents in the, in the enterprise.

Michael Krigsman: Can you describe the common patterns of successful agent deployments and AI in general? And again, relatively quickly please.

Will Grannis: Number one success criteria is picking a problem that is actually likely to be somewhat solved by agents and higher forms of automation, faster task execution. So a good example of this would be in financial services, you know, we're working with a bank right now that has seen the, that has sped up their research function. So now they can go cascade across all the documents and information that they have, and bring back to analysts and customer relationship managers synthesized pieces of research in a matter of hours or minutes, where it used to take multiple days or a week for that research to come back.

And so that was a problem. They knew that they had the data. It was very meaningful to their customers, which is number two. And so specific problem, had the data, meaningful to their customers, and it was something that would get better over time iteratively. So those are kind of key components that we see over and over again.

Michael Krigsman: What are the most significant challenges that you see as organizations are trying to deploy agents? Where do they run into trouble?

Will Grannis: Lack of data, lack of context to the agents and the models is the number one trap door. A second one is realistic expectations in the early going. These are iterative loops, and many organizations like to, to set up projects so that they're, that, you know, if they don't get it, they're used to being really experts in their field. You know, you can think of aerospace, very precision-oriented manufacturing.

It's important to have a culture where you understand that in agentic software development, the first few iterations are probably not going to have a quality or an efficiency that you really love, but it's in the iteration, in the fast iteration, that you get the results that you want. And so that's probably the second trap door is the way that we construct projects for success out of the box in many cases is exactly the opposite of the way that these projects work and will make agent development projects successful.

I will also say the third really important factor, Michael, and this goes across whether it's agents, AI, or even technology projects in general, is leadership modeling the importance and getting involved in this. The leaders that, and I'm talking about CEO, C-suite, senior leaders rolling their sleeves up. It was a very important moment to all of us at Google, and I think, you know, signaled to the market when, you know, our CEO, Sundar, was asked, you know, "Hey, you know, what are you doing with AI?" And he's like, "I'm vibe coding." And he went into a long exposition of what he's actually doing with AI.

The Role of Leaders in AI Adoption

Will Grannis: What that does is it says, "I'm doing this. We're all in this together. I'm committed to this. We're committed to this." And I can't emphasize enough how important the leader modeling the support of this exploration, this kind of new wave of exploration, how important that is in every way to agent development.

Michael Krigsman: But you're really forcing organization business leaders, therefore, to become technologists and the CEO or CFO, obviously, Google's a special case because you're developing these products, but the average, you know, manufacturing CEO, CFO, is it really realistic to expect them to get into the guts of...

Will Grannis: Absolutely.

Michael Krigsman: Really?

Will Grannis: Absolutely. And they all want to. And see, that's, the great part is my job is to make it so the technology is mostly invisible. So instead, what they see is a platform, you know, again, Gemini Enterprise, where they can go and they can just, in plain language, scope an agent, give it system instructions, give it initial prompts, click button on data sources and build an agent. Now, it doesn't mean that CEOs necessarily all CEOs should be out there building multi-agent complex workflows for, you know, highly specific tasks.

It's my firm belief that they're all capable of participating materially in this technology wave, and that's one of the things that makes it so powerful and so ubiquitous, Michael, is that AI has gone from a department or a very, weird and mystical technology to part of everyday life. If you type on a keyboard, a virtual keyboard on your phone, you've got AI under the hood that's anticipating the words that you're trying to type and trying to serve you up a word. Any search that you run every single day, it's all around you. And for the first time in my career, the platforms to actually deploy advanced AI are accessible to everyone.

Multimodal AI and NanoBanana

Michael Krigsman: Can you talk about NanoBanana? Explain what NanoBanana is. And I have to say, I just think it's incredible. So please, just, just briefly, tell us about NanoBanana.

Will Grannis: So NanoBanana is our image generation model, latest image generation model. And this is a big theme, Michael, and that is that multimodal AI is the future for a couple reasons. Number one, if one picture is worth a thousand words, a video is worth a million pictures. And as humans, how we experience the world is through our senses; sight, speech, sound. And AI now, these multimodal AI models, like NanoBanana, can take images as inputs along with, you know, prompts, and it can create new things.

You know, if you take NanoBanana and the ability to, you know, create images and you extend it, we're able to create videos as well. VO is a video creation model that we have under the hood, a multimodal model.

Extend it even more, we have a Gemini Live API. And I mentioned that use case earlier of being able to take your phone, flip it around, show it something, show the AI something, and it knows inherently that that's a cable modem, this is the model, and you can start your debugging right away. That's, NanoBanana is a glimpse into the multimodal future of AI, and that's how humans want to interact with AI. They don't wanna have to type things all the time. Sometimes they just wanna show you something. So, you know, "Hey, here's what I'm looking at." Give me some background on, you know, this. And it can explain mathematical equations. It can explain geolocations. That is, that's the future of AI, this multimodal immersive experience, and NanoBanana is one form.

Practical AI Development and Future Trends

Michael Krigsman: Lisbeth Shaw says, "How can you tell what's real from the hype with respect to AI? Then how can you turn that into something that's practical?"

Will Grannis: Measure, measure, measure, and be very transparent. I even have to coach my own team on this. When you're building something, the natural human tendency is to show all the best parts of it and to talk as though, you know, everything is great. In our team, and what I would encourage for all of you that are working with AI and especially building agents right now, is it's possible, but it's not always easy. And it's important to show the initial, is to gain the credibility and be very realistic about how it's going, and measure incrementally.

We break, we break long agentic development workflows into very incremental steps, and we measure and we talk about what we need to do to hill climb on performance and/or efficiency at every single step. Rather than saying, "Oh, it's gonna be able to do all this great stuff," we start with very atomic, very specific, and very measurable outcomes we're looking for, and we make a decision at each one of those gates about whether to proceed, whether to change, or whether to stop. And that type of transparency in every step of development and that very sober look at what it can and can't do will help bust the hype cycle.

Also, for those of you that are leading projects, would highly encourage you not to make big statements about what this, about what AI is going to do, and instead think very soberly about one of the biggest problems that your organization is facing and how might automation, speed, multimodality, how might those things combine to chip away at solving, you know, what are always gonna be some really complex problems that aren't, you know, you're not gonna solve through a magic AI wand.

Michael Krigsman: Very, very good advice. You're saying, essentially, approach it from a very practical standpoint about what problems you need to solve rather than, "AI is great and will change our lives and solve everything in the world."

Will Grannis: Yeah. And I will also say that in the history of technology, we always underestimate how profound the changes are gonna be, and we overestimate how fast they're going to happen. And I think that also creates some of this hype cycle that you're referring to, which is now we can see the potential, but that potential will take years to realize. But to the point earlier, Michael, if you don't get started now, you don't climb the learning curve, and you don't create the incremental successes that eventually lead to the breakthroughs, and you're always watching somebody else do it.

Michael Krigsman: Where is this all going in the next 6 to 18 months? What can we expect from your vantage point, from what you know right now?

Will Grannis: First off, the models will continue to improve in capability, in quality, and you'll see more multi-native multimodal capabilities. Just like today, you know, where you can go to Gemini and you can interact through voice, you can interact through video, you can interact through images. That multimodality is gonna get more and more seamless, and it's, and it's gonna feel more natural to interact with AI.

Second, a continuous improvement around the out of the box agents that are available to help people get started in Gemini Enterprise. More and more connectors to data sources. So, you know, for those of you that have, you know, SharePoint or Jira or Confluent or, you know, all these different, all of those connectors and many, many more are coming. More improvement around governance and security, and especially in where value is transacted.

So today, many of the agent workflows that I see, they're not commerce transactions, but we just released the agent payment protocol, AP2, as part of this kind of standardization stack. So, MCP, A2A, which is the agent-to-agent protocol, and now the agent payment protocol, because we're gonna want agents to fulfill tasks that include commerce. And that is going to be a significant area of focus, is making sure that that's done well, it's done safely, it's done securely. And AP2, agent payment protocol, is the first step in that journey.

Michael Krigsman: And with that, I'm afraid we're out of time. Will Grannis, Chief Technology Officer of Google Cloud, thank you so much for being with us, and I hope you'll come back another time.

Will Grannis: Yeah, you got it, Michael. It was a real pleasure. Thanks for the questions, everybody.

Michael Krigsman: And everybody, thank you for those great questions. Before you go, I want you to right this second, go to cxotalk.com, subscribe to our newsletter. We have truly great shows that are coming up, and we want you to join us. Everybody, I hope you have a great day. Thank you. You guys ask the most amazing questions. You guys are awesome. Thank you so much. Take care, everyone. We'll see you soon.