Snowflake's EVP of Product Talks Hard Truths on Agentic AI:
Readiness, Governance, and AI Economics
Most agent demos look impressive.
Snowflake EVP Christian Kleinerman cuts through the hype on CXOTalk episode 903 to reveal why AI agents fail when faced with messy enterprise data and governance gaps. He outlines the realities leaders must accept to move autonomous workflows from fragile demos to measurable production value.
Most agent demos look impressive. Far fewer survive contact with messy data, unclear permissions, brittle workflows, and unpredictable costs. Christian Kleinerman, EVP of Product at Snowflake, lays out the hard truths: where autonomous workflows are producing measurable value, where the industry is overselling, and why many initiatives fail for reasons unrelated to model quality.
This conversation focuses on the practical requirements for deploying AI agents at scale.
The conversation covers:
- Data readiness: Why agents fail without a unified strategy for structured and unstructured data
- Governance as an accelerator: Closing the trust gap so agents can move from answering questions to taking action
- AI economics: Moving from prepaid commitments to measuring ROI and controlling the cost of agentic workflows in production
- Workforce impact: How agentic automation reshapes roles, team design, and accountability when agents absorb work once done by teams
Join this live conversation and direct your questions to Christian for real answers. Take advantage of this opportunity!
Key Takeaways
Data Quality Remains the Single Biggest Barrier to AI Success
Organizations fail to extract value from AI agents when their data estates lack organization, governance, and a single source of truth.
Kleinerman states directly that if data is siloed, for example, with inconsistent customer lists or unclear ownership, no AI model will produce reliable results. Leaders must prioritize data rationalization, establish canonical data sources, and enforce clear security policies before expecting AI initiatives to succeed.
The most common conversation Snowflake has with customers centers on accelerating data quality efforts to make their data "AI-ready." Treat data preparation not as a preliminary task but as the foundation upon which all AI value depends.
Replace Lengthy Proofs-of-Concept with Rapid Iteration and Low-Cost Experimentation
The economics of evaluating AI technologies have shifted dramatically. What once required months of planning and dedicated hardware now takes an hour of hands-on testing.
Kleinerman advises leaders to try three or four technologies in a day or two rather than running traditional three-month proof-of-concept cycles. This approach reveals genuine capability differences obscured by vendor benchmarks and marketing claims.
The friction of evaluation has dropped significantly in the AI era, making direct empirical testing the only reliable method for assessing fit.
Most AI Projects Fail When Moving from Pilot to Production
The transition from proof of concept to production exposes critical gaps in data maturity and system complexity.
Pilots may work with five well-named tables, while production environments may contain hundreds of thousands of cryptically labeled data assets. Kleinerman identifies two consistent failure modes: incorrect results eroding user trust and security violations exposing information to unauthorized users.
Organizations that built their own solutions 12 to 18 months ago now face harsh reality checks during production rollouts. Leaders should anticipate this friction and plan for the messy, scaled conditions of real enterprise environments from the outset.
Episode Participants
Christian Kleinerman serves as Snowflake’s EVP of Product and has been with the company since 2018. He oversees the company’s global product strategy and vision. Christian is a database expert with over 20 years of experience working with various database technologies and has more than 15 years of management and leadership experience. Most recently, Christian worked at Google leading YouTube’s infrastructure and data systems. Prior to that, he served as General Manager of the Data Warehousing product unit at Microsoft where he was responsible for a broad portfolio of products. Christian holds a BS in Industrial Engineering from Los Andes University in Colombia, and he is a named inventor on numerous Snowflake patents.
Michael Krigsman is a globally recognized analyst, strategic advisor, and industry commentator known for his deep business transformation, innovation, and leadership expertise. He has presented at industry events worldwide and written extensively on the reasons for IT failures. His work has been referenced in the media over 1,000 times and in more than 50 books and journal articles; his commentary on technology trends and business strategy reaches a global audience.
In This Episode
Introduction to AI Data Cloud and Snowflake's Role
Michael Krigsman: Everyone's talking about agentic AI but most enterprises aren't ready. I'm Michael Krigsman and today on CXOTalk number 903 Christian Kleinerman Snowflake's EVP of Product shares hard truths on what it really takes: data readiness, governance AI, economics, and workforce impact.
Christian Kleinerman: We're the company delivering the AI data cloud. And what does that mean? The AI data cloud we have 2 north stars or 2 important directions. One is we wanna help organizations with the entire life cycle of data from the moment data is born or created whether it's a sensor whether it's an application or a device all the way through transformation and enrichment and most important through analysis and consumption of that data. How do you get true insights out of that data?
And at Snowflake my role I run the product team a number of functions product management product design technical writing data science variety of capabilities. I've been with the company 8 years. Very happy very proud of what we do.
Capabilities and Use Cases of Enterprise-Grade AI Agents
Michael Krigsman: Christian we're talking about AI agents. Today what can enterprise-grade AI agents reliably accomplish?
Christian Kleinerman: Where AI truly shines and truly has potential in the enterprise is when you make the context of that enterprise available to those state-of-the-art models where now I can go to a chatbot and ask a question and it's not generic knowledge from the internet it is based on knowledge from my company maybe my customers maybe my products. That is where I would say the most interesting use cases are coming up today.
I would classify them on 2 categories. One is read type of use cases where I just wanna retrieve knowledge. My organization has a lot of documents a lot of institutional information that has been hard to access until now. AI shines at being able to retrieve the right information so this is where you see internal assistance customer support or customer experience assistance. All of those we are just trying to retrieve information in a helpful manner.
The other set of use cases which I think is newer but they're also happening and they're wonderful use cases is when these agents can start to take action. They can start to call an API maybe close a service report ticket maybe draft and send a response to a customer maybe build a demo. There's a number of possibilities but the most interesting thing is the state of the technology AI and models I think is far ahead from the types of applications and potential that we have in front of us.
Emeritus and Enterprise AI Absorption Challenges
Michael Krigsman: Now let's take a moment to learn about Emeritus which is making CXOTalk possible. If you're a business leader navigating change or driving growth explore executive education programs with Emeritus a global leader. They offer programs developed in collaboration with top universities designed for decision-makers like you. There's a program tailored to your goals whether that's AI and digital transformation or strategy and leadership. Find your program at www.emeritus.org. That's a really interesting point. So you say that the state of the technology is in advance of what we in the enterprise can absorb. Can you elaborate on that point?
Exploring AI Models and Their Potential
Michael Krigsman: That's a key issue.
Christian Kleinerman: These models AI models can do so much as a function of the context that you provide to them. So the more information I make available I wanna feed the information about our customers for example. Now they can start to reason about this is what happened with the customer. That's more traditional descriptive backward-looking type of analysis.
But then there's so many more things that I could start thinking of doing in terms of my engagement with my customer. What should I be doing? What others in the industry are doing? Ask interesting questions and reason in ways that truly have been unthinkable in many ways for us today that these models are ready to do. So I do have a high conviction that the state of application and use cases of these models are still being explored in many areas are still being pushed forward ahead of additional developments on the models.
Challenges and Prerequisites for AI Adoption
Michael Krigsman: And what are the limitations that are driving this gap?
Christian Kleinerman: Most of the scenarios where we see projects going wrong or projects getting shut down they run through 2 types of issues. Issue number one is correctness of results or correctness of actions. The promise of AI is amazing and to the extent that you can trust the results from that AI everything's wonderful. But we have heard scenarios of a company rolled out an assistant and internal customer service reps are starting to give wrong answers. That is unacceptable so I think that plays an important role.
The other aspect is access to the right amount of data or the right context. And when I say right I mean in terms of security and privacy. We have heard about organizations that tried to roll out their own solution in the last 6 12 months and all of a sudden they were giving information to people that wouldn't have had access to that information so that leads into a security and privacy concern.
I would say to the extent that there are solutions that deliver correctness of results and enforce its existing governance and security policies that's when AI shines. And of course that's a lot of what we have been working for the last 18 24 months at Snowflake.
Michael Krigsman: What is the driver of this gap between the technology capability and the enterprise ability to make use of it?
Christian Kleinerman: I think it comes down to in many areas organizations are not ready to provide that context provide that data in a structured and organized fashion to the AI models. The canonical conversation I have with many of our customers and our organizations is when you ask a question "Tell me something about customer X Y Z" do you know where to go and answer and get information about that customer in your organization? Because if data is siloed in all sorts of places and nobody really knows what's where it's gonna be a lot more challenging for any AI model to be able to understand that.
Whereas what we see at least the technologies that we've done with Snowflake and other players in the industry are doing similar is you end up curating what we call a semantic view which is a translator between business terms and where the answer to certain business terms or certain business questions may lie. That is the type of data governance and data organization and data quality that is a prerequisite for organizations to be able to tap into those results from AI.
Michael Krigsman: Is this gap of organizations being able to adopt the full capability primarily the result of poor or insufficient data or is it something else?
Christian Kleinerman: For the scenarios that are data-driven the answer is largely yes. The most common conversation we have with organizations these days are "How do I accelerate my data quality efforts data rationalization efforts so that I can have my data be AI-ready?" And that's a combination of things. Maybe it's simply a matter of eliminating boundaries or silos between data sets. Sometimes it's as simple as saying "I don't have canonical sources of truth for data." If I have five different versions of what my customer list is it's difficult for the AI to figure out which one is your official answer.
And then the last one is security. Do I have clear policies on who can access what and how do you get AI to enable that? So I would say that that is true the data governance data quality is an important prerequisite for leveraging data-driven solutions in the enterprise. There are other use cases we have not talked about them things like coding assistants that help with enterprise application development and other things. But at least for the majority of the use cases that we spend most of our days which is enabling organizations to get value out of their data yes the short answer is it's about data quality and data governance.
Practical Use Cases and Evaluating AI Solutions
Michael Krigsman: What are some of the very practical use cases that you see?
Christian Kleinerman: I'll do in reverse order since I just mentioned coding assistants and I am completely fascinated by the types of use cases that code generation can provide for any of us these days. Some of you may be thinking "Well but coding assistant is only for developers and engineers." But we're seeing all sorts of functions of use cases being able to leverage coding assistants.
I'll tell you something that is super exciting to me. Many of our solution engineers the folks that are technical at Snowflake that are trying to engage with customers they can build customized demos for every single conversation that each one of our customers or prospective customers are interested in. This is sort of unheard of. I've been doing enterprise and databases for twenty-five years and for the longest time you had a canonical one demo oftentimes something about a bicycle shop or something like that and no matter who the audience is you showed up with your bicycle shop demo.
But that doesn't have to be that way. It's very much easier for me to say "Dear coding assistant I am about to meet with a consumer packaged goods company. These are the priorities for them based on maybe the last 4 earnings results or earnings reports and how do we show the value of the technology in that context?" And then the coding assistant can go and generate an example a demo or a sample that that can be used.
Michael Krigsman: Now let's quickly hear from Emeritus which is making CXOTalk possible. If you're a business leader navigating change or driving growth explore executive education programs with Emeritus a global leader. They offer programs developed in collaboration with top universities designed for decision-makers like you. There's a program tailored to your goals whether that's AI and digital transformation or strategy and leadership. Find your program at www.emeritus.org.
Christian Kleinerman: Coding assistants are generating value for functions way beyond what I would call traditional application developers or software engineers. I'll give you one other example on that line of thinking product managers in my organization now they're not just waiting for engineering teams to go and create products or early mods or early functional prototypes. They're doing it themselves because it's fairly easy for pretty much anyone in an organization to guide a coding agent into building solutions.
So that's on the coding side. And the other set of use cases that we're completely excited about is it just unlocking the value of data. The trend that we have seen in the last maybe 20 years is data democratization is a trend. It's something that we want more and more people in a company to be able to get access to. What business intelligence did 20 years ago which was data is not a problem of IT data is an opportunity for everyone to inform their day-to-day decisions. I think AI is materially dramatically expanding the set of use cases. We've introduced at Snowflake something we call Snowflake Intelligence which we think of it as the next level of unlock on companies being able to do more with their data.
Michael Krigsman: We have a question from Twitter from Arsalan Khan and he says... His question is provocative but I'll ask it anyway. He says "Will the AI bubble explode when organizations realize the best they can do with AI is chatbots?" I think we're talking about the underutilization of the capabilities really.
Christian Kleinerman: We already see use cases that go far and beyond chatbots. The notion of chatbots being AI comes from that first category of use cases that I described which was the more read-only capability. I just want to retrieve data. And I think there's plenty of opportunity there. But now that we see agents be able to take action I think that's very very different from a chatbot. Yes in many instances the interface is conversational in nature but that doesn't mean it is a chatbot.
If anyone that is listening has used any of the coding assistants Cloud Code or Cursor we introduced also Cortex Code at Snowflake you quickly appreciate that this is so much more than a chatbot. It's not I ask a question and I get an answer. It's more I give a command and maybe an hour later sometimes 4 hours later I get solutions answers to the problem or the prompt that I specified that are typically tasks that would take humans days or weeks.
So I would say now that agents can autonomously take action follow up on items reason what we see is effectively an augmentation of human capability that I think is unprecedented from our times. I was chatting with an organization a few weeks ago and they have a data team they have a number of analysts traditional but they also augmented it with agents AI agents. Actually they have a name for one of the agents. And they think of the agent as a member of the team. And when a request goes out that goes to the channel in Slack that every member of the team gets to answer questions the agent is part of that. And believe it or not it has a lot of productivity is very able to help with others.
We do the same thing at Snowflake. We have a number of integrations with the tools that our employees use on a regular basis and it goes materially beyond the chatbot. So I do think that the potential is way higher than many of us see today. I think on a regular basis we see that aha happening from individuals which is a "Yeah it's cute. I can ask questions I can get answers and half of the time are right or wrong." We're moving quickly past that. And once we see AI-based technologies agents taking decisions taking actions making changes on our behalf it's truly exciting.
Michael Krigsman: Christian there has been so much hype out there. How can business leaders evaluate vendor claims about autonomy about reasoning about elaborate workflow capabilities? How can business leaders look to see what is real when it comes to AI agents right now?
Christian Kleinerman: If we all pay attention to a lot of the claims out there it is dizzying 'cause many of them sound similar overlapping. It's complex. My recommendation to everyone is go and try the technology. The beauty of this era of AI and agents is most products are a couple of prompts away from someone trying it.
And I think instead of saying "I am going to run a 3-month proof of concept on AI solutions" yada yada yada. No I say "How about we spend an hour with a product and we'll see how good or not they are?" We're doing this on a regular basis just evaluating new models new products new applications of the models. And quickly you can see whether what they're doing is materially better than the state of the art versus just another quick wrapper on top of the AI model. So my recommendation because it's hard to tease apart the noise is try it. We're in a world where anyone that speaks English can largely use most of the products based on AI so that's the path forward in my opinion.
Michael Krigsman: So basically you're saying you've got to do the empirical testing for yourself. You can't rely on vendor marketing in order to determine what's real and what will work in your particular environment and use case.
Christian Kleinerman: Correct.
The Evolution of AI in Enterprise Technology
Christian Kleinerman: And that has been true for certain aspects of enterprise technology for many years. It's only that the cost of the POC is going down materially. I've been doing this long enough that I remember when if I wanted to do a POC for an appliance a database appliance you would get on a sign-up sheet and they would tell you "Well your turn to get the appliance shipped to you is 8 months from now and then you get it for a week." That sounds completely crazy in this day and age.
What's happening with AI products is just go and try it. Usually it's on the spot and an hour later you have a very good sense of what what it is. The other thing that has happened traditionally that I don't think it works that well for AI is benchmarks used to try to simulate or model customer use cases and then you could say "Well this vendor is better on this benchmark so maybe I'll go without a POC." I think even though there are plenty of benchmarks that are constantly evolving and vendors leapfrogging one another I think nothing substitutes leveraging or trying a product for the needs of your organization.
Michael Krigsman: You raise a very interesting point about the experimentation. Is that something that is fundamentally different about AI in the enterprise that it lends itself to trying this and trying that and iterating extremely rapidly?
Christian Kleinerman: One hundred percent. In an ideal world that would have been true of all enterprise technologies in the past and SaaS applications in the past. But it's harder it's expensive to go through a quick evaluation of a SaaS app or a quick evaluation of I mentioned appliances as a use case but traditional database technology. What's unique about AI is that in many use cases the time to create and test a pilot is orders of magnitude lower than some of the other technologies I just mentioned which opens this door to say "I don't need to go and just read the vendor claims and decide based on that. I can just try it."
And we hear on a regular basis I... We did this Snowflake World Tour where we take our message to a number of cities 23 cities and I heard directly from customers: "I just did a test of your technology with two other vendors and this is what we like." Which is amazing. That the friction used to be so high that that didn't happen. Now it's materially lower. I'm not gonna downplay that the cost is 0 but it's so low that it's not unthinkable to try 3 4 technologies in the period of a day or 2.
Michael Krigsman: So there's a real mindset shift that must take place for people managing and leading AI deployments.
Agents and Interoperability in AI
Michael Krigsman: And when it comes to agents how does this approach overlay the agent world?
Christian Kleinerman: Agents are the best manifestation of AI which is the AI models but they have the ability to reason through some context and be able to invoke some tools or take some action. So I would say at this point agents is the best instantiation of the AI models. And it has materially changed the effort needed to go and try one of the solutions.
There's also a number of interoperability protocols emerging. MCP is gaining a lot of traction. I think no customer conversation I have these days goes without saying "How do you plug into an MCP ecosystem and can I be part of my broader agent strategy?" Same thing is happening with agent-to-agent protocol and I think more and more interoperability is going to emerge. But the notion of companies should much easier... Again I want to be careful to not say 0 cost but much easier. Try this technologies is changing what's possible and I would encourage companies to change how they think about evaluating technology.
Michael Krigsman: Are there limitations of agents today based on the technology limitations of data and so forth? Where are the boundaries of where we go into the bleeding edge that becomes maybe dangerous say?
Christian Kleinerman: We don't know the bounds. And that was a little bit in my comment earlier that the state of the technology may be ahead of our applications. Where most companies take a slower more guarded approach is in regard to the two issues I was referencing which is one correctness of results and security privacy compliance guarantees.
Correctness of results does not just apply to did the agent or the chatbot give me an incorrect answer? The more we talk about agents taking action it is important that that action is also correct. Probably one of the last operations that will be fully automated with AI is gonna be things like a bank transfer or a wire transfer 'cause you wanna make sure that those things are a hundred percent correct. But in the meantime there are all sorts of use cases. In the consumer world all the examples are take a reservation to a restaurant. But in the enterprise world it's simpler. It's like close a support ticket or generate a set of questions for this review that I'm about to go to or generate a set of insights for maybe a board meeting. And not that the stakes are not high but they're lower than some of the more mission-critical use cases.
Michael Krigsman: Can you talk about the data and governance gaps that can cause problems with agent deployments?
Data Challenges and the Role of AI
Michael Krigsman: You touched on this earlier but really drill into the data for us.
Christian Kleinerman: This is in my mind the single biggest impediment for companies to tap into AI today. And it comes down to if the data estate of a company is not well organized or is not broadly accessible there is little magic that the AI can do. I like to say AI is a turbocharged super capable technology but it's not magic. If you have two data sets say with customer lists and they're inconsistent with one another a great AI solution may tell you "Well I found two lists and according to this list Michael is a customer. According to this list Michael is not a customer." They cannot make a judgment call on that.
So I would say cleaning up data having a single version of the truth which is sort of the holy grail in enterprise data systems is like for any question there should be only one answer or at least factual. Having that type of rigor matters a lot because all that the AI systems are doing is reasoning and retrieving based on the data. But if the data is messy nobody can help you.
If you were to ask the best analyst in your company to answer a question would this best analyst know where to go know what to do? Then there's a path for AI. If the best analyst says "No I just need to go have lots of meetings 'cause all the knowledge is in people's heads" that's gonna make it harder for AI to do it. There are promising efforts on AI trying to infer some of that knowledge from the data but that starts to add additional levels of non-determinism and potential risk.
Michael Krigsman: We have a question from Arsalan Khan on Twitter right now very specifically about the data that is institutional knowledge residing in people's heads. And there's a number of implications of that including Arsalan raises will people feel we don't need them if they share that information?
Christian Kleinerman: In most technology changes some resistance like that comes and I think the reality is what a lot of this technology can do is enhance productivity. Yes maybe rely less on tribal knowledge and what's in people's head but it also frees up individuals to go and work on additional tasks higher value tasks.
I forget who said it someone said it publicly and now I repeat it almost on every instance I get a chance which is I don't think individuals should be worried about being replaced by AI. I think individuals should be worried about replaced by individuals that know how to use AI 'cause that's the more real thing. And yes relying on institutional knowledge or tribal knowledge in my head that's not good. Now I'm gonna be called at all times of the day which you can say is good for independence and job stability but that's not long-term good for the company or for the individual.
Michael Krigsman: So how should organizations manage and govern all of this unstructured data that not even in people's heads you know documents images audio? How should organizations deal with that?
Christian Kleinerman: There's a broad trend in the industry and at Snowflake we're leading much of this which is how do we capture in what is called semantic models and semantic views and ontologies that business knowledge that oftentimes is either embedded in tools or embedded in people's minds but how do you codify it in a way that it can be made available to the AI? So that is happening. We started a cross-company in a consortium to do interoperability of semantic models. We're very excited about number of companies participating in that and it's all about how do you take that corporate knowledge and codify in a way that you can more intentionally make it available to AI. So I wanted to add that to the comment on knowledge in people's minds or embedded in tools.
Now let me shift to your question on unstructured data which your premise was spot on. Unstructured data you don't even have to know is it in someone's head or something like that? No. The reality with unstructured data A is the vast majority of data that an organization has. There are estimates that put it 90% 85 95. The specific doesn't matter but I think most of us would agree vast majority is just on and on documents and logs and emails and all of that context which is truly difficult to retrieve or has been truly difficult to retrieve up until AI.
In the world of AI there's still some amount of okay you need to go give it the right index. I need to figure out if you're gonna do this type of semantic search approximate how important is correctness and citations. But what AI has done for unstructured data is as if overnight it turned on the light. Imagine that we had all this data around us but we... If it's dark in the room you don't even know that the data is there. Now you open your eyes and like "Oh there's so much for us to learn as a company." It's incredibly exciting what AI is doing for unstructured data.
Michael Krigsman: We have really an important question from Sai Penumuru from on LinkedIn who says "Why are so many projects getting stuck at the proof of concept stage?"
Christian Kleinerman: In my mind a lot of what we saw maybe 12 18 months ago there was this general sense that AI is so powerful that it's easy for me to just go and roll up my sleeves and build the solutions. I'm gonna build my customer support agent. I'm going to build a talk to my database agent. It's not too hard.
Challenges in Scaling AI from Proof of Concept to Production
Christian Kleinerman: And many of those initial trials produce great results. In isolation I know exactly the question. I'm working with maybe 5 tables in my database and everything works great. And what we've seen I would say for sure last year plus is that by the time you try to roll those POCs or those organic efforts into production then you get the intersection with reality and reality is harsh.
Reality is like "Oh yeah you tested it with 5 tables but we have a hundred thousand tables and they're not clearly named." In the POC they were called orders and line items and customer. In the production system it's something like table one table two table three and it's impossible to know what's in there. So the reality operates at a different scale. It's messier. Back to this data quality concept it matters on how organized and how discoverable and understandable data is.
And the other piece that I mentioned is correctness and trustworthiness of solutions has stopped many pilots from going to production and it has shut down projects that went into production just because again the POC sounded great but the real system is providing wrong answers I don't know one out of four times which you can say "Well it's not too bad." But one out of four times if you're serving I don't know a thousand customers in a day that sounds a lot of unhappy customers. I think that's the friction on do-it-yourself type of solutions which is why ourselves and at Snowflake and others in the industry have seen a lot of momentum and uptake with our solutions like I mentioned Snowflake Intelligence because it's harder than it seems and now customers are ready to say "I want to put it in production. I don't want to have to roll up my sleeves."
Michael Krigsman: This issue of the leverage of agentic AI also means that when you have errors there's a magnification of those errors that can grow.
Governance and ROI in AI Implementation
Michael Krigsman: And we have a question from Lisbeth Shaw on Twitter who says "Models and algorithms have a tendency to drift or become polluted and with agentic AI that can cause havoc. How should companies address this set of issues and who should be responsible for this inside organizations?"
Christian Kleinerman: Some of these trends and problems were there and were visible in traditional machine learning technologies. When someone was creating a recommendation engine with again predictive ML the discipline emerged to how do you do evaluations and how do you monitor for drift and how do you make sure that you're not over fitting a model? And some of those techniques are effectively how do you govern monitor and maintain non-deterministic technologies. Back in the day when everything was 100% deterministic and your balance is above 0 or not that that's easy. That will not drift.
But as we got into more sophisticated solutions and ML and ML was the beginning of that and a lot of best practices have emerged in how do you govern ML into production now they're in overdrive with AI which by definition even with all the same inputs is gonna give you variable outputs. And I think the same set of patterns are emerging to govern AI solutions in production. At Snowflake we've enabled our customers to do logging of responses track evaluations be able to see that what was intended to happen is delivering the answers that customers need to deliver. So I would say there's precedent with machine learning. We need to go and do a lot more as an industry to help organizations cope and put into production with confidence solutions that are inherently varying and adapting to context.
Michael Krigsman: So on this point of the nature of the technology being non-deterministic how should business and technology leaders measure value measure return when the cost the consumption patterns the productivity gains remain uncertain? It's not the same as putting in an ERP system where you know you have your defined order flow for example.
Christian Kleinerman: In all instances the benchmark or the evaluation has to be a function of what am I getting in return for what I'm putting in? I'll give you an example. Some consumer-level mobile app... was trying to make some LLM calls in the critical path of the application just to show a slightly better recommendation. That's expensive because the uplift in the quality of the recommendations didn't come anywhere close to the cost of the AI.
So I would say that this is no different predictive and unpredicted technologies. What are you putting in and is the delta greater or not than the benefit that you get? So that one is is more the exceptional case. In many instances what we're hearing is it's productivity gains for employees in an organization oftentimes measuring orders of magnitudes. A use case that maybe an analyst can take I'm gonna make it up five hours can be done by an agent in five minutes.
The ROI in there is so large that there's plenty of room for AI companies to make money including all the platforms in between and still the company gets a positive ROI on that. So it's not to say "Yeah let's charge whatever for AI and have companies pay as much." But depending on what it is benchmarked against in many instances the ROI is so large that there's actually a lot of reason to be optimistic about this. And back to the previous question the premise of it's a bubble no I think there's a lot of real productivity in here.
Michael Krigsman: It's an extraordinary comment that you just made that the ROI is so large potentially and I think that gets right to the heart of all of this vast amount of investment that's being made in AI right now.
AI's Impact on Jobs and Recommendations for Leaders
Michael Krigsman: But we have a really interesting question on Twitter from Miles Suer. And Christian I'm gonna ask you to answer the next questions pretty quickly 'cause we're simply gonna run out of time. Miles Suer says this "Research shows that much data is immature unwrangled and without a semantic layer. It's a market limiter for every software company. And what is Snowflake working on to help with this?"
Christian Kleinerman: Data quality and data cleaning is getting the way of companies leveraging AI. We have launched capabilities to help extract semantic information from a number of sources whether it's existing BI tools or query logs. So we're helping organizations extract that knowledge. And also we're partnering with a healthy number of third-party companies startups that are working on inferring some of this semantics from data. So it's a work in progress for the industry but it's very exciting what's going on.
Michael Krigsman: Very very quickly Sai Penumuru comes back and from his earlier question and he says "Are there key solutions you can recommend to or advice for moving AI from proof of concept POC to production?" And very quickly please.
Christian Kleinerman: It's too generic of a question 'cause depends on what AI and what solution. But if you wanted to do talk to your structure data I'll plug in try Snowflake Intelligence.
Michael Krigsman: We have a question from Arsalan Khan on Twitter. Folks keep your questions coming in. We have a few minutes left. We'll try to get through these really fast. Arsalan Khan now wants to shift the conversation to the workplace implications and he says that "Performance reviews are a mix of objective and subjective opinions. When AI gets involved it's more objective. And should AI be used in hiring and firing?" And what I'm really interested in is the impact on jobs from agentic AI.
Christian Kleinerman: I would think of AI for people related matters as an assistant not a decision-maker. 2 reasons. One is AI results are still as good as the inputs that you provide to the AI and there's subjectivity in curating and choosing those inputs. And the other thing is we also established that there is some variability in the responses that AI produces. So I would say do leverage it maybe use it for comparison. We were summarizing an interview feedback last night for someone but it's summarize not replace it for a decision-maker.
Michael Krigsman: Do you have much interest and cause of concern among your customers for the impact of agentic AI on jobs and the workforce?
Christian Kleinerman: Everyone has the opportunity to do more with their data which should translate to faster business outcomes better business success and that's what at the end of the day gives prosperity. Jobs will evolve. Jobs are evolving already. But I'm on the optimist side that this is a productivity play massive productivity play but people will adapt jobs will adapt companies will do better if they leverage AI.
Michael Krigsman: What advice do you have for senior business and technology leaders on managing this period of rapid change and particularly in relation to agentic AI?
Christian Kleinerman: Don't take on large-scale AI rollouts. When you hear "We're gonna buy a hundred thousand licenses of X Y Z solution and put it out there" and it's like well do you even know if it works or not?
I think the better approach in the same way that I earlier said the cost of evaluating a technology is fairly low I would say find smaller use cases put them out in production. If it works well scale it. That's the the mode I think all of us should be in. Technology is advancing very fast. If you tried something 4 months ago and it didn't work there's a chance that it may be working today. So just stay in that constant state of evaluation trial small rollouts and scale based on success.
Michael Krigsman: So constant iteration constant trying new things because it's all changing. Take what works put what works into production and discard what doesn't work basically.
Christian Kleinerman: Correct. That's correct.
Michael Krigsman: All right. And with that we are out of time. A huge thank you to Christian Kleinerman Snowflake's Executive Vice President of Product. Christian thank you so much for being with us and taking your time. I'm very grateful to you.
Christian Kleinerman: No likewise Michael. Thank you so much for having me and I look forward to doing it again sometime.
Michael Krigsman: Everybody who watched thank you for those great questions. Before you go subscribe to the CXOTalk newsletter so we can keep you up to date on our upcoming shows. Everybody hope you have a great New Year. Our next show is in early January and we'll see you again next time. Take care everyone.

