IBM Consulting:
The CIO Agenda for AI

Andy Baldwin: I can go onto my observability layer, and I can see the 60 models that we're currently using across IBM. 60 models. When it's proof of concept, it's relatively small amount of investment that you're making. Once you go to scale, it's completely different.

CIOs accountable but not in control

Michael Krigsman: CEOs demand AI, but are CIOs ready to deliver? Andy Baldwin runs offerings and growth for IBM Consulting, which has $21 billion in revenue.

Andy Baldwin: The demands on them and their functions and their business has never been greater, never been higher, particularly with the speed at which they now need to deliver AI applications for the business. The challenge is a lot of the time they're not in full control of those AI applications. A lot of what's happened, you know, with AI, if you broadly group it into enterprise-based, and then obviously personal productivity, either provided by the company or, in some cases, companies allowing employees to use their own personal productivity tools.

And at the same time, obviously, the CTOs, CIOs are being asked to deliver the returns that are needed for those AI investments, but then sometimes they're not necessarily in full control or have full oversight of those personal productivity assets and tools that are being deployed. They obviously do have more influence, more control over the enterprise-level agents or deployments that are being pushed into the organization.

Michael Krigsman: It sounds like the old shadow IT, but now on very significant steroids.

Andy Baldwin: It's a good analogy, and I think part of the challenge as well is obviously with AI IT, and you've seen this coming through substantially in the last probably 6 to 12 months, moving increasingly away from what I would call multiple small pilots, which if you like, at that point, the actual running cost, the consumption costs were there, but not necessarily that substantial.

But obviously, as soon as we started to scale some of those AI use cases and applications, then you are starting to see significant increases in the run costs, in the compute and compute cost, the token cost, and that's really what's put a lot of, I think, focus and visibility around AI governance so that the CIO has a level of visibility and transparency of all the AI activity that's taking place across the organization, including what would be the personal productivity bit,

which I think is what's analogous to the shadow IT organization as you referenced it, alongside, if you like, the more industrialized agent-level deployment that's obviously now starting to drive end-to-end workflow redesign.

Michael Krigsman: This episode is brought to you by Gartner IT Symposium/Xpo. Ready to scale agentic AI from pilot to production? Join top CIOs and IT execs this October 19th to 22nd in Orlando, Florida. Over 300 Gartner analyst-led sessions will cover top priorities shaping IT, from AI value, governance, and cybersecurity to cost optimization, IT operating models and beyond. Get practical insights and connect with peers tackling the same challenges you are. Secure your spot today at gartner.com/us/symposium. There are so many different tentacles that extend from this lack of control.

Why AI adoption is a contact sport

Andy Baldwin: I think what's different this time is, one, the level of impact across the organization from this technology. It's far more pervasive. It's been democratized, but it's also got far greater, if you like, business value potential if the organization can sort of manage, you know, the process debt, the human, the data debt, and obviously get the change management right as part of that as well.

Michael Krigsman: Talk about that change aspect, which is also such an important facet of this.

Andy Baldwin: My own experience is, you know, in my last organization, you know, we deployed at scale, you know, 150,000 Copilot licenses across the organization to really boost personal productivity. This was in a traditional professional services organization where one or two hours of, you know, improvement actually does immediately add to the bottom line because you're effectively charging by the hour in that business model.

And I think what I saw and what we learned from that is, you know, really thinking through how you start to incentivize and encourage people to change the way they work and to use the technology and recognize that that personal productivity does require changes in individuals' personal behavior. And I think how you encourage that, how you incentivize that, how you make the training and development available, but also how you encourage people to, you know, work very differently.

And I use the phrase a lot that, you know, AI is a contact sport. It is not something that you can then just, you know, throw over the fence and hope it's going to be used, hope it's going to be adopted. You need people to be playing around with it. You need people to be using it, and you also need people to be getting to the point where, you know, they start to build their own personal productivity agents.

In parallel, you do the corporate, you know, more the end-to-end piece, the agents that the organization will have. But that personal thing is really, you know, like I characterize, it's a contact sport. You got to get people engaged. You got to get people comfortable.

And you got to help them to see the personal value and benefit of using the tools that you're, you know, making available to them, and that's beyond just, you know, the way people have used traditional, some of the chat tools and, you know, effectively inquiring, actually starting to think and change the way they work. Michael, that's been a key thing.

The other thing I would add is just I think the balance with AI deployments versus traditional technology deployments, I think there's an increasing focus on the workflow and the change management that comes with that, I think more companies are thinking harder about that than perhaps they did with some of the previous, you know, technology deployments and iterations.

Michael Krigsman: Are there unique aspects of change management, in relation to AI versus other technology changes that we may have seen in the past? Or in other words, how does AI drive these dynamics in unique ways?

Andy Baldwin: Certainly in a professional services organization like IBM, I think one of the biggest changes is the show and tell aspect of this technology, because you're able to develop an agent or a prototype at a far, far faster speed than probably we could have done with any of the other previous technology that we were working with and using. And I think that also then applies to when you're engaging with clients or engaging with people around the technology, whether that's internal or external.

You can very quickly create an MVP or a visual of this is what the agent can do, this is the way it could work. And I think that, Michael, that's incredibly powerful to get people engaged, but also interested in how they can start to think about how that capability can start to reimagine, you know, how something or whether it's a process or a workflow or how their work can be done very, very differently.

I say that, for me, if I think over the years, 30-odd years of working in technology, you know, the extent to which business owners, business line of business functional owners are now engaging in and thinking through how generative AI and the technologies and the tools can change the way their business or their function operates at a quite a hands-on level.

You know, that for me is quite different because as historically it would've been, "Oh, I've thought about something, then I'll bring in people to do the requirements, then I'll bring in people," and it would've been there's a time lag, there's a time distance, there's a need to bring in specialists to do that. Whereas I think now with what we're seeing, timescales are being telescoped, but also the ability to have business users working with people who can use the tools, whether that's forward deployed engineers or whether that's their own people.

It creates a very different dynamic in how you develop things. But also I think the speed with which you can then see it in action dramatically reduces. Doesn't mean, though, that you can shortcut the integration of those agents and the security into their legacy workflows. I think sometimes clients and people have missed that you can develop the agents very quickly, but some of the integration activity and the security provision still needs to happen, and that will take a little bit longer as you harden up those enterprise agents for full production.

Democratization forces a governance rethink

Michael Krigsman: So essentially what you've got is this very large decentralized mass of technology availability and power, and therefore the control issues that you spoke about at the outset become heightened, emphasized, and therefore all the various potential problems that may arise.

Andy Baldwin: It's been democratized. So this technology, more so than in some of the other technologies we've seen over the last, you know, 10, 15 years, has been democratized. Once you democratize something, you need to reimagine how you think about governance, oversight, you control. And obviously that democratization, it comes with a benefit.

Also comes with some costs, and also comes with some need from a control point of view, and particularly around data and, you know, the data that's being used and where that data's going and how that data is going to then be used by the tools themselves.

Michael Krigsman: The tentacles now that are reaching out extend in so many different areas inside the organization.

The real cost of scaling AI

Andy Baldwin: I think this point about transparency and observability is why these two phrases are becoming even more important because, you know, it is becoming very pervasive. You know, this technology, you know, like, say, some other technologies where it might have been developed in one place, typically this, you know, the CTO or the CIO function, and then deployed into the business. This technology is being deployed and then being developed.

It's being applied, it's being utilized by line of business owners, by people in the finance function, the talent function, as well as in operations, et cetera. And so how you get to that level of visibility and observability and transparency is really important because of that democratization, and that's why I think you've seen. You know, and increasingly now, obviously everyone's talking about token maxing and the cost and obviously the large language models.

And what also is happening because of this democratization, you know, the phrase I use is, you know, people are using a Ferrari to go down to the corner shop to buy a paper. Well, you know, you could use a Ferrari to go down to the corner shop and buy a paper. You may feel great in the Ferrari, but it's probably not a particularly good use from a cost point of view.

It may be easier to walk, it may be easier to get a bike, or actually you might have your 10-year-old Ford Fiesta sitting there. You can use that to go down the corner. And I think this is going to be the next evolution of this democratization, Michael, and this point that you said around the tentacles getting into all parts of the organization.

I think from a CIO, CTO, you need to know the observability and the transparency so you can see what agents are being used which models are being used where, and you need to do as an organization is make sure that, you know, when you need a Ferrari, when it comes with the Ferrari cost, you use the Ferrari. But where you can actually use a smaller language model or where you can maybe even use some of, you know, machine learning or robotic, you use that to solve the task.

You don't end up using an expensive frontier model. I think that's the next evolution you're seeing, you know, clients go through. And you're seeing that, and that's why you're hearing so much in the market about these, you know, agent management platforms, this notion of observability because I think in the absence of having really sophisticated users, reality is somebody needs to be directing the user to say, "Hey, listen, that's a great agent.

You're going to get as good as output on this agent using this model, which will cost a lot less than the latest version for one of the frontier model providers."

Michael Krigsman: A few weeks ago, we had Aaron Levie, the CEO of Box, as a guest on CXO Talk.

Andy Baldwin: Mm-hmm.

Michael Krigsman: And he pointed out, given the points that you're making right now, he pointed out that we've been in this time when VCs have been effectively subsidizing our token use, and those days are coming to a close. This episode is brought to you by Gartner IT Symposium/Xpo. Ready to scale agentic AI from pilot to production? Join top CIOs and IT execs this October 19th to 22nd in Orlando, Florida.

Over 300 Gartner analyst-led sessions will cover top priority shaping IT from AI value, governance, and cybersecurity to cost optimization, IT operating models, and beyond. Get practical insights and connect with peers tackling the same challenges you are. Secure your spot today at gartner.com/us/symposium.

Andy Baldwin: We have definitely moved out of the proof of concept phase now with generative AI, and we're into the industrial scaling. You know, my own organization, IBM, we've been going through this for the last 2 or 3 years, and we've deployed AI at scale across the entire organization, whether that's in delivery or whether it's in functions. I think there are very few companies who probably deployed it to the extent that we have. We call it Client 0.

But the reason I shared that example is I can go onto my laptop now, I can go onto my observability layer, and I can see the 60 models that we're currently using across IBM. 60 models. Each of those models has a very different price point, very different cost point, and I can actually see which models are being used by which tasks and activities.

What we can also now do, certainly in my consulting business, I can dial down access to one of the models if I decide, well, I don't think we need to be using that. We can direct people to using different models. When it's proof of concept, it's relatively small amount of investment that you're making. Once you go to scale, it's completely different. I use an example of when I was at my old firm.

We couldn't let people use ChatGPT, so we used the OpenAI version of it through Microsoft, and we deployed it. And I deployed it to 3, 400,000 people as we're saying, "This is how we want you to do chat." Actually, within 6 months, we were running at a $25 million compute cost within. But the great news is everyone was using it. Nobody was using any tool that they shouldn't be using. But all of a sudden, the cost ramped up because that's scale.

We're getting the benefit of it, but all of a sudden now you see the cost of running these models at scale, which is why I said you then get into a different situation of how do you manage that observability? How do you make sure you're using the right model for the right task?

And also, I think, Michael, what a lot of clients have done is, you know, there's been so much pressure to show how they're using AI, they're deploying AI, that in some cases they architected some of those things, you know, which probably without being thoughtful about scale. So in my own example of that I just shared around effectively OpenAI, ChatGPT, internally, we actually re-architected that and brought the cost down to around 2 million 'cause we wanted to deploy it quickly.

We wanted to get it into the hands of our employees 'cause we did not want them using the open source versions outside the organization from a risk and a control point of view. So we did it quickly. But we did it quickly, and we probably didn't architect it as efficiently as we could do for token consumption 'cause we never expected the consumption to be that high that quick.

Writing controls versus enforcing them

Michael Krigsman: Let's take some questions from the audience. We have a bunch of questions that are stacking up, and we'll go first to LinkedIn, and this is from a LinkedIn user named Logi Root Runtime AI Governance. And his question is: When it comes to CIOs, in practice, are they asking questions such as, how do we write better controls, or how do we make sure that the controls that they're actually executing?

Andy Baldwin: In the current climate, you've seen certainly a significant increase, you know, in obviously the focus around security. I think they're effectively having to ask both questions now because I think the days, you know, the focus around the controls from a production point of view, how the users are now using that and how they're using the tools and how they're using this is becoming far more to the fore just because of the sophistication that is now available to bad actors with some of these models that are now available.

Michael Krigsman: So there is sensitivity to this governance and controls issue that you're seeing among the,

Andy Baldwin: No,

Michael Krigsman: You

Andy Baldwin: Know, for sure. For sure. You know, and I think this is what's come with, you know, the democratization piece. We're putting these tools into the hands of more and more and more people. But obviously from a, you know, sort of a risk point of view, you have to have the right environment and the right controls in place.

And I think, you know, if you just look at news coverage in the last 2, 3, 4 months, you know, this heightened focus around security, a heightened focus on, you know, the security and cyber breaches. You know, many of your listeners will have seen the announcement we made around Lightwell, which is really around us providing a clearing house for patching of, you know, open source. Because I think the models have started to highlight, you know, gaps and issues in some of the open source code that's out there.

But I think this focus on, you know, control and how it's being used and deployed internally, I would say, Michael, is becoming as important as the focus that was there before on building those tools and building those applications.

When AI becomes the business model

Michael Krigsman: This is from Swami Vaidyanathan who says, and he's referencing a report that IBM put out, and he says, "Given that AI will be the business model by 2030, is there already early evidence that the shift from a technology-heavy focus to something fundamentally different? And what are the 2 or 3 diagnostic signals a CIO should watch over the next few quarters to know they are moving in the right direction?"

Andy Baldwin: 12, 18 months ago, a lot of the focus from a gen AI point of view in the business was around coding. It was around making productivity, coding productivity, you know, these vibe coding tools, these productivity tools. You know, then we started to see the application of gen AI into, you know, some of the call center environments with, you know, much more sophisticated, you know, voice recognition related capability through the AI.

I think what we're now seeing is this shift in pivot into reimagining AI at scale, but in some of those key functional areas. So if I look at the work that we're doing, interestingly, finance, HR, and procurement are the 3 functions that we're doing most. Well, outside of IT, we're doing most work on from an IBM point of view. So what that says to me is that the technology is being increasingly democratized.

I think the role of the line of business and functional leader in the application and deployment of this technology is becoming far, far greater. At the same time, you're seeing pressure from the CEO saying, "Tell me what I'm going to achieve. What benefits can I expect to get?" The CFO is saying, "Where's my ROI?" And because of that, I think you're starting to see increasing demands placed on the IT organization to say, "We need to be deploying, you know, these tools at scale faster."

And so some of the, if you like, signs that I think the IT organization is seeing to indicate this shift is happening is they are seeing rises in the compute cost. They are seeing rise in token cost consumption. And that actually now is getting to the point where this starts to play into the ROI that a business looks at when it starts to think about, "I want to drive this transformation of either the functional or the overall enterprise transformation."

I think you've then got, you know, Michael, I think certain industries where I think this technology shift is existential. Being very open, I think in, you know, consulting, at the moment, the impact of this technology in consulting is existential. You only have to look at the share prices of a lot of the consulting firms at the moment to sort of see that without me saying it. And so then if you look at the advertising industry, the creative industries.

So you've got certain industries that I think are going through, you know, a much more fundamental transformational shift. But I think a lot of organizations, you know, you're seeing the signs of the adoption and the usage of this technology, and it's coming up in those run costs. It's coming up the compute cost, token cost.

It's coming up with the needs at, you know, they're purchasing more and more of the licenses, whether that's from the frontier model organizations or whether it's from, you know, some of the other providers of AI tools, whether that's the ISVs or whether that's the hyperscalers.

Michael Krigsman: It sounds to me like everything you're describing is a pattern of maturity around the adoption of AI, but at the same time, a pattern of uncertainty. For example, the unexpected explosion in token use and token costs. But the bottom line is AI is being adopted, and lots of people are using it, and we're trying to figure out what's the best way to adopt it, control it, share it, and so forth.

Andy Baldwin: I mean, the technology is very versatile, and it has multiple applications, which is different to some of the other technologies that we've seen before. You know, I was heavily involved in blockchain when blockchain came out. Blockchain was a great technology, but it had a very narrow set of use cases it was particularly strong for. And this is different. And I think because of that, that's why I think we're hitting, you know, the need from a control point of view and a governance point of view, and obviously it's using data.

You know, you've got to understand, do you own that data, or is it third-party data? Is it your client data? Because sometimes these tools, if you don't have the right governance and the right data strategy, will be somewhat indiscriminate where they capture, you know, that data from.

But I think there's, you know, an important point here, and we had a client session this morning in London, with about 50, 60 clients, and a couple of my colleagues from the talent organization and the operations team who've driven a lot of the AI transformation at IBM made a great, great comment, which was, "In certain areas, it'll be hard for us to see beyond 12 months." Now, that is incredibly uncomfortable to say, you know. And I say this as an organization with, you know, 300,000 people, 80 billion business.

But actually, when we look at, let's say, the impact on skilling and the impact on development, you know, we can see the impact of the technology in the short term, but if you said 2 or 3 years out, it's really hard to gauge and judge 2 or 3 years out what really the impact will be. And I think that's what makes this technology both exciting and, to be quite candid, you know, it's quite challenging at the enterprise level because you are learning and deploying, and you're learning more.

And that's why I think, you know, the successful organizations are building in that ability to, you know, learn from how they're deploying the technology and refine then their future deployment approaches, future deployment models, and also how to get the best out of that investment in the technology.

Michael Krigsman: This would be an excellent time for folks, for you guys to subscribe to the CXO Talk newsletter. Go to cxotalk.com and subscribe to our newsletter so we can keep you informed of these amazing live discussions. So Chris Petersen says he would like to know if the consulting opportunities around AI look bigger on the pure technology side, the execution adoption side, or the governance risk and controls side.

Andy Baldwin: If I think of consulting broadly, Michael, in 3 categories of you've got advice-based consulting, you've got implementation, and then you've got the run or operate part of consulting.

I think on the advice-based, I think what's incredible is, you know, if I look at some of the tools that we've got, we're working with at the moment in consulting, I think the ability to do those advice pieces of work and getting, if you like, a roadmap, an answer to a client's question or problem, you know, far quicker than we could have done before, I think is quite profound.

And I mean that you're taking a 12-week piece of work, you can probably do that now, including the client engagement piece and the client stakeholder management, you know, in 2 to 3 weeks rather than 12 weeks. I think when you start to look at the implementation, if I take sort of like SAP implementation, if I take Oracle and some of those large platform-based transformations, what's interesting is we can certainly see, you know, faster, do certain tasks much more quickly.

So we're getting productivity gains of 20, 30, 40% in those types of programs and projects. And so it's a quicker time to value, but you're still. Fundamentally, you're still doing those different stages of the work. I think where it gets quite interesting is in some of the operate part of the business. You know, if I take something like, you know, cybersecurity. I mean, know, I've been involved in cyber for many years. You know, the days when cybersecurity, where you had.

You know, I think you're increasingly seeing, you know, cyber threats now that are agentic, and therefore the response needs to be agentic. And so when you look at some of the, you know, cyber operation centers, they are increasingly becoming automated. You have an agent or an automated response to a, you know, a threat coming through. And so in that part of the operate businesses, I think we're seeing quite significant, you know, agentification.

And a lot of our clients, when we go in and we talk to them and we, you know, propose for work or our existing clients, they're increasingly asking that question around, "Well, show me how you're changing the way in which you're delivering this. And I expect that obviously to be, you know, at a different price point than what it was before because you're going to be using less people. But equally, I expect a, you know, more uptime. I expect a higher satisfaction."

So I think the dynamic across that portfolio, Michael, of what I would say the advice, the transform, implement, and the run and operate through, you know, GenAI is quite different. And, you know, obviously we're very focused on how do we basically manage that across the 3 because, you know, as I said, I think in consulting more broadly, I think this is an existential shift that we're seeing in how you deliver work to the clients. And at the same time, clients are changing how they work, given what we've been talking about.

And at the same time, many of the technology partners we work with are also changing the way they work in terms of putting far more agentic into their business applications.

From efficiency to reinventing the business

Michael Krigsman: Andy, from a CIO perspective, how much of this is being driven by a desire or expectation of greater efficiency versus desire or expectation of we're going to change or agentic AI is going to change our business and yield new, potentially unexpected benefits as a result that go beyond faster or cheaper, greater efficiency?

Andy Baldwin: I think there's a balance. I think over the last couple of years, for sure you saw a lot of focus on our core productivity. And invariably, to be honest, it was probably because the CEO read something. He saw these numbers from someone saying you get thirty, forty percent productivity. And like any good CEO, he immediately goes to the CIO and say, "Hey, listen. Why can't I reduce your budget by 30, 40%?" So I think inevitably some of that initial focus, that initial emphasis, for sure has been on productivity, efficiency-related measures.

I think a lot of the early use cases to prove out the technology, Michael, they were focused on that coding efficiency. You know, and organizations like us, you know, we've deployed something called Bog, which is our in-house, you know, co-coding platform model to all of our, you know, sort of 300,000 plus people. And we're seeing, you know, significant improvements, significant benefits from that.

So I think that was the focus initially, and I think what you're now starting to see though, and I think this is an evolution of where companies are with this technology because the more you play with it, the more you see it, the more you can think about how you can reimagine your organization and how a process works or an activity works. And if I like to use my own example, you know, I have those 3 businesses, the transform. You know, the advice business transform and operate under my remit.

I'm a consultant by profession. The way in which I am now reimagining that advice piece of the business and how it can be done very differently and add a lot more value more quickly to clients is probably one of the most profound shifts I've seen in the market and the business. So I'm looking at that now as a growth opportunity. And so I think a lot of clients are going through that evolution. They learn more. The technology's been more democratized. You know, they're coming back.

And I think it's, a lot of the time now, the business users who are coming more to the fore, the functional owners, the line of business. You know, I think some of the projects that I'm personally involved in, it's the CFO or it's the HRO who's saying, "Actually, I can really reimagine how I can do this very, very differently than before."

It's the sort of underwriter looking at saying, "The old way I looked at underwriting and insurance, I can do this very differently, and I can have a far better, far more effective, you know, underwriting process than I had before." Because they're now a little bit more comfortable with, a little bit more familiar with what the technology can do and how the technology can bring in, you know, those data sets. So I think we'll see more focus on the growth agenda and on, you know, new business models as we move forward.

But to be honest and to be sure, most of the focus initially has been on efficiency and cost.

Michael Krigsman: I have to just respond to one point you made, which I thought was very interesting, very insightful, that as time goes on, there's a learning curve that takes place. And it sounds like AI and agentic AI are not just democratizing the technology but also empowering workers across many, many different functions because now as they become familiar with the technology, they can help drive process change. And because if those processes are being driven by AI and agents, it's easier to make those changes.

Andy Baldwin: That productivity, that democratization, you've had people looking at and starting to redesign certain parts of a process flow. I think we're now starting to sort of capture that learning and actually start to see how you can, you know, reinvent, if you like, the overall end-to-end process, not just the individual activity.

But I think this thing around the agent learning and the employee learning and how organizations, you know, capture that, I've seen I think alongside the governance framing that we've seen coming through, I think that point about the knowledge and the learnings, you're seeing this the way companies now. You know, our research showed that most major enterprises will be deploying close to 15 hundred, I would call, enterprise agents by the end of 2026. Those enterprise agents are obviously then deployed, people use them.

But the ability to feed back on those agents and how those agents can be enhanced and improved is far better, far easier than it was with, I guess, some of the other, you know, traditional technology. And at the same time, a lot of the value from the agents comes into the quality of the data that the agents are then using to sort of, you know, make the decisions. And I think that's the other dynamic we're seeing at the same time.

I think you're seeing organizations looking to improve the quality of the data that's feeding into those agents. And what's interesting is that doesn't necessarily mean they go and do a huge, you know, data management program across the whole of the enterprise. It means they can get quite focused on the data that's needed to feed into a particular agent to complete a set of tasks or the information that flows, you know, through a process.

So you get a quite a calibrated and a more focused data approach, data strategy approach with this model.

Quantum and cyber reach the boardroom

Michael Krigsman: This is from on LinkedIn. This is from Rafal Szymanowicz, and he says, "As AI becomes the core business model, how should leaders balance AI-driven growth with the governance of autonomous agents?" And then he layers on top of it, "And the post-quantum threat to our cryptographic infrastructure."

Andy Baldwin: This point about the agent-driven growth, I think you will see an increasing and a need to have the greater focus around the governance and the control around those agents because, you know, I think whether agents are being developed by the employee or they're being developed by the enterprise in one of those, you know, 15, 1600 agents that I mentioned, Michael, that are being deployed, that we think will get deployed at scale by the end of this year, early next year.

I think the point about governance and control is still going to apply. I think what's interesting with, you know, quantum coming through is I think there's been a heightened concern amongst executives and focus from leadership on the cyber threat presented by the frontier models falling into the hands of bad actors. So that, by its nature, I think has really dramatically heightened this focus on cyber protection, cyber security.

I think that was already there with generative AI anyway because of this democratization and the way in which the agents can start to pull and use data. I think certainly my company I worked in before, which was one of the largest B2B platforms data in the world, we were incredibly focused on making sure that the data, we had rights to use the data. 'Cause we have a lot of client data that we didn't have rights to use. And so we were very focused on that anyway.

I think with what's happened, you know, with the launch of some of these frontier models, I think that's taken that to a whole different level. I think the point on quantum, I think. Look, I think on, you know, we've been looking at, as an organization, you know, we're, along with, I think, Google, we're probably one of the leading organizations researching quantum in IBM. So we've been looking at quantum safe, quantum encryption for a while.

What I'd say is, I think this is very, very much on the minds of a lot of governments. I think the advantage at the moment is, you know, production and producing quantum computers is still very difficult. You know. Obviously, a recent announcement that we're going to be one of the first quantum chip manufacturers in the world. We're building a site in the United States.

But the access to the computer technology for the foreseeable, for the next few years, will be limited because of the nature of the computers, which is good from a cybersecurity point of view. But I think a lot of organizations, because of the way in which this technology will pose a threat to so much of our existing encryption technology, I think many organizations are already starting to start to think about this and to look about this. They were doing that anyway.

I think what's happened with Mythos and a couple of the models that have been announced very recently, I think this whole thing about protection and being ready, I think, has just gone up two or three notches at the board level. Because for many management, they're getting asked this by their independent board. Their non-exec directors are saying, "What are we doing about whether it's protecting from the frontier models, or what are we doing about quantum encryption?" They were asking it anyway, but I think it's now gone to a different level.

Michael Krigsman: So from a board standpoint, it's taking issues that were seen as important but maybe simmering, and now bringing them up to a stronger boil.

Andy Baldwin: I served on 2 or 3 boards in a prior life. I think these issues are always there, but to be honest, as a board and a management team of running a large organization, you naturally focus on the top two or three. Yeah. You always focus on the top 2 or 3. I think this is a great example where it was there. The board were aware of it. They might ask questions on it. I think what we're seeing now. A lot of non-execs want to know, how are we using AI?

How are we protecting, you know, data from AI? And given what's happened in the last, you know, 12 weeks, a lot of boards are asking, "Are our systems safe? Are our systems safe? You will. We expect you to have done the preparation. We expect you to have run, you know, some of these models on our software, on our open source or our applications. Tell us where the deficiencies are and tell us how you're going to patch those deficiencies so we don't have a cyber threat."

That's what the board, you know, the boards are asking more and more, I think, to the management team. And obviously, that invariably then becomes, you know, top of the to-do list for the CIO, the CIOs. And you see this at the moment. I mean, the level of interest we've had since we announced Lightwell, and obviously Lightwell has a number of other, you know, firms working with IBM on that, not just the IBM consulting.

But the level of engagement and interest we've had from boards, especially in financial services, especially in banking, especially in insurance around this topic, you know, has seen huge increase in the last 12 weeks.

Soft landing or jobs apocalypse

Michael Krigsman: This is from Braden Kelley on LinkedIn, who says, "What is your view on what an AI soft landing might look like versus a jobs apocalypse or great labor force contraction scenarios as a result of AI?"

Andy Baldwin: There's invariably going to be some hype. There's going to be some, you know, stories, rumors, and it's going to cause, you know, anxiety. If I, you know, look at it from the work that, you know, we're seeing and we're doing, I think couple of things. So one, I think, you've got to look at this from a geographic point of view because obviously there's a number of markets around the world which have got incredibly tight labor markets.

And therefore, where there is any, you know, redeployment, redevelopment, there is clearly an opportunity there to have people quickly redeployed, realigned, either within the same organization or actually within the economy. Yeah, if you look at, say, Japan, for example, even China, I mean, the extent. They've got really tight labor markets. I think when you get into some of the other, you know, parts of the world, I think there's a point here around training and development.

You know, I was involved in some work quite recently in a session with the British government, and you know, I made the point that, you know, we need to be thinking about a level of AI skill and knowledge for all of our students coming into the workplace. At the same time, we need to be thinking about how are we equipping all of our workers, you know, to have a level of AI literacy and understanding.

There's obviously a role to play there for organizations like mine, but there's equally a role to play there for the government in terms of how they incentivize that and how they encourage that. The other thing we shouldn't lose sight of is, you know, as, you know, these technologies get deployed, you know, the skills that sometimes they need are not necessarily the skills that you think they would need. So to give you a very personal example, my eldest daughter just graduated from university about 18 months ago. She's an English major.

She's actually now in the AI division of one of my competitors, and she's doing sort of consulting work using a lot of the AI tools. 18 months ago, she hadn't even used some of these tools, but she's joined this company, great company, great training. She's now doing that. Now, the technology is allowing and enabling her to do probably work that would've taken her longer to build up the experience before she can do.

And so, you know, I think this notion that we will automatically see, you know, a jobs apocalypse, et cetera, I think is too broad-based. I think inevitably there will be some impact in certain activities and certain roles. I think then the ability or the opportunity is how do you redeploy, realign those skills and those people? Again, I give you a personal example from my organization.

We've identified 15,000, 20,000 people from a skills point of view that we look at these skills and we say, "These skills are in declining demand from clients." So if we do nothing, over a period of time, demand will invariably shrink, and we'll need to, you know, shrink that group of people.

But actually, what we've been doing is making that very transparent to people, making it transparent that they're in a skill set that is seeing less demand, and then laying out the personal, you know, learning plans or development plans for them to basically be retrained in some of the skills that where we see more demand or increasing demand that we can't meet. And so I think the critical thing about this is this is not about individual companies.

It's going to require, I think, a collaboration, Michael, between government, corporate, and the education establishments in a particular market. And we've got to accept some countries have a different set of challenges than others in terms of where they are today, but also what they're going to need to address going forward.

Michael Krigsman: This one is from Greg Walters, and it's a pointed question actually about the consulting industry. And he says, "Will there come a time when a person asks the AI the same questions they would have asked a team of consultants and get essentially $100,000 worth of work for almost nothing?"

Andy Baldwin: I think certain advice-based activities, let's take an example from my former life. There are certain questions you want to ask about tax, tax rules, tax laws in a particular jurisdiction. Will the tool be able to give you an answer to that question? Yes, it will. Will the tool be able to sign a confirmation from a regulatory point of view that it did do that? No, it won't. So I think there's going to be examples where the tools can do that. There's going to be some where they're not.

But bear in mind when you think about broadly about consulting, yes, you want to get to an answer, but often you want to take a management team, or a group of leaders on a journey to get to the answer. The AI is not going to take you on that journey.

It might give you the destination, but it's not going to tell you how to go there, and often that's the change management process you need to go through to actually reach the right conclusion, but also have that stick from a change management point of view in the organization.

Where CIOs should start proving control

Michael Krigsman: So the mechanistic answers will eventually get taken over by AI, but not the taking the folks on the journey as you described. Question from John A.K. Akinmade, and he says, "2-thirds of CIOs are accountable but not in control." That's an IBM stat.

Andy Baldwin: Yeah.

Michael Krigsman: Where should they start proving how AI is operating or proving it is operating responsibly? And very quickly, please.

Andy Baldwin: I think the key thing here is around the governance, and it's around the observability and the transparency that they have, and they can see, if you like, the agent landscape across the organization, which agents are used for what activities by who in the organization. I think when you have that observability transparency as a starting point, you're in a far better point to be able to exert influence and control, and then also work out which agents. You know, agents, like anything, can be retired.

They can be, you know, combined with other agents. That observability and transparency is how you get to that optionality and that ability.

What makes an AI pilot succeed

Michael Krigsman: You described pilots earlier. A year ago it was all about pilots. Can you describe what are the attributes that make successful pilots work versus those that quietly die and go away?

Andy Baldwin: I think it's a couple of things. So one is around, could you see business value or benefit coming from that pilot? Did the employee or client satisfaction show there was benefit from, again, from doing that. And then a point of view around this sort of increasingly when you go to scale, what's the cost and return of using that agent for that set of activities in terms of what you did before?

I think a lot of proof of concepts failed, Michael, because they either failed because they never really had the data to be able to drive the agent, or effectively they developed a proof of concept that was trying to solve a problem that when you got into it was not that material enough a problem to justify the investment.

Michael Krigsman: So selecting the right problem and ensuring you're set up to succeed, meaning that you have all of the data that you need to get it done.

Andy Baldwin: Exactly. Exactly. And look, I think a lot of clients have gone from, you know, 12 months ago, 18 months ago, people were doing like, "Where can we apply agents into this workflow?" But frankly, there are tools that will do that now for you. They will basically generate the agents based on your workflow. So that activity's gone. What people are now doing is saying, "Of those 20 agents, if you like, in finance, what are the top 2 or 3 that are going to have the biggest impact?

And now I'm going to focus on those top 2 or 3, rather than playing around with all 20."

Modernizing legacy without breaking it

Michael Krigsman: Andy, enterprises sit on decades of legacy systems. How do you recommend that CIOs rebuild that old technology and those old processes without disrupting the company in order to adopt AI?

Andy Baldwin: The quick answer is, you know, with great care and consideration. So look, I think what we're seeing with a lot of the technology, you can take a process, and you can get agents, and you can have the prototype agents literally in a matter of days, you can now do.

What takes a lot more thought, and this is where I think the CIOs, CTOs need support and help explaining this to the business, what takes a lot of thought is how you then embed that into the legacy workflows in a way in which you don't create unintended consequences. And I think that still takes time because you've got to build in security, et cetera.

I think the other big theme that's happening, though, and this goes to the heart of, I think, to your point around legacy systems, I think there's a lot of organizations are looking at some of their legacy applications and saying, "This legacy I use for a system of record, and I use it for a system of engagement. Actually, I still want to use it as a system of record because it's functionally, you know, it's efficient, it could process a lot.

But actually, I really want to rethink how my employees or my clients engage into that system of record." And I think that's where you're seeing the real power of tools like Copilot. You're seeing the power of tools, you know, of like Watson Orchestrate, in that you can have a different user experience but still access the source system. So you're keeping the legacy. What you're doing is you're wrapping new technology, new ways of interacting with that technology through the engagement layer.

And I think we're seeing a lot of that focus from organizations. So they're preserving, if you like, some of the legacy, the system of record piece, but they're really reimagining that engagement layer.

Michael Krigsman: So essentially, as you say, you're protecting that system of record and the storage of the data as your underlying core-

Andy Baldwin: Yeah

Michael Krigsman: And then you're allowing

Andy Baldwin: Changes. But exactly. But that engagement layer as well, you know, I think what we're really seeing now is in that engagement layer, you can embed the data context. You can embed some of the workflows that were there, you know, before, and you can also tailor some of that to the individual based on the agents they then use to access some of the data.

So that engagement layer, I think is where I think there's some really interesting work going to take place in the coming years with organizations as they start to think about bringing that engagement, data context, decision rights, and the use of some of these large language models and some of the other AI tools into that engagement layer, then interrogating this system of record.

Accountability and the CIO's next move

Michael Krigsman: When an agent makes a costly mistake, who is accountable: the vendor, the model maker, or the CIO?

Andy Baldwin: I think it's going to depend on the example, but let's be honest, I think in some cases, let's take if you're using AI as an advice agent, for example, then the person who's providing the advice is going to be responsible for the agent 'cause that's their agent. They've done it, they've programmed it, they've processed it. I think if these are agents provided out of the box from the vendor, then obviously the vendor's going to be responsible.

So I think the answer to your question, I'm afraid, is contextual depending on the agent and who produced it and the role it's playing.

Michael Krigsman: So eventually there will be this shared set of accountabilities.

Andy Baldwin: One of the interesting areas that's emerging is obviously, effectively assurance of agents, so that how do you have comfort that they will consistently perform the same task and give you the same output again and again? And obviously in certain activities that you need, you know, a precise output, that's incredibly important. And actually, if it's regulated, it's unlikely the regulators will allow you to do that at this stage. They'll probably still want a human in the loop. Where it's not, you know, it's more probabilistic, then actually that's probably fine.

You'll be able to use those agents because you're still requiring on the end user to use some judgment about what the output was.

Michael Krigsman: What is the one thing that you believe CIOs should be doing now?

Andy Baldwin: I think it's just reinforcing with the business that AI is a contact sport. You know, the CIO can deploy the tools, can deploy the models, but the business needs to be engaging, needs to be using it, and needs to be getting the value out of that. And then that needs to be being fed back to the CIO/CTO organization so that they can then, you know, make the appropriate adjustments. But this is about, you know, for me, it is about a contact sport.

You've got to have the end user engaging and using the technology, and then create that feedback loop to the rest of the organization.

Michael Krigsman: Well, it's been a great conversation. Andy Baldwin is the senior vice president of consulting offerings and growth for IBM Consulting. Andy, thank you so much for taking your time and sharing your expertise with us today. I'm very grateful to you.

Andy Baldwin: Thank you very much. Thanks for having me.

Michael Krigsman: And thank you to everybody who watched, and you guys who asked such excellent questions. Before you go, go to cxotalk.com and subscribe to our newsletter so we can send you notifications. We do these conversations all the time, and you should join us. Thank you so much to everybody, and thank you, Andy. Have a great day.