When AI Goes Off the Rails:
An Executive Action Plan

Introduction to AI Challenges and Expert Backgrounds

00:00 Michael Krigsman: AI systems can go off the rails, run amok, and hallucinate. So on CXOTalk 888, Lord Tim Clement-Jones and Dr. David A. Bray explain how to anticipate, manage, and prevent these failures before they escalate into enterprise-level crises.

David, tell us about your work.

00:26 David Bray: Currently, I am chair of the Accelerator at the Stimson Center, which simply means we are striving to figure out how we do policy and program in parallel because the world is changing so fast. Other than that, I have served in different roles ranging from countering bioterrorism, to CIO of the FCC, senior national intelligence service executive.

00:45 Michael Krigsman: Tim, please tell us about your work.

00:48 Tim Clement-Jones: I'm a member of the House of Lords. I speak there on science, innovation, and technology, particularly AI, and I co-chair our cross-party group on AI, in Parliament. And, I consult with a number of organizations on, AI policy and regulation.

Understanding Generative AI and Its Risks

01:08 Michael Krigsman: When we talk about AI, generative AI in particular, running amok, what exactly does that mean?

01:17 David Bray: One of the interesting things about generative AI is that he actually generates its own content. Before then, with different other flavors of AI, we had things that were rule-based systems, we had natural language processing, but generative AI is actually able to generate its own content. And if you look at it, whether it's an image generator or a large language model, it's sort of doing prediction. In large language models, it's next word prediction. In images, it's trying to respond to the prompt you gave it.

And then as a result, usually they have these things called random seeds. They also have temperature controls that introduce a little bit of randomness. Now, even if you turn down the temperature control, turn down the seeds, and say, "I don't want any randomness," nowadays, with these very large models, because you're doing compute in parallel, there's still some non-determinism because these race conditions that send things to different processors, you could do the same prompt twice and it may not actually result in the same outcome.

Plus, a lot of these models have what's called observer effects, which, if they observe a prompt has been asked, when you ask the same prompt a second time, it's already been altered because it observes that same prompt. So we're dealing with things that are not deterministic. They're not rule-based.

And so as a result, when enterprises roll this out, they may think it's going to do a certain thing, but as you mentioned, there's hallucinations. We may try to get hallucinations down. Let's say we even get it down to single digits or 1%, you'll still never be able to reliably predict when it may actually do something that's completely not factual. Again, it's doing sort of multidimensional pattern matching, but not necessarily factual.

02:47 Michael Krigsman: Tim, you're in the House of Lords, and so you're looking at this from a policy and governance perspective. And so from your standpoint, what is the impact of this run amok or go off the rails or hallucination aspect of generative AI?

03:06 Tim Clement-Jones: The impact of a hallucination, when it's fake news, or even when it's misinformation or other sorts of misbehavior, if you like, by the generative AI model, can have extraordinary consequences. You know, for instance, And of course, humans quite often link to a misbehaving AI, what really matters.

And now, we see much more retail use of large language model. This isn't just about corporate adoption. This isn't just about a large language model adopted by a big business going amok. This is individuals using AI models for their own purposes, which are then, in a sense, creating this problem.

So, you know, it could be elections. It could be people sending out misinformation causing public unrest. There are all kinds of areas, or indeed, discriminatory decisions by an AI system. So it's the very prevalence of AI as well, as well as the technical problems with it, which cause the problem as well.

04:24 Michael Krigsman: You know, look, people make mistakes. AI makes mistakes. So what's the big deal?

04:28 David Bray: It's when you don't double-check the machine. So for example, we saw, for example, this was early on in the days of ChatGPT 3.5, there were apparently some There was one or more lawyers that basically used it to generate some references for a court case. They looked realistic, 'cause again, what this is doing is it's doing sort of multidimensional pattern matching. They didn't go and check to see if those court cases actually existed. And then when it got before the judge, lo and behold, the very court cases that they supposedly were citing did not actually exist.

And so you need to recognize, generative AI will produce things that look very realistic but could be completely fake.

But the second example I'll give is there was a competition back in November where they explicitly told the machine, "Under no premises should you ever transfer funds," and this was made publicly available. Anybody could ask it a prompt, but you had to pay a little bit of money, so it was sort of like a self-funded contest. And by attempt number 490, so by prompt number 490, even though the machine had been told, "Do not transfer the funds," guess what the machine did? It transferred the funds.

And so that's where, again, you have to recognize, these are not rule-based engines. And as a result, what they give you may be unreliable, or if you tell it not to do something, you may find ultimately it does.

05:39 Tim Clement-Jones: Yes, and we have our own examples in the UK of publicly used machine learning tools in government, you know, having deprived 200,000 people of their benefit on an illegal basis. And of course, our most celebrated case, in a worst possible way, is all our subpostmasters who were prosecuted by the Post Office, who were using It wasn't quite AI in those days. In fact, it was an expert system, and how much worse it would be if it wasn't an expert system, I tell myself, which caused prosecutions, people to commit suicide, and they're still awaiting compensation from our government for the wrongs that were done them, for being jailed unlawfully.

So, you know, this trusting faith that people have in computer based decisions, if you like, putting it very broadly, is something that carries through into AI, particularly when AI has this ability to converse with you and seem rather human. You know, let's face it, they pass the Turing test nowadays almost certainly. I think that's probably well out of date. And so, it has this kind of human look about it which generates public trust, which makes it extremely dangerous in that space, in that case.

Trust, Skepticism, and Responsible AI Deployment

07:11 Michael Krigsman: So, this trust aspect, this kind of false confidence that leads us to believe that we are receiving, quote-unquote, "the truth", is that really one of the core, most dangerous aspects, would you say?

07:31 David Bray: If you define trust as the willingness to be vulnerable to actions of an actor you cannot control, we perceive competence, we perceive maybe integrity, we perceive benevolence in these models, and as a result, we are trusting them when, in fact, maybe we shouldn't. And that's partly because of marketing, that's partly because of what we've been told. But as he said, they appear human, and the more they appear human in their conversations with us and interactions with us, we don't step back and have the skepticism.

But even before this, there was actually, in the '80s, where they actually played the bargaining game, and the machine was pre-programmed to make offers. And so, it was never going to deviate from this, but people still penalized the machine for doing unfair offers even though it was pre-programmed. And so, this is actually more of a reflection of human nature, which is we try to treat things as if it was ourselves when we really need to step back.

And I often tell people, "AI is not artificial intelligence, it's alien interactions." Do not assume at any point in time is it thinking anything like us. And as a result, when you are looking at different AI methods, including generative AI, know their strengths and know their weaknesses, and brief your board, brief your leadership, brief your government, because those different strengths and weaknesses may be your comeuppance if you use the AI in the wrong context.

08:44 Tim Clement-Jones: David's so right about that, Michael. And what is so interesting is we defer now to these large language models. Look how polite we are. I mean, David may kick the computer when it gives the wrong answer, but an awful lot of other people say, "Oh, thank you so much. That's a great answer." And, you know, this polite interaction with a machine is, you know, quite extraordinary, but this is what we are now programming ourselves to do, basically.

So again, you know, we are building up this kind of false platform, really, and we should be much more skeptical.

09:20 Michael Krigsman: It does take on the character of a person because it presents the illusion of consciousness, of sentience.

09:32 David Bray: Yes. That's, I think, the biggest thing I want to tell people. At no point in time is it conscious or sentience. This is just fancy multidimensional mathematics. The trouble is, you do have some people that are, for whatever reason, either talking about it because it gets clicks or talking about it because it's feeling They themselves feel anxiousness. But you need to recognize at end of day that the current flavors of AI that we have are really just really advanced multidimensional mathematics.

And so when you deploy it, I think this does get to the interesting things. You need to be ready for it to both technically go off the rails because you may have used AI in the wrong context. You need to be ready for the fact that maybe the AI is doing exactly what the algorithm should be doing, but you trained it poorly on the data, that the data was non-representative, or there were things that were missing from the data. And as a result, it's going to reach conclusions that do not match what we want it to match.

Or finally, those two things could be done right, but what you missed was the human interactions as a result, and so you begin to see exactly as Lord Tim said, people who may be going to an AI model, asking it for news, and going down an increasingly mixture of some real news plus hallucinated news, or news that's being skewed to the person's attention or interests. And as a result, they believe this to be truth without going to other sources to do the sort of skeptical trust but verify.

AI Risk Management and Corporate Integration

10:51 Tim Clement-Jones: Absolutely, and it's very seductive. And then you get this overdependence now being created, and that's one of the things that I worry. It's the de-skilling aspects in the future as well.

11:03 Michael Krigsman: Preety Narayan on LinkedIn says, "Do you think we'll reach a point where AI risk officers are as common as chief information security officers? And if so, should they have a direct line to the CEO or even the board?" That's kind of a pregnant question. There's a lot there.

11:25 David Bray: One of the things that I'm increasingly hearing from CIOs and CISOs, chief information security officers, both in the private sector and the public sector, is we all know that in cybersecurity, complexity is actually what erodes strong cybersecurity. And AI, guess what? It's one of the most complex things we're bringing into enterprises, into organizations. And so, yes, in some respects, there is going to be a convergence between the things you need to do for cybersecurity, as well as for AI risk.

And one of them that I would say that we need to do more of already is move away from what was called forensic space, sort of like looking for very specific patterns of something. And instead, we need to do what's called patterns of life. What does normalcy look like in your organization? What does normalcy look like with this system? And then if you see something odd, that could be a hardware, that could be a software problem, it could be a cybersecurity issue, or increasingly, if it's something that has AI behind it, it could be the AI doing something it shouldn't do.

And so I actually think there's gonna be a convergence between patterns of life-based approaches for cybersecurity, as well as for what's needed to be done to make sure AI does not go off the rails for companies and for governments.

12:31 Tim Clement-Jones: Absolutely agree with that because I don't think in the future we're going to be able to separate out that risk role from, if you like, the cybersecurity role. After all, we know that AI can be a cybersecurity risk in itself, and therefore, the risks of AI start moving into that area.

But I do agree with our questioner about the importance of assessing the risk. It needs to be mainstreamed within a corporate structure, and it needs to be very, very close to the CEO, the audit and risk committee. It should be integral to the way that CEOs, the boards, adopt artificial intelligence. And if they don't have that approach to risk, then they are very vulnerable when they start adopting it.

So, you know, this is very much part of the structure that I think you and I would propose, David, for corporates to adopt. And, you know, my worry is that it's adopted piecemeal without that overarching assessment at the very beginning of what kind of structures you need to keep the corporate safe, to make sure that its purpose is being delivered by using these new tools. It's just a sort of random sort of idea that is going to improve productivity, and we're just going to install a few tools here and there. I think we have to be much more holistic about the way that it's delivered.

AI as Insider Threat and Workforce Adaptation

14:01 Michael Krigsman: Is there a greater danger from AI risks than traditional security risks? Security is now a board-level topic because of the potential reputational, financial, legal damage that can ensue. Where does AI stand from a risk standpoint and the prominence of the risks?

14:29 David Bray: My assessment of the current flavor of AI, the current generation we face, it's a lot like insider threat. Just like you need to be worried about insider threat where you don't know that human might do something that's maybe not malicious, but they did something wrong. They just sent 250,000 names publicly that should not have been named publicly. That's just an example. Or maybe they are malicious. Maybe they're actually disgruntled employee or they're actually getting paid to do something. Or maybe they're getting coerced.

And so you need to treat almost when you bring AI into enterprise, it actually needs to be a way to upgrade and you have the conversation with your board about, "We're going to treat AI just like we need to upscale our own insider threat issues." Because again, this AI, you know, if heaven- I would not recommend using DeepSeek, but if you're using DeepSeek, you should be aware that it's also sending metadata and information overseas, and are you comfortable with that?

So, I would say it is a risk. It is a significant risk at the speed and scale it can do. However, I think we have tools very similar to how we actually approach insider threats within companies. The same thing can actually be done for AI too.

15:34 Tim Clement-Jones: And I think the shrewdness of the question, Michael, is that I don't think many boards are there yet. I think that David's absolutely right about what should happen. But I don't think that is currently happening.

There's a great focus on what the opportunities are, but it's not really looking about the risk, how it fits within the risk appetite, because I don't think the risk is necessarily appreciated or the level of risk is not yet appreciated.

So, I think boards, I mean, my worry about boards universally is they don't necessarily have the skills yet to really assess at that level the kinds of AI that they should be adopting, how AI will fit with their values and their objectives and so on. But, you know, these are quite big issues across the board.

16:26 Michael Krigsman: Let's jump to Twitter and take some questions there. This audience is so smart, so intelligent. I love these questions. Here's one from our friend Anthony Scriffignano, who, of course, has been a guest many times on CXOTalk. And he asks this, "As a new generation is emerging that has learned to let the machine do the work, how do we get the workforce to use their excess capacity to think for themselves and focus on new unmet challenges?"

17:02 David Bray: There's two buckets. First is, as they are using the machine and working with the machine, as Lord Tim said, and I agree enthusiastically, still be skeptical. You know, just 'cause the machine has given you an output, you almost need to sort of double-check the work. And it doesn't mean you have to check everything, but you need to gain some expertise.

The second thing is we now need to figure out how do we actually sort of do a combination of not just one individual to an AI, but teaming across multiple people and machines. Because it may very well be the future is, especially for an entry-level employee, you may actually have either as a deputy boss or even as a boss, your initial boss might actually be giving you tasks through an AI system. Because what you're initially doing is the more, you know, sort of learning entry-level work, and you have to be comfortable with that.

But then what's your recourse if you think the AI is giving a wrong task, or the task is no longer fit for purpose? Because it might do that. And so, I think we actually need to actually begin very early on, maybe as early as middle school, exploring what does it look like to do combination hybrid human-AI teaming, and then figure out what should be the actions to both be skeptical of what's coming from the machine, so you triangulate it. And then, two, what do you do if you think whatever the AI is asking to be done or whatever task is being given to the team doesn't make sense? When does the human raise a hand and say, "I don't think that's right. I think something's wrong with the machine"?

18:28 Tim Clement-Jones: Absolutely agree with David. It starts really further down the pipeline at school in terms of the kinds of critical thinking skills that we need. The trouble is, we're starting from here. And therefore, corporates, you know, they need to assess the risk of, if you like, over-dependence on these tools where there is no proper human in the loop where there probably should be. There's an over-dependence on the tools, and there is no real human intervention, which actually could get the same kind of result.

So, the skills of individuals using these tools are going to be absolutely critical. And, you know, we no longer need now to amass information. We don't need to, in a sense, gather the data, or analyze the data in many ways. But drawing conclusions from it and critically assessing it is going to be absolutely, you know, right front and center for the kinds of skills we need in our businesses.

And, you know, again, it requires an understanding at the top level of our businesses as to what, you know, can usefully be done by humans versus machines. And I don't think that understanding is quite there yet.

And I think, you know, David, you do a lot in this area. And it's really a question of bringing that right front and center. I think this idea that we're all gonna have a great deal more thinking time sitting there in business and this is going to allow us to dream up new creative thoughts, I think might be slightly optimistic.

AI as a Sparring Partner and Policy Implications

20:08 Tim Clement-Jones: But I think it is gonna be a partnership because, you know, what they talk about AI in the legal industry now is the sparring partner. And I think that's a lovely analogy basically, where you're testing propositions, and you're kicking the machine and saying, "Well, what about this? What about that? Can we do it better this way or that way?" And you have somebody that you can argue with. So, I quite like that.

20:33 David Bray: I think Lord Tim hit it spot on, which is there's actually some articles that say the reward for your AI partner is more work. You know? So it's not necessarily gonna free things up.

I love the sparring partner example. Partly, well, so in January of this year, the US administration pushed out some export controls, and they rushed them out the door before there was a change in leadership. I kind of felt like they were rushed out the door. And so, I actually went to an AI model, a GPT, and I said, "Pretend you want to get around these export controls. And you not only want to get around them, but you wanna make money." And sure enough, guess what the AI did? It told me five ways to do that.

And so, exactly that, as we look at different policy things, or we look at different decisions, we should at least red team them. Red teaming coming from the world of, what could possibly go wrong, or how might they be misused or abused? I think this is increasingly where, again, but it's not that you turn to the AI and just turn it all over. It gives you thoughts almost as a sparring partner, as Lord Tim said, and then you as the human adjudicate, "Yes, that makes sense," or, "No, that couldn't possibly happen."

21:35 Michael Krigsman: This would be an excellent time, folks, to subscribe to the CXOTalk newsletter. Go to cxotalk.com, subscribe to our newsletter, so we'll notify you of discussions like this, so you can participate. Do it now.

AI Regulation and Public Trust

21:52 Michael Krigsman: So here's a really interesting question from Arsalan Khan. When He's talking about the US, but I think this is broader. "When the US," he says, "removes AI guardrails, is the US population or the world really ready?" Tim, I think this one's tailor-made for you.

22:10 Tim Clement-Jones: Although the US, I mean, individual states are different. I mean, obviously we've got some regulation in California and Colorado and some other states. But by and large, we don't think of the states as having federal legislation on this. Although I see that even the president has expressed some frustration about that. And I wonder whether federal legislation isn't coming down the track.

But yes, I mean, I think that there should be a desire on, if you like, the ordinary US citizen to have standards that are universally applicable to the kind of large language models that we've been talking about. And there are those international standards. And interestingly enough, you know, US institutions like the National Institute for Standards and Technology have developed some very sophisticated standards as well, which could be adopted.

So, you know, if the willingness was there to legislate or regulate, it wouldn't have to be very complicated. And I think it would provide a much greater degree of public trust, which is the trade-off. You know, I'm a great believer in having a level of regulation which doesn't impact on innovation because it creates public trust. But you know, I think there's still quite a long way to go.

And, you know, the trouble is that we risk having different regulatory regimes across the world. So, my view is that what we need to try and do is agree international standards, which mean that we can adopt these tools universally across the board. And then we don't have to sort of think jurisdiction by jurisdiction in all of this.

But if we don't have regulation, then of course that puts the onus back on business to make sure that we are kept safe and we can trust the AI tools that are being used. And, you know, the more, the less regulation you have, the more responsibility business has in a sense.

24:11 David Bray: It's not so much that the US has removed guardrails. It is that they have not enacted anything at the federal level that actually is a legislative guardrail. Now that said, just this week, on Tuesday, we did see a new executive order and some additional strategies come out on AI. And if you read them, they actually call for NIST to be involved. And so there actually is movement this way.

I think it is interesting though, because it could end up be that the US right now, for various reasons, is going to lean more through executive orders and through executive policy versus congressional. The other thing that I also would raise, at least from a US perspective, and this is just one way to go about doing this, is that we do have existing laws in the healthcare sector, including HIPAA. We do have existing laws in the banking sector, including the Bank Secrecy Act.

It could be that we actually see Congress, instead of trying to do one monolithic AI legislation, which might prove difficult for various reasons, may have the different committees go for an upgrade. What does AI mean for in the healthcare space? What does AI mean in the defense space? What does it mean in the banking space? And so the US may have a slightly different approach than what Europe has done or other countries, but it is the case that, yes, we have not done anything at the federal level legislative yet. But as Lord Tim said, stay tuned.

AI Risks and Corporate Governance

25:27 Michael Krigsman: And this is from Kurt Milne, who says, "Will it take a material public failure for C-level teams to shift focus from AI opportunity to also focus on risk?"

25:42 Tim Clement-Jones: Sadly, that is probably true. I wish it wasn't. I wish we didn't have to have, you know, a big data breach for people to get excited about cybersecurity. Or, you know, some of the other crises that have caused corporates to change their behavior. We I don't think we've yet had a major scandal involving AI.

I mean, you know, I suppose it would be where, you know, a company's marketing material was entirely created by AI and it turned out to be, you know, completely eating some creative's lunch or something. I mean, I can't quite envisage the kind of corporate earthquake type of disaster that would happen.

But I That may well be right, but I very much hope that the work of people like David penetrate the corporate world so successfully, Michael, that we don't have to have a crisis that causes behavior to change. I'm a believer in voluntary corporate governance if that can be achieved.

26:52 David Bray: It's a tag team effort 'cause what you're doing too, I remember we met in 2017 and you were chairing the UK's strategy for AI. So you were well ahead of the curve as opposed to some things I could say more in the US where we're still behind the curve.

But the other thing I would say, Michael, is there already are some examples where people have used AI for helping to write code. Some of the code that the AI suggests compiles and some of it does not compile at all. And so that should be warning indications, do not blindly trust the machine.

But even more interestingly enough, and I think this was in the last week or so, unfortunately, one of the AI agents that help with writing code completely deleted a company's code library. So that should tell you there are things that are warning clouds on the horizon that if you do not think about a risk-based assessment and you put this in charge of something that's very tied to your intellectual property or tied to your finances, bad things can possibly happen.

27:49 Tim Clement-Jones: I should mention the whole area of copyright infringement, Michael, has become incredibly controversial in Europe. And you could say that, you know, for reputational purposes, it's the AI model developers which are at risk of very, very strong reputational damage if we don't find a proper solution to the, you know, training on copyright material and so on.

And then, of course, earlier on, David mentioned the lawyers who, you know, had false citations. And in that, you know, I mean, the legal profession now, they've wised up pretty quickly to the fact that they need to have closed systems which can be relied upon. Not just asking ChatGPT to give you a, you know, a legal pleading. That is highly dangerous. So, there's already been quite a bit of wising up.

28:41 Michael Krigsman: Let me just also mention that it's interesting if we look at traditional security and data breaches, there have been massive data breaches. I think all of us have had our Social Security numbers and personally identifiable information, you know, spread out there on the web. There's no escaping it. And still these breaches happen all the time.

29:06 Tim Clement-Jones: One of our major retailers, Marks & Spencer, you know, they were unable to stock, unable to deliver, I mean, you know, for months and months. And I think they're still suffering from the aftereffects. So yeah, I mean, risk management, risk assessment is, you know, so crucial in all of this. And, you know, AI needs to be added to all of this. And our first questioner rightly raised that. It needs to be very much integrated into the whole risk management process.

Agent Management and Spatial Web Protocols

29:40 Michael Krigsman: Here we have a question from LinkedIn. It's from Mohamud Jibrel and he says This is to David. He says, "Do you see the need for new agent management tools and platforms? For example, managing agentic AI sprawl with security, compliance, cost, and so forth."

30:01 David Bray: Short answer is yes. The slightly longer one is it may not look like some of the tools of the past because this is not steady state. Because we just talked about how generative AI and other AI approaches are possibly non-deterministic and that they are generating their own content, if you do what has been done in the past with forensics and you assume steady state, you will always be behind.

And so that's where I'm really excited about sort of patterns of life where CIOs and chief information security officers can go to their boards and say, "Look, we can actually get a win-win where we upgrade the cybersecurity posture of our organization while simultaneously also getting it better, in better shape for managing AI risk and actually detecting AI risk." Because we're looking at what the normal patterns of life are for our information systems and our AI systems and then if we see something odd, we're stopping it and doing a pause, and trying to figure out what happened here.

So, I think that's necessary. The other thing I would also give a nod to is, for certain flavors of AI, especially the ones that are pre-compute, IEEE, after about five years, released just about a month ago, called the Spatial Web Protocols. And just like how everything on the web that we use HTTP to find things on the web, and we use HTML to actually describe a webpage, now it's actually- you can actually use what's called HSTP, which actually allows you to basically address anything in space and time, almost as if it was like a web domain or web URL. And you can use HSML to actually write things up, and bound things by space and time.

And so, now what does that mean for AI? We can actually write descriptions and say, "In general, autonomous AI systems that are flying planes should not plow into the ground. In general, cars should not actually drive into other cars." You can actually begin to have space and time constraints.

AI in Space and Business Applications

31:45 David Bray: And even more interestingly, there's some early indications that you can actually take that and actually use it for policy as well, 'cause policy often has boundaries. JPL Jet Propulsion Lab with NASA has been using this as experiments on NASA on the moon, to actually, because of the lag effect, giving these sort of like, the commander's intent to the possible moon rover or whatever is on the moon, but then allowing the AI that's on board actually on something that would be on the moon, try to navigate within the spacetime dimensions.

And so, this gives me hope that for some of the future of what we do with AI, especially if it is pre-compute, we can also bound it by space and time constraints.

32:22 Tim Clement-Jones: The other element of this is absolutely right, at the corporate level and the adoption level by business, but of course, you know, agentic AI, some of it's snake oil, some of it not, has that autonomy, but it also is now becoming retail as well. So, you know, again, we have to make sure that, you know, if a business has a thousand employees, they're not all thinking, "Yeah, I'm going to adopt this agentic AI. I'm going to put all these tools together in a semi-autonomous way," because that is going to be a threat to the business as well. Not just cybersecurity or data risk, it's, you know, decision risk.

There are so many aspects to this, and so it doubles the risk as far as I'm concerned, where you have that degree of autonomy in an AI agent that, you know, could unravel an awful lot of things further down the line. So, you know, again, that means that boards and senior executives need to be very, very mindful.

AI Policy Creation and Regulation

33:27 Michael Krigsman: Arsalan Khan comes back, and he says; this is really intriguing; he says, "Why can't we have AI write its own policies?" What are the pros and cons of the AI deciding the policies, and implementing the policies?

33:43 Tim Clement-Jones: There's no reason why you shouldn't. As long as you've trained the AI in the right kind of a way, you can be a monitor, you can You know, I think you have to have some sort of generative adversarial kind of approach to it. You can't have I don't think the same, you know, AI should be looking at itself, but I think what you need is one AI looking at another, and that is going to become extremely prevalent, in my view, where you have a mixture of tools. One is monitoring another, one is risk assessing another.

You know, after all, we know how quite a lot of work has been created. I remember when there was a painting in Paris that was then sold in, created in Paris, sold in New York for half a million dollars, called, I think it was, Eduardo de Bono, and it was created by this generative adversarial network. And it was created by one AI criticizing the work of another, until eventually it produced this rather splendid portrait.

Well, you know, let's take that forward a few years, and say, "How do we make sure that, A, we create the policy, and then make sure that it's complied with?" So yeah. I mean, I think the chief compliance officer has already got tools of that kind in the box.

35:02 Michael Krigsman: This is from Chris Peterson, who is maybe taking a slightly contrarian view here, and Chris Peterson says, "Is there a balance we can strike between AI regulation and AI innovation?" And here's the point, "Especially given the velocity of change, and the billions and trillions of dollars riding on the success of big AI businesses?" In other words, "Sure, guys, we have to manage risks, but let's also not throw out the magic as we manage ourselves risk into the ground." Thoughts on that?

35:42 David Bray: If you use, whether it's human-produced or you use another AI to produce constraints for what an AI should and shouldn't do, then you allow innovation to happen within those constraints. And so if you have a separate system that is monitoring that harm is not coming, either mentally or physically, to a human being, then yes, that gives you an additional degree of confidence that you wouldn't have if you had just the AI system by itself.

You may tell the AI system, "Don't harm humans. Don't harm them mentally or physically," but as we know, unfortunately, whether a wrong prompt is asked or the data was incomplete, or the non-determinism sometimes, things can happen.

So, I think when I brief companies and boards and even governments, I say, "You know, what you really should be doing is not necessarily setting five-year goals, but setting five-year visions for the how you want the technology and society or technology in your company to perform, and then use constraints as your friend." And so just as what Jornsing was saying, that you could use AI to actually sort of bound in other AIs, there are other approaches too, and we've seen already, I've already seen there's some certain companies that are now using AI to help you coach to do better prompts to an AI.

And so that's already a proof point that this is possible. So, I think at the end of the day- You definitely can have both. And given that, and as Lord Tim said, you know, regulation will always be playing catch-up with the technology. By using constraints as opposed to setting very fixed things, which actually might be either out of date or restrict innovation. If you do constraints, that allows you to still protect the things that need to be protected but allow people to explore that space as well.

37:18 Tim Clement-Jones: I've never believed that good regulation is the enemy of innovation. I've always thought of it as actually the friend, because we want the right kind of innovation that benefits humans, that doesn't substitute machines for everything that we do, that is for our benefit, and doesn't create some of the harms that we've been talking about today.

And I think the way you do that is the way you set the principles out of the kind of AI that we want to see. And then we try and adopt standards, mandate particular standards. And we've got to be agile in the way that we do that. And the great thing about standards is they can change over time, you know? And we talked about NIST earlier, but there are many other organizations, IEEE, which David also mentioned earlier, ISO, the International Standards Organization. And all of them are trying to move towards interoperability internationally.

And for me, that is, I think, the goal. If we can get that and we can mandate that sort of quality into large language models, into other forms of AI, then I think we've made some real progress. But I think the idea is that the idea that no regulation means we can just have a complete free-for-all and produce any kind of innovation, whether it's good or bad for humanity, I think is what we need to push up against.

38:44 Michael Krigsman: Is there a distinction between this balance of regulation versus innovation, when it comes to generative AI versus any other new technologies? Tim, any thoughts on that?

38:59 Tim Clement-Jones: I always use the example of the automobile. I mean, you look back at the automobile, what created the growth? I mean, it was rules of the road, it was manufacturing standards, you know? And it was ten years when you look at Fifth Avenue photographs, 1903 to 1913, that came in pretty fast. You know, it was horse and carriage in 1903, automobiles except for one horse and carriage in 1913 going up Fifth Avenue.

So we'd been here before, and we didn't just allow, you know, the automobile to run everybody down on the roads. Of course, we started with a red flag, which was probably, you know, maybe safe in the circumstances, but very soon, we got sensible regulation and, look, how innovative has the automobile industry been over the last hundred years? Enormously. So I'm actually optimistic that we can get the right form of regulation without stifling progress and innovation.

Opportunities for Smaller AI Players

40:02 Michael Krigsman: This is from Chris Davidson. And David, he says On LinkedIn, he says, "Most of the conversation around AI centers on innovation from big companies, OpenAI, Anthropic, Microsoft, so on. As an equity partner in a new AI integration startup," he's curious what you have to What you think are the biggest opportunities for the little guy in the marketplace. In other words, what are the areas of value on which smaller companies can stake a claim with minimal risk that they will get crushed by these big players in the next three to six months?

40:41 David Bray: So I'll give three real quick. One would be, while these big players are focusing on generative AI, and generative AI is actually using algorithms that, you know, deep neural networks that goes back to late '80s, early '90s. So it's a technology only possible today, but it's not necessarily new. I would say, look at additional approaches that are coming out, work of Karl Friston on active inference. I mentioned that that's actually something that can be pre-compute and be bounded.

And I think the future is not gonna be one AI method to rule them all. It's gonna be mixed model. I mean, we know that's actually what happens with us, is that you use different things. And we may find that generative AI is really great for replacing natural language processing, but as you see with the papers that are coming out, these models are not reasoning. They're only doing pattern matching and to the degree that pattern matching and reasoning matches are doing that. So first is, there are other AI approaches that are not getting anywhere near the oxygen that they need, partly because generative AI has consumed so much.

The second though is, we need AI at the edge, including generative AI at the edge. And so while these companies are selling you sort of the monolithic platform, you got to subscribe to their services, I want something that I can run on my phone. I want something that I can run on my laptop without it calling back. And so it's got to be able to operate in low, low, low bandwidth environments and low processor environments. And you've seen, you know, DeepSeek, that was example of distilling a model.

AI Innovation and Edge Computing

42:03 David Bray: I'm not necessarily saying use DeepSeek. But you also see that both Berkeley and Stanford took ChatGPT 3.5 and they went to LLaMA, the open weight model, and found ways to actually ask ChatGPT how to upgrade LLaMA to be almost as good as ChatGPT 3.5. So we need more innovation about AI that can run at the edge in disconnected environments or low processing environments, 'cause at the end of the day, some people may not want to connect back to the cloud.

And then finally, this is actually more the human dimension, that right now, some of these companies I really try to have empathy for the fact that there are some companies out there saying like, "It's going to displace 50% of jobs." Well, actually, I don't think that's the case. I think while jobs may be lost, people will retrain, and so it's actually an either/or.

And so what I want to see is AI companies and AI startups that are actually helping people navigate the chasm between, "You used to do this. Now you're going to be doing this with AI. How do I accelerate your learning journey so that you're" Yes, your job has been let go, but you're now doing a different type of job and you're actually much more productive. And so navigating that in a way that gives people agency as well as reducing anxiety. I think there's gonna be a whole industry there.

And I think right now, anyone who's just saying it's just gonna be a loss of all jobs, period, without saying, "And at the same time, new jobs will be created." I'm really interested in the new jobs that'll be created and how we actually have private industry help people skill up for that.

Future of Jobs and Data Stakeholderism

43:28 Michael Krigsman: Lord Tim, thoughts on, very quickly, on this issue of opportunity. How can small players survive?

43:35 Tim Clement-Jones: Yes. Very quickly, I thought David's point was very shrewd. My point would be, actually, data is still going to be king in all of this. The large language models are a commodity. I don't think they've made any money yet from the large language models.

So, my view would be, I would go to open source or Distill, in the way that David discussed, and find a proprietary database that I had an exclusive license to, which had a particular use, maybe in healthcare or education or agriculture, something like that. And I was the only AI developer who had the access to that data and was able to use my AI model. And I would get an open-source model to help me with all that. And I think then I would be in business. But, you know, I'm not an AI developer. What do I know?

44:28 Michael Krigsman: We need to spend a few minutes before we're done here talking about jobs and the impact of AI, generative AI, on jobs. David, do you want to take some crack at this first?

44:44 David Bray: When I met Lord Tim back in 2017, what really resonated at the time with the UK government, they were proposing data trust. Here in the United States, we call them data cooperatives. But this idea that businesses or people could come together, like maybe we're all musicians or we're all artists, and we actually say, "We are willing to let our data be used for the following purposes." Well, here we are now in 2025, looking at 2026. We need that in the United States.

I mean, 'cause I actually say right now, we have a few companies that seem to be repeating the lessons of Napster, where they have acquired data, but not necessarily respected intellectual property, not necessarily respected the equity of the people that produced it. And it also makes it hard on whether or not I can trust it, 'cause when the machine gives me an answer, where did it come from?

So, I think this does put the premium on exactly what Lord Tim said, which is the future of jobs is, whether it's an interesting data set or it's people that are musicians, artists, entertainers, or maybe it's us and we just care about finding a cure for Parkinson's. On top of it, though, we no longer have to ship the data. One, that's a cybersecurity risk, but then two, what's called federated learning, where the algorithms actually come and learn in situ, on the data itself, and we can actually monitor what's being learned. And then when it leaves, we actually record that, and then there's some either financial exchange of value or some non-financial, some other benefit that is equity in whatever cure for Parkinson's or whatever research was done.

I think the future of jobs is both rethinking data so that we can actually make sure people have some sense of stakeholderism in their data. And then, again, it's getting out there and saying this whole narrative of, "It's going to kill jobs, period," as opposed to, "It's gonna displace jobs." Much like how when the automobile showed up, well, yes, horse-drawn carriages and those sorts of things, they went away. But then there were taxicab drivers, there were truck drivers, there were people that maintained cars. New jobs were created.

And so the faster we can actually help people both be aware that new jobs will be created and then help them with that journey, I think we'll actually reduce some of the anxiety that's present in societies at the moment.

46:44 Tim Clement-Jones: I take that data communities aspect extremely seriously. I think valuing our data and making sure that collectively, in the way that David describes it, we can make use of that and value it and monetize it, is going to be very important. I chair an authors' licensing and collecting society, and I can see the equivalent happening in all kinds of areas, not just the creative industries. But, you know, farmers collectively, for instance, with their data and being able to use that. And, in a sense, parlay, you know, license it. Make it valuable for use by AI developers.

So, I think entirely right. The second thing I would say is that I think that human empathy will still be a requirement. I think those human creative qualities are still going to be important in the future. So, but And then, of course, what we already mentioned is that critical thinking that's needed. Because if we don't think critically, then frankly, AI is definitely going to take over, and we're not gonna have really much of a function, particularly if AGI comes along anytime soon.

47:59 Michael Krigsman: Do you have thoughts on what kinds of jobs will get displaced by AI, and what kinds of jobs will not? And very quickly, please.

48:10 Tim Clement-Jones: If you can stick into Claude or ChatGPT, you know, "Tell me, give me a business plan for the next five years, you know? And this is my vision, and these are the annual reports of my competitors." then, you know, I think you're, and that's exactly what you can do nowadays. then you're gonna find that kind of job going.

So, I think it This is a white-collar revolution, basically. And I don't think we've been here before in anything like the same way. And it's not a blue-collar revolution that David was talking about, drivers and so on. I mean, you know, we have a gig economy, but that doesn't And they're driven by algorithms in terms of their performance, but that doesn't mean to say that those jobs are going. People are still gonna be riding around on scooters and motorbikes in the future. We're gonna be the ones The professionals are the ones who are at risk.

49:04 Michael Krigsman: David, literally in one sentence, what jobs will leave and what jobs will remain? Being displaced or not?

49:10 David Bray: So much like the printing press where scribes went out of work, but typesetters arrived, if you were in the business of web development, interface development, that'll increasingly be done by AI. Anything that the past is like the present, that'll be done by AI. The new jobs will be the jobs that are novel, where the past is, you know, past does not inform the present, where you have to think on your feet, where each day is different, and the people that are then, when they look at what comes out of the AI, are being those skeptics, being those critical analysis.

Ethics, Misconceptions, and Trust in AI

49:41 Michael Krigsman: Arsalan Khan says something interesting, "Whoever defines ethical boundaries defines the direction of AI. If the world doesn't have consistent ethical boundaries, then AI is just a reflection of our own biases." Who wants to just very quickly take a crack?

49:59 David Bray: That's an interesting statement, but let's think back to the 1700s, Arsalan. There were plenty of things that people thought in the 1700s that were ethical that we would not think are ethical now. Same thing in the 1800s. I often say to my British counterparts, in World War I, the British thought that Q-boats were ethical and submarines were not ethical, and switched to World War II, that had flipped. Ethicals are socially and temporally defined.

What I think he's saying, though, is what are the principles we want to hold true to? And I think that is an important conversation to have. But recognize that ethics, you know, there are plenty of things we thought 200 years ago were ethical that are not ethical now. Let's make sure that as we look at ethics now and we spend time on it, maybe what we really should be saying is, "What are the principles and the constraints that we care about?" As opposed to just ethics.

50:44 Tim Clement-Jones: There are principles that have been agreed internationally. They may not agree in terms of regulation, but the OECD principles, China signed up to those, you know, G7, G20, so I think I'd be more optimistic than Arsalan.

51:00 Michael Krigsman: Looks like Wei Wang from LinkedIn is gonna get the last word here, and I'll just ask you each for one quick soundbite sentence. She says this, "When there is a gap in popular perception and the technical reality of implementing reliable, reliant AI solutions, what advice do you have for teaching and explaining clearly the misconceptions and to build the rigor into helping convey the complexity?

51:36 David Bray: So, it sounds like the individual was trying to make the case to either like a board or a C-suite or whatever, and I would say, bring examples, 'cause there are plenty of examples, but also let, you know, and I wouldn't necessarily do this for the board 'cause they may not have the bandwidth, but for your employees and your staff, let them play with the technology and see when it works and when it's fallible, 'cause then that actually makes it much more real and it's visible.

However, I think this is a case where, again, whether it's case studies or experiential itself, you almost have to sort of show and make it real, because otherwise, people will not see the value, and again, that fact that we see the machine and we think it must be like us because it's talking to us or it's giving us text, we need to recognize that our own human emotions will run the risk of missing the fact that we have to be skeptical of these alien interactions.

52:22 Tim Clement-Jones: We are gonna be the victims of our own trust if we're not careful, and, you know, I'm a great believer in encouraging a culture of curiosity, but I think we have to encourage a culture of caution as well. I think we just have to get the balance right between the two, and if we can encourage that, and it's not just business, it's government, it's families, you know, and we're way behind the curve, unfortunately, because technology is moving so fast.

52:50 Michael Krigsman: Well, with that, we're out of time, and clearly there's a lot more to discuss, so I hope you'll both come back and be guests on CXOTalk in the future so we can continue this conversation.

53:03 David Bray: Thank you very much, Michael. It was great to be here.

53:07 Tim Clement-Jones: Truly appreciate it, Michael.

Closing Remarks and Gratitude

53:08 Michael Krigsman: And a huge thank you to Lord Tim Clement-Jones and Dr. David A. Bray, and to the amazing audience who asked such excellent questions. Thank you all.

53:21 Michael Krigsman: We have incredible shows coming up. Check out cxotalk.com, subscribe to the newsletter, and we'll see you again next time. Take care.