AI Misadventures and the Adversarial Economy

Introduction to AI and the Adversarial Economy

Michael Krigsman: Every time AI enters the real world, it meets resistance, competition, shifting markets and human behavior. I'm Michael Krigsman, and this is CXOTalk episode 890.

We're discussing AI misadventures and the adversarial economy with prominent business leader Steven C. Daffron and Anthony Scriffignano, one of the world's top data scientists. Let's get into it. Gentlemen, welcome to CXOTalk.

Stephen C. Daffron: Thank you, Michael.

Anthony Scriffignano: Thank you.

Michael Krigsman: Steve, tell us about your work.

Stephen C. Daffron: We are a private equity fund that focuses on financial technology. AI, of course, is a major part of that.

We have a model – as investors, of course, since we're a private equity fund, but also as operators, since our fund is made up of people who have actually built and run companies. And we're innovators. We actually think hard and spend money on trying to be at the innovative edge of where financial technology is going.

So AI is literally in our bloodstream, and we work through it pretty hard.

Michael Krigsman: Anthony, tell us about your work.

Anthony Scriffignano: Right now, I am a Distinguished Fellow with the Alfred P. Loomis Innovation Council at the Stimson Center, a Washington, DC think tank focused on what I would call action research. So not just writing white papers, but actually doing stuff that matters – all in the “for good” category.

And I’m also involved in some space-related ventures.

Michael Krigsman: When we talk about the adversarial economy, what do we mean and how is it being shaped by AI? And maybe you can share your views on this.

Stephen C. Daffron: Our economy is adversarial because we choose to make it adversarial. Rather than fostering cooperation and mutual benefit, we teach each other – and I've been working for 50 years, I see how these things develop – we teach each other to exploit vulnerabilities and weaknesses in systems or individuals so we get an advantage.

It’s not new. If I hear from members of Wall Street… remember Gordon Gekko, “Greed is good”? So competition is good, right?

Anthony (interjecting): So competition is good.

Steve (rejoining): Right, within limits. Philosophically, there are limits to how that adversarial economy works. Competition taken to an adversarial extreme is not good.

Aristotle said competition can lead people to positive outcomes – honor and excellence – and motivate them to strive for the common good. But he also said in excess it leads people to self-love and a disregard for fairness.

Well, candidly, AI makes that adversarial economy easier. When AI makes this adversarial mindset a constant, pervasive presence – making everything feel zero-sum – we can see it. No one trusts anyone. AI reinforces that. It makes the adversarial extreme easier to accept because it puts it at arm's length.

You can see it: look at all the places in digital markets where AI is used to bias data structures or manipulate algorithms. You want a list? Just Google “adversarial attacks on financial reporting.” Google that and watch the list scroll.

We've made it so that AI actually makes the adversarial economy easier – because it puts it at arm's length and it doesn’t feel as personal when an algorithm is driving it. It becomes even more endemic – and the broader public will see this when you have chatbots deliberately designed algorithmically to capture user attention and then use that attention to extract their vulnerabilities so we can manipulate their responses or behavior.

That's adversarial to the extreme. Now, we choose to do it. We operate in this adversarial financial-economic ecosystem – there's no doubt about it. There are misaligned incentives, there is heightened competition. We have to choose how to manage that. Those of us in this space can make those choices.

Michael Krigsman: Anthony, thoughts on this?

Philosophical and Technical Insights into AI Misadventures

Anthony Scriffignano: You know, one thing that's not 100% clear to me, Steve, as you're talking – is this fundamentally good or fundamentally bad? And maybe I would add: how is it fundamentally different now that there's AI in the room?

Steve (rejoining): Well, fundamentally… Again, I still like philosophy as much as I like data science. My favorite philosopher, Aristotle, would say: “In moderation – at the median – it’s good, because you need competition to move the world forward.” It’s bad if you allow it to go to either extreme – to have no competition, or to have too much.

And what AI does is allow us to take that extreme competition and move it further away where we don’t feel it. “We didn’t manipulate the chatbot to get Grandma to share her intimate details with us so we could exploit her bank accounts. We didn’t do that – the algorithm did it. AI allows us to be adversarial but not feel like we did it.” That’s not good.

Michael Krigsman: Anthony, let me ask you this: how does AI depersonalize our psychology – the psychology of participants in the economy – so that, essentially, we can screw each other over and feel good about it?

Anthony Scriffignano: AI doesn’t care. It’s a bunch of math – math designed to achieve a certain goal – and we tend to anthropomorphize it. We talk about what it wants and what it’s doing. Part of the problem is we don’t have the right language.

If we talked about it “convoluting” instead of what it wants, people would very quickly fall asleep. So we have to be careful about what’s happening here.

We used the term… Actually, I think you used the term in the title of this episode: misadventure. And I like that term, because misadventure can mean many things. An AI person will say, “Oh, you’re talking about hallucination. Let me tell you about hallucination.” By the way, “hallucination” is just a fancy word for when AI does something you didn’t expect and don’t think you asked it to do. Sometimes it turns out you did ask it, and didn’t realize it.

So certainly, Steve is talking about algorithmic bias. A lot of times these algorithms – the mathematical equations and processes under the AI – are trained on a certain type of data, and then you let them out into the wild and the real world doesn’t look like the training data. They go ahead and start doing things based on what they were trained on, which is inappropriate. Sometimes they even consume their own output as input – you get these recursion loops. (This is sometimes called overfitting – when you have the wrong training data, and then the model starts feeding its own outputs back in, it becomes increasingly confident it’s correct because it’s heard it before… not realizing it’s only heard it because it said it.)

But you were going down… Let’s go back to this: it’s true and it’s not meant to happen, but I’ll tell you that in many cases, the adversarial nature of this is: if we can design something that allows us to extract more information – even if it’s information we shouldn’t be extracting – we tend to do it.

Stephen C. Daffron: Absolutely.

Anthony Scriffignano: It’s adversarial because if we can do it… if we don’t do it, the competition will. So, let’s talk about two types of adversaries here.

Anthony Scriffignano: One is adversarial in the sense of being antithetical – it’s the AI doing something we don’t intend or want it to do, and we want to get there first (or do it better than it does it), right?

Stephen C. Daffron: Yes.

Adversarial AI and Ethical Challenges

Anthony Scriffignano: Those are actually two different flavors of the problem. Another kind of adversary is more subtle: if I understand what your algorithms are doing, or even just what type of AI you’re using, I don’t have to invade your systems. I don’t have to invade your turf. All I have to do is poison the “milk” that you’re using to make those decisions – and that’s another type of adversary here.

There’s misinformation, disinformation… You can hide the data that would allow the AI to reach the conclusion it needs to reach – basically prevent certain types of data from getting into the algorithm. There are so many ways to manipulate this, which begs the question: how would you even know it was happening without using another AI? So this is definitely a big, big kettle of fish you’ve opened up, Michael.

Michael Krigsman: We have a very interesting question that’s come in on Twitter from Arsalan Khan – he’s a regular listener and always asks provocative questions. He says this: “In this adversarial world, who is right or wrong depends on who are the gatekeepers and who sets the guidelines for the guardrails. How do you understand what is rogue AI and who is not – what’s not?”

Anthony Scriffignano: I would even double down: it’s not as binary as that. Very often, there are different regulating authorities – different “gatekeepers,” whatever you want to call them – that want different things. So you can’t actually be “right,” quote-unquote, with respect to all of them at the same time.

One example: Steve’s talking a lot about harvesting information – that’s one goal, right? Another goal might be privacy. I might want to harvest your information so I can customize the app and improve your experience, but I also want privacy. Well, those are opposite things to want, and you can’t satisfy both at the same time.

There isn’t a single entity who says, “This is where the dividing line is.” There isn’t someone declaring, “These guardrails are the correct ones and those are the wrong ones.” Part of it comes down to choices – the choices each company, each developer, each CEO makes.

Balancing Ethics, Privacy, and AI Development

Stephen C. Daffron: Let’s balance an easy one – not so easy, perhaps, but straightforward to understand. Think about the privacy of the individual versus knowledge about the individual that we can develop and then extract rent from that individual.

If we allow AI developers to extract the maximum amount of information and use it to design exactly the right triggers to get the desired response from that consumer, that’s good for the company and probably good for the company’s bottom line. But it’s not necessarily good for that consumer. And it’s certainly not good if I use that data to predict that you’re about to commit a crime that you haven’t committed yet.

Michael Krigsman: Oh, like Minority Report!

Stephen C. Daffron: Here we go again, right? So there are lots of ways you can take this too far. There are certainly ethical principles of AI – the OECD has a well-known set, for example – and lots of other sources where smart people have gotten together and said, “What do we, as a collective group of experts, believe we all can agree on?”

Well, it turns out universal agreement doesn’t really exist, because depending on where you are in the world, there are countries that value national security over personal privacy. There are places that are completely capitalistic. There are others where people demand the right to be forgotten. Those are all completely opposite values to hold.

Philosophical Perspectives on AI and Responsibility

Michael Krigsman: All right. Kent Sparks – who is Provost and Vice President for Academic Affairs at Eastern University – says this: “From a higher education provider, I applaud your thoughtful concerns about the ethics of AI, which add steroids to the adversarial dark side of our competitive system.” And here’s his question (and it’s a good one): “Can these issues really be tackled effectively apart from political solutions?”

Anthony Scriffignano: Tackling them apart from politics is nearly impossible, because there’s politics in everything. But I would say: whatever you do, you should do it on purpose.

I spend a lot of time with academia – I try to be a good counselor there. Some very big questions right now… For example, what do we even teach that will be relevant by the time these students graduate? How do we understand provenance and permissible use in the context of peer-reviewed research when the “peer” doing the review might now be an AI agent?

There are some really big questions we don’t have answers to yet – but there’s also a huge opportunity cost. The cost of doing nothing is not nothing – if you stand still, you’re going to slide backwards farther and farther.

So we have to be good stewards of this amazing technology – and we have to do it deliberately, on purpose. We will not be perfect, and there will be politics.

My approach is not a “solution,” but it’s a recognition that each of us has to develop our own philosophical perspective. That worldview is what should guide our approach.

Individually – as well as collectively – we choose to make the economy as adversarial as it is. We can also choose to do the right thing. When there’s data that can be made available for the right reasons – to feed starving children, to ensure people don’t get sick, to give people the healthcare they need – and we choose to do that, that’s a good thing.

Choosing at the same time to use AI in ways that exploit people or hurt people… It’s not always that complicated. Sometimes it just comes back to philosophy – back to Aristotle’s “golden mean.” Find the middle. Too much competition – that is, allowing any company to do anything as long as it improves the bottom line, even if it hurts people – is too much. Too little competition and we have nothing to drive the economy.

Finding that middle ground, and being willing to say “This middle ground is the right mix of good to move us forward” – and sticking to it. And how do you teach that? Well, frankly I think you teach it with some philosophy, not just math. I like the math part, but you also need the philosophical part – understanding that what you do has consequences.

One of the most important questions you should ask whenever you use AI is: “What do we have to believe in order to do what we’re about to do?” – and then do it deliberately, on purpose.

Michael Krigsman: Are we having a technology discussion, or are we having a discussion about one’s viewpoint on self-interest? I mean, what does any of this have to do with AI?

Anthony Scriffignano: AI is embedded in everything we do right now. The words we’re speaking are being transcribed by something, and they’re being synthesized and inferred upon while we’re speaking them. So ignoring the technology behind the scenes is foolhardy.

However, if you only lead with the AI – if you run around with your AI hammer asking, “What can I hit with this hammer?” – that is equally foolhardy. You have to do both, and you have to do them at the same time. You can’t ignore either side of this equation.

Challenges in Designing Resilient AI Systems

Michael Krigsman: Let’s jump to another question – this one from LinkedIn. Andrew LaMarca – he’s Head of Fraud and a B2B fraud expert – asks: “How can AI systems be designed to remain resilient and trustworthy when traditional metrics often do not signal emerging threats?” It’s a really interesting question.

Anthony Scriffignano: Definitely the way we measure things needs to change. When you talk about fraud – I’ll broaden it to what I call malfeasance, because a lot of times the bad behavior happens in anticipation of fraud, before it’s technically fraud.

The way we’ve measured these things in the past is based on canonical understandings of things people do wrong (identity theft, misrepresentation of facts, etc.). But now, with AI, you have a whole new category of novel fraud or novel malfeasance that we don’t have names for yet – and we certainly don’t have metrics for yet.

The good news is there’s a lot of AI out there that can detect emerging patterns of behavior – not necessarily adjudicate whether they’re bad or not, but detect that “I’ve seen this behavior before, and it’s starting to become more prevalent.” And now smart people – like the person who asked this question – can go look at that behavior, and we can separate the noise from the signal and point them to what’s going on.

This isn’t looking for a needle in a haystack. This is looking for needles in a stack of needles – all the data is potentially valuable. The way we measured things yesterday is nowhere near good enough to measure these things in this kind of context, which is highly multi-modal with massive amounts of data.

Stephen C. Daffron: But I think you can do some practical things, Andrew. You can’t predict everything, but resiliency is the new value. Resiliency – understanding and being early to recognize that something’s happening.

What do you do? First, if you’re being a conscientious, prudent AI developer, you have key metrics and you monitor those metrics in near real time. You keep an eye on your accuracy, your precision, your recall – your mean absolute error, or the mean squared error in the case of regression models – constantly. And you use that feedback.

Then when something does happen, you react to it. You watch for prediction errors. The prediction errors that occur – they’re not randomly distributed. These new kinds of malfeasance will actually cause changes in those prediction errors. Find them, understand them, analyze them early, and then watch what happens afterward.

You’ll also see growth in your residual errors – look at the differences between the actual and the predicted values. Watching those over time allows you to… You can’t completely predict the new malfeasance, but you can have ways of finding it, finding it early, analyzing it, and reacting to it.

Back to the competition point – that’s good competition. The people who do this best – the folks in the marketplace now who are really good at developing resilient metrics that let them know something was happening first, and therefore react to it and save the pain that would come from that malfeasance – those are the winners.

Anthony Scriffignano: Let me just double down on something Steve said, since you brought math into the room. There’s a concept called elasticity, which is normally used in economics, but I’m going to use it in decision-making. If you think about decision elasticity – how wrong can you be and still make the same decision that you’re making?

You will never have a perfect measure of bad behavior, because the best bad guys – if they think they’re being watched – will change what they’re doing. So now you’re modeling what they used to do, not what they’re doing now.

The good news is that you can use math to figure out how much of the observed error is explainable versus unexplainable. And when the unexplained error – what we call random variation – starts to overwhelm the assignable-cause variation, guess what? Something new is happening that you don’t understand. Time to go back and figure it out.

There’s really good math and really good AI that can be pointed at problems like this, but you’ve got to ask a very different question.

Data Quality and Concept Drift in AI Systems

Stephen C. Daffron: Bad data. Bad data. Data drift. You started with one set of data… When the AI model’s performance decreases over time because the real-world data it’s encountering changes – did your governance change? Did some data come in that you didn’t recognize? Did one of your upstream data sources change?

I watch this every day in financial technology because markets change really rapidly, and the AI that’s been constructed often treats its input as relatively constant. But the AI has to be built to acknowledge change when it happens – otherwise you wind up with a data-model mismatch.

There are two measures that are very easy to implement in any system: the character and the quality of input data. Character means the nature of the data – the metadata, the provenance, the sources. And quality means the statistical measures – the distributions, the measures of central tendency, all of that. Anyone can monitor those things.

In most cases, what we see is the data gets “onboarded”, and then people are onto the next shiny object. Nobody’s paying attention to the character and quality of data. The other side of this, which is equally problematic, is concept drift. When you build the AI, you have a relationship between the inputs and the outputs – that’s where you start. But over time, those inputs and outputs shift.

And that drift can make your AI perform poorly without you even recognizing it. That can also happen when your customers start using your product for an unintended purpose.

Anthony Scriffignano: Oh yeah.

Stephen C. Daffron: And that happens all the time.

Stephen C. Daffron: That’s actually the most important kind of concept drift.

Anthony Scriffignano: Yeah.

AI in Investment and Operational Strategy

Michael Krigsman: Steve, I have a question for you. You’re running a private equity fund. How do these kinds of issues impact your thinking about your fund and your investments, and so forth?

Stephen C. Daffron: Our model is what we call an “IOI” model. We’re investors, but we’re also operators, and we’re also innovators. So this kind of thinking – on the innovation part of that model – affects how we as operators run the companies we invest in.

Our norm is to take a first-hand view. When we invest in a company, we take a direct, first-hand view of how to manage that company in a way that allows this kind of growth to happen. And that ties into where we’re going with AI, especially these days.

We have portfolio companies whose performance is dramatically improved by the introduction of AI. We buy companies – we invest in companies – where AI hasn’t been used, and we can then bring it to bear so that we create some of that value.

The value creation that comes with AI is a function of how you invest and, even more importantly, how you build the operational resilience of that company. If you just layer AI on top of existing processes, it’s a recipe for disaster. Running around with AI and saying, “How can we use AI here? How can we use AI there?” is the wrong approach.

AI is a tool – it’s a tool in your toolbox along with PowerPoint and everything else you use. And there are places where it makes sense. So if you said, “We’re going to improve the performance of a company using AI – and particularly this approach of AI” – great. Let’s measure it before we implement. Let’s come up with the how and why – build the KPI before the fact – and be clear on why we think this particular tool or approach is going to be right.

And by the way, how do we know we’re compliant? How do we know we have the right data? You have to ask at least a few difficult questions then push the AI button.

Michael Krigsman: Yeah. Because there’s a lot of “ready-fire-aim” out there right now. But doing it your way – that’s not fun.

Challenges in AI Investment and Returns

Stephen C. Daffron: I’m sorry, but you know… this is the real world, right? I will also tell you: I think there are lessons to be learned here. It goes back to both choosing to do the right thing, and also recognizing that it’s good to do the right thing.

For example, the latest data from (I’ll just say) Silicon Sands is that 75% of public companies who are trying to use AI now aren’t actually hitting the ROIs they expected to hit. Do you know why? Because they’re not thinking about it the right way. They try to chase “cheaper” or “faster” without considering the entire process – without measuring the trust in their products, the resiliency of their processes, the cost of their tech stack, the cost of compliance failures…

Anthony Scriffignano: Absolutely. And all of those things have to be addressed before the fact – which is honestly one of the reasons I like the private equity space, because we can get a lot closer to the action… we can sail closer to the wind, to use that metaphor. We can be closer to seeing what it takes to make the right kind of investments in AI to get the right kind of returns over time.

Stephen C. Daffron: For example – and I hear this complaint all the time in our portfolio companies – there’s this argument: “Well, we’re not getting the same gross margins in our AI businesses as we do in regular SaaS companies.”

No, you’re not. Gross margins on AI companies will be lower – and slower – initially, because you’ve got special work to do to build the underlying data structures first. Only once that data architecture is working can you layer on the specialization you need and have the AI brought to bear (with a human in the loop). Only then can you start receiving those benefits.

Now, if you look at the payoffs for companies that do it well – I won’t cite them, but anyone can look up the Silicon Sands data and you’ll see exactly what I’m talking about – the companies who do this well raise money at much greater rates and much faster than companies who don’t do it well.

But it is harder, unless you get your KPIs created before the fact, and you get everyone from the CEO to the CTO – and especially the CFO (sorry, CFOs) – on board. I’ve been a bit critical here, but CFOs are often the ones who want to make these metrics match the old SaaS business… and this is not the old SaaS business.

Anthony Scriffignano: Sadly, the reality is that if you can’t tell your overlords and your stakeholders, “We’re doing AI,” somehow you’re seen as behind the times. But if you don’t focus on what Steve’s talking about first – or at the very least, at the same time – if you don’t get the data right, if you don’t get the compliance right, if you don’t get the tech stack right, if you don’t get the KPIs right… you might be able to check that box and say, “Look at this thing we built,” but I promise you, you’ll be licking your wounds in very short order for one or all of those reasons.

Financial Frameworks and Technical Oversight in AI

Michael Krigsman: I wanted to do some CXOTalk shows with CFOs on how they look at AI investment and balance risk and innovation. I asked several CFOs I know, and I can’t get anybody to really talk about this publicly. Most of them aren’t very happy with their AI investments right now – unless they’re an AI company and that’s their actual product.

Most of them… you know, they’re not seeing the return. (I’m gonna… You’re gonna get crushed with comments disagreeing with this.) Of course there are examples of great success, but that road is paved with lots of, you know, “whoops.” And the CFOs… it’s also very hard to measure in the enterprise, because it’s not like there’s an “AI” budget line that things get charged to. Part of this is tech debt. Part of this… There are a lot of issues.

So you need the CFOs engaged, and there needs to be a focused effort on this. This is one of the conversations that we have across the industry: how do we frame the right financial model for this? Because gross margins for AI companies are different over time. They’re probably 50–60%, where a standard SaaS company would be running 80–90%. That’s just the beginning of it.

Stephen C. Daffron: Exactly. You have to figure in the cost of the requisite data engineering. I can tell you, the requisite data engineering is going to cost triple what you think it will in the beginning. You have to get that right up front to make this work.

You have to build the foundations for the large language models – those take time. You have to acknowledge the higher compute costs that go with bringing AI to bear. You need specialized technical oversight. To be candid, this is where I see a lot of gaps – because we think a software engineer is a software engineer. Sorry… There was a time when I would have called myself a software engineer, and I cannot do this.

It takes specialized technical oversight to make this work effectively, and you need to be prepared to pay to make that happen first – because if you don’t, those ongoing R&D investments won’t actually pay off.

Michael Krigsman: So basically what you’re saying is there are a bunch of suffering CFOs out there.

Anthony Scriffignano: Suffering is a choice.

Michael Krigsman: Suffering is a choice?

Anthony Scriffignano: If you asked Buddhism, they’d probably tell you that suffering is a choice. It’s your choice.

The Red Queen Problem in AI Innovation

Michael Krigsman: Hue Hoang says, “In AI, they’re seeing a Red Queen effect – how a Red Queen effect can rapidly lead to inefficient decision-making driven by influential figures, such as prioritizing hype over empirical validation.” Here’s the question: How do you, as leaders, identify and mitigate the negative impact of such influential individuals or practices within your long-term innovation trajectory?

Navigating Influences and Decision-Making in Innovation

Anthony Scriffignano: They become the shaman. And we don’t stop to ask, “What would we have to believe to follow them?” – because we’re kind of too busy sinking in the quicksand.

So it’s really important to pick your head up. Otherwise you’re either swimming in the wrong direction, or you’ll get hit in the head with the ball – either of which makes for a really bad day.

So it’s important to pick your head up here, and it’s important to watch how the environment is changing while you’re solving this problem. While you’re “AI-ing” the problem, make sure the problem itself isn’t changing.

And also, ask a few questions about why you should believe that shaman – what that shaman is selling, what data is being used to form that conclusion. Don’t just run in that direction because it looks like a solution.

Influencer problems become serious problems when you allow an influencer to be determinant. One of the things we try to do in operations – operation during innovation – is to step away from the immediate problem, pick your head up, and look at all the players.

Say, “I know she thinks that, and I know she’s powerful. I know she’s really smart and really articulate, but you have to shut her up for just a moment so that other people can ask their questions and listen to each other – and not allow…” This is why strict hierarchies don’t work so well in this space. This is horizontal rather than vertical.

Challenges in AI Strategy and Global Collaboration

Michael Krigsman: Ravi Karkara, co-founder of a global air and water generation initiative, asks on LinkedIn: “Is there a national strategy on AI in education to prepare the next wave of AI-skilled workforce?” Thoughts on that?

Stephen C. Daffron: Yes and no. There are guidelines, there are wishes and hopes – and recently there’s been more than just wishes and hopes – but there isn’t one ring to rule them all. We are nowhere near that yet.

Anthony Scriffignano: I think the answer is no, frankly. If you go into the academic world, there’s no coherent academic synthesis. Definitely in the government world there’s no coherence there. In the mathematical world, there’s more (because that’s where innovation tends to come first), but there is no coherent strategy that I’ve seen.

Stephen C. Daffron: The most recent thing in the US was the AI Action Plan – which… right… you know, it’s a plan to have a plan. There are other parts of the world where there are AI frameworks. The EU, I would point you to – there’s an unbelievably complex framework that’s been published, and a set of regulatory guidelines as well.

But even there, regulation and policy is never going to keep pace with innovation. I think what will lead us in the right direction is that a “pearl” will start to form around the “grit” of a particular problem.

Stephen C. Daffron: And I think that problem will probably be the cyber problem – that we’ll start seeing AI being used more and more by malefactors to create problems, and therefore we’ll be forced to develop a uniform strategy to contain that. And that’s exactly what you see happening in a particular country in the world right now – a pearl forming around that grit.

Stephen C. Daffron: I’m not sure I want it to only go in that direction, though. As an example, if you look at medical research and innovation – I do want them to go faster, but I also don’t want to give up all my personal details.

Anthony Scriffignano: Right, because in order to have the cybersecurity we want, the level of personal detail you have to be prepared to give up is huge. What you see in a lot of parts of the world that have more – I’ll say egregious – structures around this is: if you peel it back, you find you can do a lot more if you stay within our four walls, but you can’t do so much when you leave our four walls.

Most of these problems are global problems. And so that kind of thinking can be very dangerous at times.

Data Protection and Ethical Challenges

Michael Krigsman: Claudia Carlino asks, “What are your thoughts on how to protect your data from misuse without being completely risk-averse?”

Stephen C. Daffron: The place where we’re not paying enough attention to security is in protecting our data. I would spend more time and more effort – I’d be more obsessive – about protecting data than we are right now. And I realize that’s a hindrance to rapid growth, and it makes people say, “Well, I want to build my models in the light of all the data that’s available.”

When you do that, you are opening up Pandora’s box. I’m much more along the lines – and perhaps it’s because I’m old and slow – but I would much prefer to have data you can control, and then use that data to actually reach an endgame while controlling that data.

I’m just… I’m seeing so many places where you can create bias in the data that yields an unexpected and unpleasant outcome because you didn’t control access to the data. You can’t completely do this. There are all kinds of initiatives around the world around data.

The broader concept is what’s called data rights – you know, who gets to benefit from the monetization of a corpus of data.

Anthony Scriffignano: There are three frames you can think about. One is trust: whoever’s using your data will sign some sort of agreement that says they won’t do X, Y, or Z – and you can do bad things to them if they do. That’s great, but very hard to rely on.

The second is audit: you can put steps in place that allow you to watch what they’re doing with your data and make sure they’re not doing what they’re not supposed to. Again, very difficult – they’ll kind of move things around before you get there, or they won’t let you look where you’re supposed to look.

And the third is: do things to the data. There are increasingly sophisticated things you can do to put fingerprints in data – there’s differential privacy, where you can mathematically monitor changing trends in the data to understand if it’s been manipulated. And certainly there are “trust” solutions with blockchain and things like that, where you can know that things are unperturbed from the point of dissemination to the point of use.

All of these things are necessary but not sufficient. At the end of the day, data’s going to be a little bit squishy out there, and we have to understand that the older it gets, the less valuable it gets – all true data isn’t true at the same time.

So if you’re making the data, then… you know, you can do all of these things, but you’re never going to be completely protected.

Michael Krigsman: What about personal fines in cases of serious data breaches or bias infiltration? If 100 million names are released and my… say, my credit card shows up out there – how about the CEO goes to jail or pays, you know, a $20 million fine?

Stephen C. Daffron: In many parts of the world, you have that already. Not here in this country. So if 100 million… (Yeah.) The question for us is – the question Claudia’s asking is – how do you protect the data? If you try to protect it by saying after the fact, “I’m gonna hang the CEO if they allow this,” sorry – that’s not protecting the data. It’s already out of the barn.

Balancing AI Development and Data Protection

Stephen C. Daffron: And frankly, part of this comes from being a little more draconian at the beginning of the process. We’re going to build this set of AI for these purposes, with these kinds of models – which means we… You guys will recognize this – there are some places you actually build the stable that you’re going to use, and you’re restricting yourself by the data that you authorize for use at the beginning.

That slows progress, and people don’t want to do this. The shareholders, the investors – they want to go faster. But those who recognize that the danger of allowing that broader data access outweighs the advantage of going faster… They’re after the adversarial economy.

If I allow it to go faster – especially, think of medical data – think of the damage if you allow the medical data of 100,000 New Yorkers to be exploited because you wanted one particular health insurance company to have a better bottom line. The outline of that kind of failure means that you should be spending more time and more effort being more draconian about protecting the data.

AI Liability and Governance Challenges

Michael Krigsman: Smrithi Mohan – General Counsel at a company which looks like it owns SmugMug and Flickr – says: “From a legal standpoint, I’m increasingly concerned about how liability is or is not assigned when AI systems cause harm or are exploited adversarially. In a world where AI agents act semi-autonomously and their failures stem from complex supply chains of data, models, and algorithms, who do you think should bear responsibility? And how should the law evolve to address these distributed risks?”

Anthony Scriffignano: There isn’t a simple answer to that, and there isn’t a complete answer to that. There are certainly AI codes of practice that have been published – many, in fact, around the world. I’d point again to the EU – it probably has one of the more robust ones.

One thing you can do is start to hold people liable, just like you do when they violate the Constitution, or when they violate the code of ethics in medical practice, or when they violate the code of practice to get your trading license. You know, lots of different practitioners have codes of practice, and at least you can attest to their liability when they act outside of those general guidelines. That’s a good start.

The problem with AI is the problem of agency. If I hire somebody to deliver dynamite for me and they trip and fall and hurt somebody, they’re acting as my agent – and you could come after me. It doesn’t work that way with AI agents. If my AI agent goes and does something biased or maligns somebody, you have to be able to trace that back to my volition – and it’s nearly impossible to do that.

Michael Krigsman: Okay, time out. I’m going to try to get a word in edgewise while you’re on a practical response here…

Stephen C. Daffron: I won’t talk about the law – because I’m not a lawyer and don’t… frankly, I think that’s tertiary. Primary is first figuring out the right thing to do at the level where the action’s being taken.

So every company – every CEO, every General Counsel for that CEO, every CTO working with that CEO – should be saying, “What are our policies on AI?” At Motive Partners, we have a very clear, unambiguous policy – written by lawyers and practitioners – for how we practice AI at Motive Partners. And we do so carefully.

And every portfolio company we invest in, we give them guidelines – “Here’s how we think your AI should be developed, and the cares you should be taking with these guardrails.” Start with that. The practical value is, it’s companies who make the choices – companies led by people (women and men in the CEO, GC, CTO seats). Make those decisions in line with clear, unambiguous policies and stick to it.

Then, when the law… the law will catch up and say, “If you have that and you’re doing the right thing, we’ll reward you. If you don’t have that, you’re not doing the right thing, and you’re using this to exploit Grandma and her 401(k), then we’ll pin you to the wall.”

You have to be careful to also watch what your AI is doing, because it does a great job of working around what you told it not to do. There’s a great example out there – I won’t name the country – but in a country where they weren’t supposed to use gender to make a particular type of decision (it had to do with parole). So they just redacted gender from the AI.

And they later deconstructed it and found there was still a gender bias, because in that particular language female names ended in A or E – and the AI sort of convoluted around the vowel at the end of the name, because it didn’t have the gender. So you can’t just take your hands off and say, “I’m good to go – I followed the policy and now I can push this button.” You’ve got to pay attention to what’s going on.

Michael Krigsman: Lisbeth Shaw says, “Is it the confluence of all the factors – adversarial economy, etc. – that produces these AI misadventures? If so, you can’t control everything, so what should companies do?”

Stephen C. Daffron: Yes, it is the confluence of all these factors. And the worst thing you can do is say, “Well, gee, it’s complicated, so there’s nothing I can do.” The most important thing here is, the way a clam eats a whale – just one bite at a time, right?

You take the most significant step you can take, purposefully, in a direction – on purpose – and you lather, rinse, repeat. This is blocking and tackling. There’s nothing new here. They had this problem when lightbulbs came out. They had this problem when electricity came out. This is not a new problem.

Do it at the company level. Do not try to do this… do not give every developer, every agent, their own ability to make decisions. Do it at the company level. And I’m a capitalist, and I believe this is one of the ways that competition actually works for us. When companies see that it’s in their best interest to be resilient and to do the right thing, we’ll get a better outcome.

Michael Krigsman: This is a really interesting – simple – question, but not such a simple answer, I suspect, from Simone Jo Moore.

Michael Krigsman: She says, “Governance versus the law are often two different things. How do you manage a chaos situation where these two are far behind the advance of AI use?”

Stephen C. Daffron: Governance versus the law… The law is always a lagging indicator. The law is a function of the judicial process, which occurs… in the next decade, after most of the math has been done. Governance, on the other hand, can be done proactively. It’s not one-and-done, it’s not a proof. Governance is a statistical process.

Think about how AI has developed over the years – from where we had the initial machine learning to where we’re now up to agentic AI. Governance has to evolve at that same – or even faster – pace, and that can be done by the people who are actually building the AI.

So don’t… you don’t wait for the law to tell you what to do. You think through – forgive me, philosophically – is this the right thing to do? And can we do this in a way that actually moves both the right competition forward, but also does the right thing for people? And then that governance becomes part of your policy, and then you enforce that policy.

Do it with the right kind of intent, and the law will catch up.

Anthony Scriffignano: The only thing I would add to that beautifully poetic answer is: governance starts with first principles. What do we believe, and how do we know that we’re being true to those beliefs? And if you don’t start there, you wind up with the traffic code – you get so many different policies that you can’t possibly comply with them.

So you’ve got to go back to your first principles.

Stephen C. Daffron: Which I think is philosophy.

Anthony Scriffignano: That’s what I… Yeah. That’s my philosophical approach to this – it calls for first principles. I have the same thing.

Stephen C. Daffron: I wanted to call it epistemology, but then he would be—

Anthony Scriffignano: There we go.

Stephen C. Daffron: (laughs) There we go – another round.

Michael Krigsman: And with that, a huge thank you to Stephen C. Daffron and to Anthony Scriffignano. Gentlemen, thank you both for being here. You were brilliant, and I can’t thank you enough. I’m grateful to you both.

Stephen C. Daffron: Our pleasure.

Michael Krigsman: And thanks to everybody who watched – and especially you folks who asked such great questions. Now, before you go, subscribe to the CXOTalk newsletter. We have great, great shows coming up.

This episode will be posted on the CXOTalk website by Monday. It’ll be lightly edited, and there’ll be a summary and all kinds of great information. That’ll pop up next week. So, check it out – and everybody, we’ll see you again next time. Take care now.