The AI Attack Lifecycle:
Digital Forensics and Intelligent Threats

AI-powered attacks at unprecedented speed and scale

Michael Krigsman: The death by 1,000 cuts argument is actually true. You don't need to steal from ransomware attacks millions of dollars. All you need to do is hit enough people at 300 a pop and do that at scale with SIM farms that are being found all across the US, and all of a sudden, the $300 per person times 10,000 equals the same as a single ransomware payout before. This kind of scalability has not been able to be achieved before AI.

AI and autonomous agents are wreaking havoc with cybersecurity. Rob T. Lee is Chief AI Officer of the SANS Institute, and our co-host today is David Bray, Chair of the Accelerator at the Stimson Center. Rob, why should we care about the AI life cycle and security? Why does this even matter?

Rob T. Lee: You're untapping a massive problem when we start to talk about the speed and velocity that's able to be achieved with autonomous agents. You look back just 4 or 5 months ago and Claude Code being utilized and detected by Anthropic, basically automating offensive capabilities in the attack inside organizations with minimal human in the loop to be able to make sure that hallucinations are minimized. Things that used to take months, weeks, are now being accomplished in minutes and seconds.

And it's hard to wrap your head around what that specifically means for potentially detection, being able to stop it, and being able to see what's going to happen even 6 months after the fact, what the adversaries and capabilities of even an individual are capable of accomplishing inside a network, much less attacking an individual for scams.

Michael Krigsman: David, sounds pretty scary.

David A. Bray: It does. At the same time, I think every decade, something new shows up on the scene, and this is the challenge of our decade. Exactly as Rob said, the speed and scale at which generative AI is able to allow an attacker that might not be that sophisticated, or possibly what would take a long lead time, to piece together an attack and do the reconnaissance and actually execute the attack is unprecedented.

But it means we need to adapt, and it means we need to find strategies that allow us to identify what we would call abnormal patterns of life happening in enterprises and adapt and respond faster. I look at it as the attacker has evolved their strategies. It's time for the defenders to do the same.

Cyber defense lessons from military history

Michael Krigsman: Rob, how have AI and autonomous agents changed the nature of threats, changed the nature of attacks? And as David was talking, I saw you nodding your head in agreement.

Rob T. Lee: I'm a big history buff. Every single time there's a massive technological change, there's a massive change in tactics that need to follow, and much less the strategy. You could go back to the introduction of the airplane, World War I, the tank especially back then, going back to the Civil War, the line defense with Napoleonic, what we ended up seeing then, all the way up to the aircraft carrier. I basically moved this mindset into warfare, into what's going on with cyber defense and cyber offense as a whole.

As a result of the speed and velocity, you end up taking a look at essentially the blitzkrieg, what happened in World War II. The same kind of mentality has occurred. Things that used to be planned out that would take weeks, months for an offensive campaign were now shortened to days. This type of mentality really created a shock and awe to individuals trying to defend against it. That type of shock and awe is going to force a strategic and tactical redefinition in the way they're applying cyber defense currently. How fast can we ingest and leverage threat intelligence, the tools, tactics, and techniques that the adversaries are using, and be able to leverage the capabilities that we have in front of us?

AI changes not only the offensive capabilities, but the defensive ones. The hard part, though, is being able to leverage the defensive ones in ways that are legal and able to reduce potential regulatory violations. But if you imagine, for example, if multiple different organizations get hit simultaneously, if they're ingesting those automated detected TTPs and then spread it out worldwide, we'd be able to have a chance to counter what the offensive teams are doing at the same velocity that they're accomplishing it.

But right now, even if you start saying, "How would that be accomplished?" we have a hard time just sharing threat intelligence, handing pieces of paper over, or even calling each other on the phone, much less having a universal LLM that is disseminating TTPs worldwide. The technology's there. The question is, can we even use it to leverage against it in a proper cyber defense capability that changes the tactics?

Federated learning as a shared immune system

David A. Bray: I'll add plus one hundred to Rob for pulling from history, because I think that helps us address it. It actually helps you be a good stoic. In the midst of whatever is happening, you can say, "Themes and rhythms like this have happened before, and we will adapt." And I think he's spot on that AI, while right now is currently super empowering attackers, to his point, we could actually have an AI that, again, if you look at what the TTPs are, what the attack surface looks like, what are abnormal patterns, and having that shared, I'd go even one further.

About fifteen years ago, I was pushing for almost a public health approach to cybersecurity, which is in public health, we don't share the specifics, but we share what we're seeing in the different states because each of the fifty different states have their own privacy laws and things like that.

And one of the biggest barriers for sharing insights as to what's happening at one organization with many organizations is, one, they're worried about revealing their own vulnerabilities. Are they going to reveal their intellectual property? What might be interesting is there is an approach called federated learning. And federated learning, instead of having the data be shipped somewhere, the algorithm comes and learns there. Imagine actually that we had an algorithm that you were willing to let learn on what your organization is seeing, and as a result, you get the benefits without the specifics or the data as to what other organizations in your industry are experiencing.

So maybe you're all in finance, or maybe you're all in commerce, or maybe you're all in transportation. You could actually say, "Here are the current attacks as of the last hour that we have been seeing across the world in terms of attack surface activities that are being exploited." And as a result, you do that without having to send your data to anybody, and you're collectively more secure.

Why incident response must match attacker velocity

Michael Krigsman: Rob, thoughts on how incident response and forensics must evolve, especially since AI can navigate breached networks, erase its own tracks, and corrupt evidence. How must incident response and forensics evolve as a result?

Rob T. Lee: There are so many different ways that we could potentially utilize the data that needs to be employed. That capability of potentially being able to do sharing easier needs to be accomplished. Incident response and threat hunting are now pivoting specifically on what you know and how fast you can know it. So incident response and being able to identify attacks is crucial. If you're looking for how they're pivoting across a network, how they're potentially doing data exploitation, essentially looking for the Death Star plans inside a network so they could get it out, or potentially embedding themselves to cause havoc in the future, critical infrastructure, for example, Salt and Volt Typhoon.

The hard part, taking a step back and looking at incident response, is that somehow we need to achieve some sort of parity, meaning that as the offensive teams are able to move at increasing velocity, incident response needs to be able to match that. Here are a couple of challenges with that. I believe currently in 2026, and this could be different even 6 months from now, most executives making decisions whether they're going to RSA and seeing these new solutions that are out there will have a hesitancy about deploying a solution that is going to remove the human in the loop.

Offensive teams are going to feel more comfortable removing the human in the loop faster than you as a company that says, "I'm going to take on that risk," because if an incident response or detection tool says, "Hey, I found something, I'm going to block it automatically," without a human analyzing that, what happens in that situation?

Bad things could occur, so are you placing the organization at greater risk if you deploy these defensive tools that haven't been tested fully? I would be a little bit concerned right now not having a human saying, "Yeah, we should block that, turn off that node," in order to prevent that attack. Offensive teams are still able to say, "Well, if it trips over something, not my fault." The cyber defenders can't take that risk, so as a result we're not going to be able to achieve the same velocity, and that alone is going to create parity issues even if we match capability to capability, AI versus AI. How do you reduce that risk so we're able to match velocity versus velocity?

I don't think we've solved that problem yet technically, because who's going to trust, "Hey, Rob, how do you know you're not tripping over something that's going to disconnect us from the internet and cause problems?" Technically we may be able to get there, but how do we know it's going to work effectively? Offensive teams: "Nah, if it trips over something, no big deal."

Attackers weaponize defenders against themselves

David A. Bray: And to Rob's point, he underscored a very essential risk calculus, which is the defenders want to keep the business running while also keeping it secure, whereas the attackers are, as he said, indifferent. Because the attackers will be more likely to pull solutions that don't have a human in the loop, we've got to figure out ways that can help the defenders move at the same speed.

And I think the biggest barrier we have is that despite all this talk about digital twins or whatever, we're really not that mature in terms of enterprises being able to run a rapid parallel simulation that says, "If my security tool opts to do this, what are not just the first order, but the second and third order consequences that could happen to my enterprise? And as a result, I might be trying to secure it, but the next thing I know I've actually blocked this application and now my customers can't do business."

The other thing I would also say is if I was a really sophisticated hacker, I would make it so that you did it to yourself, and as a result your security team is now dealing with not only having caused the business to go offline, but now the company is upset at the security team. They're demoralized. This would be, hopefully, something that an attacker wouldn't do, but I've seen in the past where using demoralization is a tactic of certain threat actors.

Michael Krigsman: You can ask your questions, and you should take advantage of this opportunity. When else will you have the chance to ask Rob Lee or David Bray pretty much whatever you want? So if you're watching on LinkedIn, pop your question into the chat. If you're watching on Twitter/X, use the hashtag CXOTalk.

And on this point, Anthony Scriffignano, who's a very prominent data scientist and has been a guest on CXOTalk a number of times, makes the comment that one approach in attacks is for the bad guys to focus on a target that is already distracted in response mode.

Coordinated distractions hide the real attack

David A. Bray: This is exactly what we have seen not just in cyberspace, but also in the real world, where you get someone distracted and focused on responding to one activity. Maybe you've got something that looks like a hardware issue or a software issue with your application, and the next thing you know, while you're busy focusing over here, the attacker is coming in and going laterally or something like that. To Anthony's point, what generative AI in particular now allows attackers to do is to actually plan something that's sophisticated, which did not used to be the domain of most attackers.

It was really the domain of nation state actors, and it took them months, both to do the reconnaissance and the coordination. Now, what I'm hoping is that we can actually evolve almost a science where on the defender side, we can quickly say, "I see these isolated events," or what appear to be isolated events, but they're actually part of a more coordinated campaign. And I know Anthony Scriffignano has worked countering fraud and tackling fraudsters, and usually the challenge is that you're seeing the tip of the iceberg of what is a much deeper campaign.

So this is exactly why what SANS is doing is so important and why we have to evolve what defenders do because now with both bots and agents, you could have five or six fires going on at the same time, and the question is, while those five or six fires are happening, how do you know there's not a seventh one quietly actually exfiltrating data out of your organization?

Michael Krigsman: Rob, thoughts on this kind of multifocal set of attacks, a campaign of attacks?

Rob T. Lee: Distraction. Most teams are exhausted, and they're looking at the same screen every single day. As a result of that, it leans into one of the reasons why defensive agents that can work twenty-four/seven and never get exhausted will potentially be a game changer. The challenge though is we're only seeing offensive teams leverage this against a single target right now. What happens when you start having agent to agents coordinate and communicate with each other simultaneously across critical infrastructure, hitting power and water simultaneously? How do you have two teams at the same parity level on the defensive side to prevent that?

The shock and awe that could occur there, where you could have down to the millisecond execution across multiple different nodes, financial networks, supply chain, critical infrastructure, electrical, hitting boom, boom, boom. And imagine what happens on the criminal ransomware side, when they wait until multiple parts of a supply chain are vulnerable. Think about some of the things that happened in the past with ransomware. Imagine multiple different hospitals and insurance networks and pharmaceutical companies hit simultaneously with ransomware attacks. These types of things are definitively possible, and the scale of offensive attacks grows larger.

When you go back to distraction, exhaustion, limited teams, a workforce where we can't simply hire more people, leveraging AI is the only solution. And we come back to: how do we balance that with what we're seeing the offensive teams do, and how many potential attacks will need to occur before the risk calculus is so great that you're going to have to say we must potentially embed ourselves utilizing some of these new technologies, remove some of the human in the loop, and increase risk as a result of what can happen if we don't.

I could go on for hours about this topic, but this is possible, and it likely is occurring right now, and it's a matter of time before a geopolitical incident occurs, and they're going to do a show-me session of how vulnerable we all are.

Autonomous agents expand the attack surface

Michael Krigsman: Let's talk about autonomous attacks. We have OpenClaw, and agents are the new, I don't know, the new refrigerators. Everybody's got to have one, and we're going to have thousands of them. This just makes the entire problem more complex, as Rob just said. They don't sleep. So who wants to jump in and talk about that.

Rob T. Lee: The good thing about OpenClaw and ClawdBot is that the one thing you hear out of everyone's mouth initially is, "Don't install it on your own system." And it shows people are thinking thoughtfully about turning on autonomous systems to go wreak havoc with their own data, and it shows the risk calculus that will happen with security. You need admin access to do anything useful to shut down attacks, but we're already seeing people say, "Don't install OpenClaw or ClawdBot on your own system with root access," because it could cause havoc. These two things intersect and are not equal on the technological side.

David A. Bray: Spot on. And I actually saw a story where someone was posting about how they were supposedly in Ukraine, maybe they were, maybe they weren't, and they decided to install it on their system because it would make their home smart. Personally, whenever I see the word "smart device" or "smart home," I substitute the word "hackable" because that's really what it means. And I don't know if I would do that in the middle of an active war zone, because that's probably exploitable.

So I do think, to Rob's point, one, there are cautions about being very careful because these things will possibly wreak havoc. The other challenge, and this gets to the fundamental challenge of generative AI, is that generative AI is trained on past patterns. And while you can ask it to possibly remix those patterns in interesting ways, it will always be tied to its past training set. It's not necessarily really good at novelty. In fact, when you try to present it with novelty, that's why you sometimes get something that's a pattern that doesn't actually exist, which they call a hallucination. It's just something it's never seen before.

And so this is actually more of a challenge on the defender side because as we train these models to be defenders, with those 24/7 agents that are helping with defense, the challenge is that if a novel attacker really wants to exploit our attack surface, they will gain, through various means, insights as to what the agents are expecting to see or possibly may see, and therefore come with something completely different. Which is why, as Rob was saying earlier, we've almost got to have an immune system response that says, "I've never seen this type of attack before. I'm going to try and rapidly characterize it. I'm going to try and figure out what the intent is, and then I'm going to percolate the lessons not just across my enterprise, but ideally through federated learning, I'm going to share those insights with others in my industry," because this is where the novelty on the offense side puts defenders at a disadvantage precisely because generative AI is limited to its past training sets.

Humans as accidental security governors

Rob T. Lee: I have one thing to add from a technical perspective. I had a conversation just yesterday with James Lyne, actually, as we were chatting about our RSA keynotes. The attack surface with automated capabilities like ClawdBot and OpenClaw, when you think about it, imagine if you're saying, "Hey, go read my email." Essentially you're setting yourself up for the ultimate spear phishing, as it will go click on every single link coming in to see what it is. The human at that point is the security defense because the human can't click as fast as the automated capability.

We've exploded the possibility of just the autonomous capability acting like a human exposing additional attack surfaces, because if you throw enough balls in the air, one of those is going to be hit by one of these autonomous agents. When you think about increasing speed of being able to respond to many emails and marketing and tweets and LinkedIn messages, imagine throwing enough spear phishes at that capability. One of those is going to land, and it's going to happen at speed. The human actually ends up being, just because they're slow, accidentally a security governor.

David A. Bray: This points to the very challenge that while we need to use AI to help with our defenders that are already exhausted and already over capacity, AI itself is an attack surface. We have indications, I wouldn't say confirmation, but indications that some bad actors have identified that when AI is being used for coding, for vibe coding, it reliably hallucinates, and they have identified those reliable hallucinations and gone out and written a code library for the very hallucination that the code library is supposed to address, and made it open source. What could be possibly wrong when you actually call a code library that didn't exist until a bad actor wrote it?

So this gets to: not only do we need to use AI for defense, but as we're using it in an enterprise, it itself is an attack surface, whether we're using it for enterprise operations or to write code. If anything, cybersecurity professionals are going to be needed now more than ever. However, we've got to figure out how we up our game and also give them the head space and breathing room to deal with everything that's happening.

AI swarms only need to be right once

Michael Krigsman: So if you can expand this aperture a bit. Anthony makes the comment, "Swarms, AI swarms can also learn from how they fail and attack repeatedly in a more targeted way." Thoughts on AI swarms, and then we'll take some questions.

David A. Bray: That gets to the challenge, which is I could inundate your enterprise if I know the capacity of the actual people to respond. I've seen swarms before, I've experienced swarms before, and Anthony is spot on. Given that they can do permutations and they can try to be adaptive, they only have to be right once out of 1,000 or 10,000 attempts.

This also points to a lot of historical cybersecurity practices, and I know there's been a shift in the last 5 years, but a lot of historical cybersecurity practices relied on known CVEs, critical vulnerabilities and exploits, and the signature. Well, the challenge is that if the swarm is changing its signature every second, that may not appear as a CVE. That may not appear as something that a threat device is registering as an attack. So swarms are definitely problematic, and it points to, and I have a sense this is where Anthony's going with the question, that we cannot just rely on past trainings.

What we need is almost an algorithm that says, "I see what looks like non-normalcy." It's almost a second derivative of it, because what we're looking for is, "I don't know why I'm seeing this abnormal pattern in my enterprise. It might be hardware or software, but it might be a sign that a swarm is trying to mess with me."

Michael Krigsman: Rob, thoughts on swarms?

Rob T. Lee: I feel we have not seen many specific examples of swarms versus speed. And I flip that a little bit. When I go back to swarms, the swarm I tend to think about is the swarm of developers that land on a new capability. Take a look at what happened with OpenClaw and ClawdBot. How long ago was this someone's basement project, a weekend development? Cowork is really another good example. From Claude Code to Cowork being released was 10 days. When you end up having the development life cycle go from something that used to take years down to weeks, that is the swarm capability I'm looking at, because instead of just one or two developers, you have hundreds of developers potentially iterating on code or even the code iterating on itself faster than we're able to potentially comprehend.

Now, to take that to the positive side, even though you have capabilities that go from infancy to fully vetted capability within weeks, cyber defenders can leverage the same capability, because offensive teams tend to still be fairly tight-held on their capability development, 3 or 4 developers. What if we flip that and potentially have hundreds if not thousands of cyber defenders start working on capabilities for defense, and take a ClawdBot and make it into a very specific cruise missile that no one's ever seen before, not something that took years to develop, but something that might take weeks and could be deployed as a counter mechanism inside the networks? What we're able to see on development speed, we could potentially use on a defensive mechanism for good.

David A. Bray: This changes the notion of patching. Instead of patching being something that takes months, patching could be done in weeks or even a day.

Perverse incentives punish good defenders

Michael Krigsman: Let's jump to some questions. This is really my favorite part, where we get to hear from the audience. Keep them coming. All right. We have a question from Arsalan Khan on Twitter/X, who says, "How should businesses be held accountable, or should they, for cybersecurity issues?" So where does the responsibility and accountability lie?

Rob T. Lee: Doing incident response correctly still means you messed up. What I mean is that the incident occurring in the first place means that something went wrong. And even though the incident response team may have interdicted it, may have stopped the Death Star plans from being stolen, you don't get medals as an incident responder for stopping that. In the military, you do. Even if an incursion happens, if North Korea flies jets into South Korea, even though they encroached in the airspace, if we're able to launch missiles to interdict and stop them, everyone gets a medal.

That does not happen in cybersecurity. The accountability and our mentality and culture have still not changed to reward proper cyber defense. So accountability incentives are perverse as a result of this. If you're a CISO and you do a good job, you detect the incident, the incident happened, you failed. Those are the perverse incentives that exist today, and they're not going to change because AI's out there.

Where are the medals for the incident responders, the SOC teams that detect the incident in the middle of the attack and say, "Hey, we stopped it. They did not achieve their operational goals"? They're still looked at from a liability and risk perspective as having failed.

David A. Bray: If I could do a hallelujah to what Rob just said, because this is the challenge, is we still have boards and we still have CEOs that think this is a preventative exercise, when the reality is if you're going to be on the internet, it is going to happen to you. What you want is exactly as Rob said. You want a team that responds to it rapidly, that actually tries to counter it, and where success is simply: one, we detected it, and two, we responded to it and did our best to mitigate it. But that is a mind shift, and I will do anything, by the way Rob, to help SANS get that message out there.

Because unless they want to unplug themselves from the internet, bury whatever they're doing underground and make sure no human being touches it, these things will happen, and they will happen with increasing frequency, partly because of heightened geopolitics, but also because of the capabilities that are out there. And if you're going to use AI, what I am trying to do is go to boards and to CEOs and say, "If you're going to be adopting AI for enterprise to engage your customers or improve your internal operations, you should in parallel upgrade your cybersecurity posture," because that will actually make it so you're more responsible in your use of AI, while at the same time hopefully making it so that you can detect if bad actions are being sent your way, because they will be sent your way.

Acceptable risk is not zero risk

Michael Krigsman: Let's jump to another question. And by the way, I have to say, hearing you both talk, I'm ready to hide. I'm ready to do the ostrich thing. Stay home, maybe put mattresses up against the windows.

David A. Bray: I know someone that can help you get a Faraday cage if you want one.

Rob T. Lee: But as a parent, do you just pull your kids inside and say, "Hey, never venture outside. You're going to wear a helmet on your bicycle"? There's a certain amount of continual risk no matter what you do that you have to assume. You can't prevent any bicycle accident from happening. You just set the rules of the road. You say, "Look both ways. Don't run with scissors." There are certain things that are acceptable risk, and without that acceptable risk, there's no learning, there's no growth, there's no acceleration. As a parent, you recognize you can't prevent all the bad things.

But again, no ostrich head in the sand. We need to lean forward, especially today. Business transformation, everyone is trying to figure out how to deal with AI. You just need enough of a parachute, enough of a rope around your waist to say, "I can take a certain amount of risk safely," but not with zero risk of ever potentially falling into the chasm.

Michael Krigsman: But it used to be that the risk came from getting hit by a car. Those risks remain, only now we have this very large, ongoing, permanent additional attack surface because we live our lives online.

David A. Bray: Correct, and also consider this. If you have a smartphone, you have the capability to call anybody at a moment's notice, hopefully with their permission, track them or any asset using AirTags or other means anywhere in the world, again hopefully with their permission, and download satellite footage as recent as 15 minutes ago at .25 meter resolution. You have the CIA in your pocket, circa the late 1970s, early 1980s, and 2 billion people on the planet have it. So we've super empowered you to do things that are unprecedented. That also means that bad actors can do things that are unprecedented.

I don't think we want to roll that back. It just means, again, we've got to figure out defense in depth, and we've been here before. In the 1910s and 1920s, there was TNT. TNT was being used by anarchists to do bad things, but we didn't go into lockdown mode. We figured out defense in depth, and now it's less of a threat and we continue to live our lives. So I think we can figure it out, but it's going to take new approaches.

And part of the trouble with cybersecurity is that both CIOs and CISOs initially reported to chief financial officers, then later chief operating officers. And so it was seen exactly as Rob said: cybersecurity was a preventative function, as opposed to what it actually is, which is what you do as you're interacting with the world and doing commerce. And so what you want is a team that does get the medals for rapidly detecting, rapidly containing, and rapidly mitigating what will happen, because that's the nature of living in a world where...

Rob T. Lee: We have massive SIM farms that are out there. But imagine that instead of something that hits your kid or potentially you, your parents, all your neighbors, that kind of scalability is now achievable with AI. So the death by 1,000 cuts argument is actually true. You don't need to steal millions of dollars from ransomware attacks. All you need to do is hit enough people at 300 a pop and do that at scale with SIM farms that are being found all across the US, and all of a sudden the $300 per person times 10,000 equals the same as a single ransomware payout before. This kind of scalability has not been able to be achieved before AI.

Every AI tool chain is a target

Michael Krigsman: Let's go back to some additional questions, and I'm still not convinced that the ostrich approach is wrong. I'm still thinking maybe I need to just simply hide and that's it. Maybe by the time we're done with this conversation, I'll feel more confident that I can actually go out in the world.

All right, this is from Chris Petersen on Twitter/X. And I should also remind you to subscribe to the CXOTalk newsletter. Go to cxotalk.com now, subscribe, and we will send you notices of upcoming shows. This is from Chris Petersen, and he says, "Feeding AI tools good data often requires complex tool chains. How do we keep those tools and chains from becoming the attacker's prize targets?"

Rob T. Lee: You can't. It's the same thing as having a UPS delivery person walk straight in the door. The more you end up with a larger attack surface, you can't eliminate it. You take a look at what enterprise is now leveraging across simple Zoom sessions, what we're doing here from Teams and so forth, the likelihood of someone being able to penetrate that is becoming more prescient.

I go back to the Chinese and North Korean threat actors that, instead of hacking a network, just get hired as an employee. That's using your own tools in HR, in humans, plus technical, sending you a laptop that has the keys to every access point in your network already set. This is going to take individuals to realize that even attack tools, the common management capabilities in our organization, are able to be exploited.

And the attack surface for a single human is really interesting. About 20 years ago, a single individual would have two or three nodes: a phone, a computer, maybe a cable modem. Now, take a look at how many nodes are across my home, your home, 30, 50 nodes, with every single appliance and potential TV. The enterprise network size from 20 years ago is about the same as the single household network size that a single individual has today. It's kind of mind-numbing when you count up how many individual devices you have that are connected to the internet. Every single one of those is a potential attack surface.

Michael Krigsman: Here is a question. Reza Fatahi says, "There are good opportunities at the application and OS layers. Each angle is a different application. Each layer is a different application." Any thoughts on that?

David A. Bray: You're going to have to find ways to rapidly do independent verification and validation of what you're using as your tools, whether they're for data ingest or applications or things like that, and it has to be faster than what's currently being done. You can't just rely on published critical vulnerabilities. You really need, for the things that are absolutely essential, to be constantly red teaming how an application or an OS might be used against you.

And the good news is, again with AI, I've seen instances, I won't name the big tech company, but they did two parallel paths. For some of their cybersecurity functions, unfortunately they initially went down the path of replacing their cybersecurity people with AI, which I do not recommend. But in another function for their mobile security, they had the AI focus on known critical vulnerabilities to free up their cybersecurity professionals to look for new exploits. Well, guess which team outperformed? It was the humans plus the AI, because you have to always be looking for new vulnerabilities. You cannot just rely on published known vulnerabilities.

Hiring for the hacker mindset

Michael Krigsman: This is from Hue Hoang on LinkedIn, and she says, "As a defender and as an attacker for technology readiness, do you see the direction of talent development moving towards what previously was called the 10X engineer or a full stack engineer?" So the idea of talent and preparing talent to handle these issues.

Rob T. Lee: I go leadership by vulnerability, or vulnerability by leadership, depending on how you want to frame it, is that we're all learning. I'm learning. I had very little knowledge about AI 2 years ago, and I'm struggling on a daily basis. And this is where, from a talent management perspective, there's this idea that we need to hire people with AI experience. It's really hard to assess right now because by the time you hire them, their knowledge may already be out of date. So from a talent management perspective, one of the things I think is more crucial than ever is trying to identify individuals that have a massive capacity to learn quickly.

It goes back into the old hacking mindset, and what before was considered criminal. The original mindset was, "I wonder if I put this thing in over here, do this thing, tie the wires together, and take a step back." That is a fiddling mindset, and that's what hacking was considered, even going back to the MIT days where they invented the term. That type of talent is hard to come by, and it's also hard to assess.

What I'm seeing organizations do, and I credit Kate Marshall who mentioned this on her Substack, is that you start seeing organizations using hackathons. We ended up seeing, I think it was NVIDIA or Anthropic, just do one where if you end up being the top competitor in the hackathon, you're not only offered a job, you win a cash prize. It assesses someone's ability to work through problem-solving challenges, to demonstrate their mentality, to show you're the right candidate. That is the only way organizations are going to be able to hire the right people given the massive technological change and the velocity at which it's occurring.

Detecting what normalcy looks like

Michael Krigsman: David, this one seems tailor-made for you. This is from Simone Joan Moore. Earlier you spoke about immune system response, she says. What about the systems that are off, compromised, or whose DNA hasn't developed the signal or detection system for it? What do we do?

David A. Bray: You have to instrument your enterprise to have what's called patterns of life. What does normalcy look like for these systems, both in terms of their operations and the human use? So then if a persona that looks like your CFO or your CEO, at 9 PM on a Friday night, suddenly seems to be accessing a finance system in a way that is non-normal, that might be a red flag. And this gets to Rob's point, which is we have to have ways of defining what is reasonable risk calculus.

You may want to stop that transaction until the chief information security officer actually has a conversation and says, "Was that really you, CEO or CFO, or was that some bad actor trying to exploit your persona to actually pull money out?" And so that requires enterprises to be using their logs and that instrumentation in a way to establish patterns of life, but there is precedent for this.

The other thing I would say real quick, and we haven't gone here yet, is that the good news is generative AI is really good at passing the Turing test. The bad news is generative AI is really good at passing the Turing test. I get concerned about audio, video, synthetic media being used to convince your humans to do a seemingly legitimate activity where nothing was really breached, it was just AI itself being used as a prod for social engineering on steroids, to have money or data walk out the door. So generative AI itself is going to require us to have better ways of verifying that those workers and those people you're talking to really are legitimate.

Building trust in autonomous defenders

Michael Krigsman: This is from Demetris Georgiou, who says, "Looking into the future, when do you expect that AI agentic defenders will be able to be trusted as an experienced human without a human in the loop? And please answer for both IT and OT environments." Rob, you want to jump on that one?

Rob T. Lee: When you end up looking at the standard certification, accreditation, and process of human beings, you get your college degree, you get your certifications. There are assessments that someone goes through to establish the baseline of minimal qualifications to do a job, and the trust is that there's still some third party doing this assessment.

I think the industry, and for anyone going to RSA in a few weeks, walk around the floor and everyone is going to have massive solutions, agentic defenders. Ask them, "How do you establish trust in that capability?" Well, we have our own rubric. We went through our own battery of exams. I could go out there and say, "Hey, my IQ is X." Well, how do we know I'm self-assessing my IQ versus having a third-party independent capability establish that trust? The reason I use trust is that all these different accreditation, certification, degree programs, and assessments are all aimed to establish trust. Trust in the agentic defenders needs to have some sort of trust mechanism assessed by a third party, not software accreditation. Does it handle reasoning equivalent to a third-party human being?

Michael Krigsman: This is from Dan Turri, and he says, "Generally, is industry accepting of integrated logging tools, or is there a need for external immutable logging? It seems like we have rolled out tools without pause for validation and integrity."

David A. Bray: It depends. Certain enterprises are more instrumented than others. The finance sector has to be because they're regulated. I think what he's asking, though, is whether it's sufficient to have a log where you don't have some immutability to it. And I would say yes, you want to have immutability to the log because, as we know, some of these attackers will mess with the log or even destroy it. So the short answer is yes on immutability.

I do want to add to this question and tie it to what Rob was also saying. I think we're going to find, especially for agentic AI, and I realize there are several companies plowing ahead with it, that if something has a filter or safety applied after compute, post-compute, I get very nervous about post-compute agentic AI ever being 100% safe. And again, you can figure out contexts where it's acceptable to have that risk.

I want more and more AI agents where I can have filters, boundaries, and rules applied pre-compute, because I think the moment you let the machine compute something, just by the nature of how hackers and attackers work, they will get it to do something you don't want it to do. So I would encourage increasingly: look for people that are pushing solutions that are pre-compute versus post-compute. That's a nuance that unless you're a computer scientist, it's not something people ask for. But pre-compute is where we need to put more attention.

The permanent offensive advantage

Michael Krigsman: We have a question from Christopher Jablonski. Rob, I think this one is perfect for you. This is from Christopher Jablonski. He says, "Going back to how offensive teams seem to have a permanent competitive advantage as they are more comfortable with removing a human in the loop, by how much does this issue and other such asymmetries delay autonomous security from reaching reality?"

Rob T. Lee: I think that's the reality for the near term. And I actually think, to a certain extent, it is necessary because I think from a cyber defender's perspective and development in the industry, we've become a little bit assumptive that we have some sort of parity and that we're able to block and tackle what the offensive teams have been able to do for the past 5 to 6 years. Now this is going to create leverage in the industry and hopefully a lot of innovation that will be able to match it.

We're assuming that we're going to be able to leverage AI to AI. I go back to what the offensive teams aren't going to be able to do, which is create AI ecosystems where defenders can talk to one another. You even just take a look at ClawdBot and some of the capabilities it's already been able to leverage, AI judicial systems for different agents, its ability for AI agents to ask humans to do certain tasks. That physical level task could have an AI agent who's your manager tell David, "Hey, I need you to go over to this system and unplug it from the wall." Even though that seems rash, there are ransomware cases where that is literally what the response mechanism is to make sure that system is not infected. There are ways for innovation that are unexplored on the cyber defender side that this asymmetry is going to force, and I'm excited to see how this develops.

Michael Krigsman: Michael Moorehead throws a monkey wrench into all of this.

David A. Bray: Yes.

Michael Krigsman: Michael Moorehead says, "Sure, all of this is true, but once you remove the human in the loop, AI errors are a greater problem, and the bad guys simply have to hack the AI defender and convert it into a dual agent."

When will we remove humans from the loop

David A. Bray: Yeah, that is spot on. We haven't dived into the different types of AI. There are AI methods that are more rigorous than, say, generative AI. But I will give the analogy, and I think Rob's right, that in the short term, the advantage is on the offensive side. Defense has to up its game.

But we do have US naval vessels where if the captain of the ship flips the switch, there is no human in the loop. If something is coming at that ship and threatening it, the system will actually be fully autonomous in defending the ship. He obviously turns off that switch when he's not in a danger zone. He's not doing it in home port. But that is an autonomous AI, of a different name, not generative AI, that does successfully defend the ship.

So I think what we need to get to is, through a combination of more rigorous AI that's less brittle than generative AI, combined with formal methods and other means, we can get something like that, where if you decide you're now under siege, you flip the switch and the autonomous defenses can go on. But right now, as we've already observed, we don't have enough knowledge of the second, third, and fourth order effects in an enterprise that it wouldn't be used against us.

Rob T. Lee: I believe on the defensive side, the human in the loop is going to remain, and when it's not, there are two things that are simultaneously going to happen. You're going to be able to board a plane, look up in the cockpit, and there's not going to be a pilot there. If you're comfortable with that, you're comfortable with cyber defense having no human in the loop.

The second side of that: you go into the doctor, it says you need surgery, and a humanoid robot comes in with a scalpel and is ready to sedate you. Are you totally comfortable with that? When these two things happen near the same time, I believe the human in the loop will be removed from cyber defense. We have some years to get there.

Michael Krigsman: Okay, I go back to what I said earlier. I'm going back to hiding once again. Because that seems like the most prudent response to all of this. All right.

Rob T. Lee: Have you gotten into a Waymo yet without a driver?

Michael Krigsman: I live in Boston, and no, I have never used a Waymo without a driver.

David A. Bray: And even on the Waymo example, let's say Waymo's got everything working in the AI. We saw when the power went out, they stopped, and that was not a cyberattack, it's just the power went out. I've also seen foreign jammers that can basically squelch any comms, any geolocation within a 3-kilometer radius. The reality is the world is interconnected, but this is why it's important to have folks on the defender side.

AI trust, hallucination, and information warfare

Michael Krigsman: Let's very quickly go to another question. This is from Haider Sousa, who says, one of the current AI challenges is that there is no way to distinguish when AI provides the correct information or hallucinates. How can we boost the trust of the information provided, and what role should governments or official entities take? Rob, maybe you can talk about the issue of boosting trust in the information coming from AI systems, and David, very quickly, you can talk about the role of governments and official entities in this.

Rob T. Lee: From a trust perspective, how do you trust it's not going to hallucinate? Do you trust a human? You employ someone, how do you know they're going to be 100% perfect in their answers? How do you know they're not going to read something and interpret it incorrectly?

When you look at prompt injection, it's no different than information warfare campaigns from, say, Russia claiming there's a massive groundswell of feeling on a particular topic and then everyone believes it and that becomes the truth. How do you prevent humans from doing it is the same answer as how do you prevent agents from doing it. We have to start looking at these technologies as capable of reasoning. They're able to be convinced by bad data. They're able to potentially interpret things incorrectly.

If you're not able to trust humans, and we have them employed everywhere, including in critical infrastructure, flying planes, going back to that example, and even bad data coming in there has resulted in bad things happening, there's very little basis for assuming the system will be perfect. With AI and agents, one plus one is going to equal three in many cases, just as it happens for humans. Mistakes will happen, and we're not going to be able to get around it.

David A. Bray: If I can say spot on to what Rob said, I often define trust as the willingness to be vulnerable to the actions of an actor you cannot directly control. That could be a human, that could be an organization, that could be an AI. And what it shows is that if you perceive competence, perceive benevolence, and perceive integrity, you're willing to trust. Well, the reality is, as Rob was saying, foreign actors can manipulate those perceptions so that you either do trust something you shouldn't or vice versa. But these dynamics are true for humans as well, so we should approach it the same way we hold organizations and humans accountable, recognizing that they will make mistakes, but holding them accountable if a mistake results in harm or loss of life.

I don't think you can have government regulate what is good information. That is a slippery slope to really bad things happening. I will give a shout-out to Ellen McCarthy's Trust in Media cooperative, which is trying to give people the tools to do this themselves or in communities. And the last thing I will say, I have seen already that generative AI is really good at building psychological profiles of each of us, thanks to our digital exhaust, and figuring out that if it presents this information to David or Michael at the right time, it can possibly get them to take a particular action.

That's not just a cybersecurity risk, that's a collective free society risk. We need to recognize that if our emotions suddenly spike, if we get angry or fearful, that might be a case where an AI agent is trying to deliver information to mess with our emotions and get us to take a particular behavior. And that's where I tell people: always take a breath if all of a sudden you feel your adrenaline skyrocketing.

Third-party assessment will become mandatory

Michael Krigsman: Rob, this is from David Batz. What factors should be evaluated to determine the need for a third-party AI risk assessment? Really quickly, please.

Rob T. Lee: Almost 100%. Are you going to hire someone with no assessment of their skills, capabilities, knowledge? It's going to be 100% across the board. We're going to need to figure out some sort of trust assessment capabilities for AI. Right now it's a chatbot, and we think of it kind of like Google. But once we start letting autonomous decision-making occur in your home, that's where you're definitely going to want to ask who assessed that thing to be able to make a decision, even on lower-threat things that are occurring in the household. It's going to need to exist somehow. You're going to want to know how does this thing know its skills, know its actions, and reason properly.

Michael Krigsman: Reza Fatahi comes back with a very quick question. He says, what are acceptable latencies for a defense infrastructure for a decision or a deliberative process, and for key decisions or a deliberative process in terms of APIs? Do either of you happen to know that?

David A. Bray: There's no magic number, but I think what it's showing, as Rob has been emphasizing, is that we do need to up our game on the defender side, and it's about identifying: is this something suspicious but we're not under siege? Are we now under siege, and do we need to increase our op tempo? I don't think you can necessarily give a specific latency number for the enterprise, but it does say you should be able to rapidly respond and mitigate what you perceive to be attacks at a much faster rate than currently, and that's going to require new playbooks, which is why I'm excited about the innovations that will happen on the defender side in the years ahead.

Security culture starts with leadership

Michael Krigsman: Rob, what does a security culture look like in practice, especially given AI and autonomous systems and all the things we've discussed?

Rob T. Lee: I don't think it's any different from any other career field that's out there. Everyone is running around with a lot of angst staring at AI. Is this going to replace my job? What new skills am I going to need to maintain in the future? The security culture is led by the managers. If the managers are not really emphasizing that AI is going to have a positive impact on you, your career, and the business, then that's where the skilled security cultural vulnerability actually exists. It has to be led deliberately, and they have to show that you're not at risk from learning AI and looking to AI to increase efficiencies in your organization. Otherwise, you're putting the organization at risk based on the offensive capabilities that exist today.

Boards must reward detection, not just prevention

Michael Krigsman: David, looks like you're going to get the last word here. Organizations are building critical systems on data they can't fully verify and very often don't fully understand. What should boards of directors do, and should boards treat data provenance as a fiduciary responsibility?

David A. Bray: I want to double-click on what Rob said, which is that boards, most importantly, even before they tackle data provenance, should be asking, "What type of culture are we creating for our cybersecurity professionals?" If it's a shame and blame culture, if it's, "How dare this ever happen," then I guarantee you your company is not going to exist in 3 or 4 years probably. If, however, it's the idea that this is part of doing business, we want people who are learning, we want people who are adapting, and that includes data provenance, but also: how many people verify that the hardware chips really are what they say they are? That's a geopolitical reality too.

That's not something the board has to get into the weeds of with cybersecurity professionals, but they should be asking, "What are we doing to learn and adapt and actually have the ethos of a culture that says it's about rapid detection, rapid mitigation of data issues, hardware issues, or cybersecurity issues?"

Michael Krigsman: All right, and with that, we're out of time. This conversation could go on, and I hope you will both come back and join CXOTalk again so we can dive more deeply into these issues, because we barely scratched the surface today.

David A. Bray: I want to do it with Rob over a beer, so maybe next time in person.

Rob T. Lee: We'll ostrich ourselves first, and then have the beer underground.

David A. Bray: Then we'll be a lot more comfortable.

Michael Krigsman: Well, that sounds good. A huge thank you to Rob T. Lee, Chief AI Officer of the SANS Institute, and to David Bray, Chair of the Accelerator at the Stimson Center. Gentlemen, thank you so much, and a huge thank you to everybody who watched. Before you go, again, subscribe to the CXOTalk newsletter at cxotalk.com. We'll send you updates. You are now part of our community, and we want you. Join us. We have great shows, amazing shows actually, and we'll see you again next time. Take care, everybody. Bye-bye.