Why Trust Limits AI Agents in Banking:
Oliver Bussmann on the Controls That Must Come First

Oliver Bussmann has spent more than 35 years in enterprise technology, inside banks and outside them. He is the former group CIO of UBS, was CIO of SAP before that, and now advises banks across Europe. On CXOTalk episode 925, he described what actually paces agentic AI inside a regulated institution, and it is not capability. "The technology is there," he said.

He opened with the event he expects the industry to face. "I'm waiting for an incident in the industry. Hallucination is, or some other issues, then your whole trust and reputation can be gone in a few minutes."

Adoption is wide, production is narrow

Bussmann put current usage at "Roughly 50% of financial institutions are already using agents in an environment," then qualified it in the same breath: "I think most of them are proof of concept." Most of that activity sits in software development, where he sees a move "from, I would say, co-pilot work with using agentic AI to autopilot solution with using agent in different function from back office to IT to marketing."

The delay between those two states is structural, not technical, because every prior wave imposed the same tax. Each new technology, he said, means "every time if you implement a new technology like, you know, a couple of years, software as a service, cloud, or blockchain, you have to adjust your risk method, your control environment, your processes." The result for agents is a long runway, in which "we will see a lot of testing, proof of concept, but huge production cases always takes time to make sure everything is bulletproof."

Geography adds friction. Bussmann noted that "European regulation and laws are much more restricted than the US one," and that for an institution operating in many jurisdictions, "regulation drives fragmentation."

Trust is the asset that cannot be rebuilt quickly

When the host suggested he was looking past the mechanical failures, the hallucinations and making things up, toward a higher level of judgment issues, Bussmann refused the split. "It's both," he answered, before naming what sits underneath: "the biggest value that, you know, a bank has to take care is trust." Customers rely on the institution to process "all my business in the right way and it's accurate, it's traceable, it's auditable."

He pointed to a recent example of erosion. "KPMG a couple of weeks ago was under pressure," he said. "They put a research report out there and about usage of AI in financial services, and there was a lot of hallucination in there, wrong data," and financial institutions "had to push back on that."

Then he named the failure mode that makes banking different: "if you look at big crisis in the bank industry, bank run is based on people, customers are losing trust." An accuracy problem in a bank does not stay inside the workflow where it happened.

Risk classification sets the autonomy line

Asked what separates an agent a bank will let advise from one it will let execute, Bussmann pointed to classification. "It really depends on how these use cases have been classified as a risk that could lead to wrong outcome impacting customers, impacting businesses." That rating then determines the controls, because "this classification drives the level of cross-model validation, human verification, cross checks, et cetera."

He was specific about the top of the risk scale, and about what he meant by it: "everything that could be biased into a credit decision or a security oversight, it means monitoring people on the street." That, he said, is what "I classify this as a high-risk topic." His reading of the regulators' motive was consumer-facing. European supervisors are "keen on consumer protection, client protection," and want assurance the technology is "not driving a disadvantage for consumers."

For high-stakes output, he applies a rule from his own practice: "cross-model validation for super important tasks is mandatory because it gives you a much higher confidence that whatever you produce is correct and validated."

Engineering shifts toward guardrails and evidence

Bussmann sees the engineering role inverting, and he scoped the observation to the software companies he works with rather than to banks. "If I look from a typical software company perspective, I think the engineering team and head of engineering is, you know, the job description is changing from, you know, actually doing the software coding, et cetera, into defining how these agents are being defined and set up, and also that there's sufficient testing and oversight in place."

Inside the banking answer, he described the same direction, saying "the engineering function is changing to really guardrails, quality assurance, et cetera," with risk and audit "constantly monitoring the environment." Human oversight does not disappear, because "there will always be a supervisor, human supervisor in the loop," and the management question becomes "how many agents you manage as a department head in the future, in respect to how many people you're managing."

He also insisted on measurement before claims, saying "baselining the productivity is absolutely necessary," because only then can a bank test whether the reported multiples are real, using metrics banks already run: "how many lines of code you develop, their quality KPIs, how the code is being structured."

The bottleneck moves to the business

When an audience member argued that coding was never the real constraint, Bussmann agreed without hedging, saying that "if you have set up your right agent, coding is not the problem." The work shifts to people who can define what should be built, so "we will have much more focus on people, front office, business analysts, product managers dealing with the issue instead of, you know, that the engineering team is becoming the bottleneck."

That shift reaches hiring. He reported that "demand for junior software engineers is dropping, you know, the job advertisements are down by 40%," while firms increasingly want graduates with an AI background placed inside business units.

The limit he does not expect to move

Asked whether AI native banks will emerge, Bussmann accepted the premise and then drew a line. "Do I think over time there will be an agent-based bank out there? Yes." What he has learned across 35 years is that "clients love to talk to a person, to a human person," so his conclusion was flat: "I don't think that we will have a fully automated agent-based bank out there without any human-to-human interface."

His forecast for the coming year is still bullish. "I'm very optimistic that we'll see a lot of agents across, you know, all functions in production with the bank because the upside, the productivity gains and the quality improvements are significant." The driver will be competitive rather than technical, because once a rival posts those gains, "the peer pressure will even drive more adoption."

Watch the full conversation with Oliver Bussmann and read the complete transcript on the episode page.

CXOTalk prepared this article with AI assistance from the verbatim transcript of episode 925. Quotations are unedited from the broadcast.