Managing AI Agents Is Now Everyone's Job:
Salesforce's Chief Ethical and Humane Use Officer

Paula Goldman argues that anyone assigning work to an AI agent becomes its manager and is accountable for its work. Goldman is Salesforce's first Chief Ethical and Humane Use Officer and EVP of Product. Her new book is Manage the Machine. In her words, "We are all managers of AI agents, and it behooves ourselves to think of ourselves like that because ultimately we are accountable for the work that they are doing on our behalf, and we need to know how to manage these tools to get the right results."

For example, Goldman treats the hallucinated legal briefs in court filings as evidence of overreliance on AI. She keeps checking a tracker on the internet: "It's now well past 1,000 cases of actual citations by courts. It just keeps happening." Her reading: "Does that mean that the tools are not fit for purpose? No, it means that there's a principle around human accountability and oversight that we need to be leaning into a little bit more."

A friend of hers, a partner at a big law firm, considers how much autonomy to entrust to AI, much as she thinks about a new junior employee, and Goldman noted that the friend is not conflating people and AI. The friend's approach, as Goldman relayed it: "I give them some rope. I see how they do. I give them enough context. And then I see what the result is. And then based on that, I can expand the scope." Goldman calls that the skill of delegation. "The book is called Manage the Machine for a reason."

Human at the helm sets terms before the agent runs

Goldman's alternative to human in the loop, which she calls human at the helm, is to set an agent's parameters in advance and have anomalies escalated to the person. Once ChatGPT came out, she said, people understood human in the loop "maybe in the wrong way": "It was like, okay, AI is going to draft this email, but I'm going to edit it and send it, which is fine, but doesn't really work for the world of AI agents." "It is not practical for a person to look through 50,000 data points and verify each one," she said. "It sort of defeats the point of having an agent."

People stay accountable and in control, she said: "You're able to set the parameters in advance, that anomalies are going to get escalated to you. So that's what we call human at the helm."

First step with an agent: Define the goal and limitations

"When you're setting up an agent, the first most important thing you do is you say, what topics, what is the goal of this agent? What do you want it to be working on? And then you have this on-off topic control, which then puts certain topics or tasks out of bounds," Goldman said. That step "sounds very commonsensical," she added, "but it is the most important thing."

Asked by host Michael Krigsman about hallucinated facts in a marketing campaign, she answered with limits on the agent's domain of knowledge and brand tone: "This is why you need those controls that are not probabilistic, that are in fact deterministic, and you get the right results by mixing these things together."

A LinkedIn viewer asked how a company proves its policies were followed. Her answer began with the controls built into an agent: "Audit trails are one of the most important of them because it gives you receipts, tells you exactly what the agent did." Asked whether that record would satisfy an outside auditor, she said yes, the data is there "if one wants to bring in an auditor," though she called that field "a very nascent space."

Put the human first, as IKEA did with its couch

Goldman argues for having people challenge their own thinking before the AI gets the task, a practice her team is starting to call human goes first. Her example is IKEA, which wanted a new concept for a couch and brought in AI: "Surprise, surprise, it keeps reverting to the mean. It keeps coming up with the same cliché of a couch." IKEA then did its own thinking first, and that led to "Couch in a Box, which was this lightweight aluminum frame that could carry with one hand." It weighed about ten pounds and ended up in a museum exhibit.

An AI model has access to more data than she could read in a lifetime, Goldman said. "What it doesn't have is the contextual understanding of, I'm sitting here with you exchanging visual cues. I know, how am I feeling in my gut? What is all my experience leading up to this moment?"

Advice to CEOs starts with who is in charge of the AI

Goldman's first piece of advice to CEOs: "Think about the accountability structure that you have around AI. You wouldn't open a new store or hire a new person and give them no one to report to, no one in charge of the store."

Her second piece of advice: "Employees deserve a seat at the table in shaping how AI is going to shape their work." One friend visited an auto factory that was training its workers on robotics, and the instructors were deliberately people who had been on the shop floor. "Your people remain your essential advantage," she said.

Expect the noise around AI to settle, as it did with the internet

Goldman's larger point is that managing AI is not as hard as the public conversation makes it sound. Her comparison is the internet: "It's noteworthy that you and I are not having a conversation right now about the internet." They might have been 15 or 25 years ago, she said, "but we've sort of metabolized it. And it's not novel. We know how it works. And this is the moment of confusion that we're in right now with AI." She summed up the work this way: "I do think it's actually not that complicated. We're just talking about good delegation, thinking through what we actually want the tool to do on our behalf, and making sure that it's not going outside of those parameters."

After ChatGPT came out at the end of 2022, she said, accomplished people began calling her, with some anxiety, to ask whether AI would make them obsolete. The point of the book was to say, "Actually, no, you can understand it," and "that confidence in our own ability to manage these tools is actually how we get the right results."

Watch the full conversation with Paula Goldman and read the complete transcript on the episode page.

CXOTalk prepared this article with AI assistance from the verbatim transcript of episode 931. Quotations are unedited from the broadcast.