AI and Collective Intelligence for Smarter Decision-Making

Why AI will never replace human judgment

Sandy Pentland: Do they just set up the AI and walk away? If they do that, they lose a ton of money.

Michael Krigsman: Companies optimize AI for individual productivity, but is that really the right approach? Alex "Sandy" Pentland is a professor at MIT and Stanford with over 165,000 citations for his work on collective intelligence and trust.

Sandy Pentland: AI is, in some ways, never gonna be as smart as people. AI is trained on backward-looking data, so it can't really know what's going on in the future. So when it says, "Here's an answer," it's just based on the past, and as we all know, past results do not justify future answers. That's one thing.

The other thing is they're not context sensitive. It's like we live in this world, and we see how people feel about things and, you know, all this sort of little incidental stuff, how do the young people feel? That context is important in figuring out what's gonna happen next, and AIs don't have any of that.

And for instance, a way to think about this as just sort of a bellwether is look what people do in finance. Finance is very straightforward, whether you win or you lose.

But what they do is they realize that all the data that they have is actually backward-looking. It's usually overfit, so it doesn't predict the future very well, and so they have people in there adding their intuition, and in fact, that helps returns absolutely. It is just not a question. People plus AI is better than AI, and it's better than people. And a lot of the changes come from sort of when things change or when there's risk, the people become really important.

What is community intelligence?

Michael Krigsman: You have systemized thinking about this. Your latest book is called Shared Wisdom. You talk a lot about collective intelligence, or your term is community intelligence. How does that fit into this picture you're describing?

Sandy Pentland: We live in a world where we have this thing called culture. There's stuff that everybody knows. Some of it's right, some of it's not right, maybe it's not posed quite right, and then there's stuff we've heard about, which is sort of broader context. And when you're running a company or an organization, you really want to have a culture that is compatible with what it is you're trying to do. You need to have people aware of all the things that are going around that might come and, you know, sideswipe you. So that is community intelligence. That's awareness of the whole sort of environment that you operate in, and the shared wisdom is the things that everybody believes, some of which might not be true. So you need to continually involve that, and the way you do that is you have.

You try things out, you see what they work, you have a discussion about it, you see what other people did, and it's this process of refining the wisdom about what happens that produces the culture. You might recognize this as science. That's really what science is. People do these experiments. They talk about it with each other. Eventually, they figure out how to do things better and predict the future, and now you can just build things 'cause you now have these sort of rules about stuff until the rules change.

But you know, so that's the sort of thing that a company is meant to do, is it has to have a culture and a sort of a community intelligence that fits with what's actually happening so you can predict where to put your efforts and what to be able to do, and AI can help you with that, but it can't replace that anytime in the near future.

Three ways AI augments human intelligence

Michael Krigsman: How does AI overlay onto this, these concepts of shared wisdom, community intelligence, and the culture you're describing? Because so far, what you've said is not that different from issues of integrating technology into organizations going back to the Stone Age, or at least till the early days of computers and enterprise software.

Sandy Pentland: Let me give you an example. So I was on a panel with the CTOs of some very major corporations, and we were talking about the real problem is getting the right data, but that's for later in this discussion. The thing that they sort of said that they had done for everybody, and I've since seen other people doing it, is they've created AI buddies for every employee. So what is an AI buddy?

An AI buddy is something that's read all those manuals and all the reports and the newsletter and knows what the people down the hall are doing and the people in another, and so you can use it to keep you in the loop, to provide context for your decisions, to know what people do, what works, what doesn't work, and that's really different than just something that, you know, produces a number or some other sort of thing. People don't read those manuals. People can't read all the different sort of reports out there, but the AI can, and it can pick out the things that might be relevant for you. So it's like having a really brilliant librarian, perhaps, or something like that to sort of help you stay in the loop. That's really critical.

A second thing that AI can do, and I talk about this in the book, there's the sort of, you know, personal things. There's the group discussions. The strength of humans is combining views and battle testing them against each other, and, you know, everybody hates meetings. Why? Because we get distracted, and there's all sorts of weird stuff and power dynamics.

And so we've built tools using AI, and you can look at Deliberation.io, all free, all open, where we built little tools that essentially just reflect. They don't give answers. They say, "I hear people saying this and that." And what that little sneaky sort of reflection does, is it keep people focused on the problem, and it makes sure that the loud voices don't dominate as much as they tend to do.

And so this little, tiny thing is just like a mediator, except it hasn't got, it's not human. It's just an AI. Makes people's meetings more than twice as efficient and effective, according to the people, than having just humans. Think about that. If you could raise the efficiency and the accuracy and performance of your meetings by a factor of 2, you'd be golden. And it turns out that's a very simple sort of AI thing to do, as results on their summarization.

And then the third thing that people need to do is people need to do what I think of as exploration. You know, there's a big world out there, people are trying lots of things. Some of the things are pretty interesting for what you're doing.

Maybe not exactly the same, but you ought to hear about them, and we're always here finding things where, "Oh, really? They're doing that?" And that's interesting. Maybe if we did it some other slightly different way, we could do that too. That sort of exploration of the world is something AI is just really good at.

And so using it for exploration, using it for finding consensus and battle testing ideas, and then using it for people, keeping people in the loop, are wonderful places where it doesn't detract from the human intelligence, it augments the human intelligence. Yeah, some tasks will get more productive. That's all fine, but actually, what happens is that the tasks will become much more complementary and reinforce each other.

Instead of having a bunch of people off doing things separately, you now have a team, a great big team, that's like, looking to the future, looking side to side, and making decisions and acting together. And I think everybody knows that that's what a leader has to do, to get everybody going in the same direction and pick the right direction.

Training people to question AI

Michael Krigsman: You're really talking about using AI far beyond efficiency. It's not just a matter of executing a particular process more quickly, but really into the heart of innovation. 

We have some questions that are coming in now on LinkedIn and Twitter, X, and I just want to remind everybody that right now you can ask your questions, and I urge you to do so. Take advantage of this opportunity to ask Sandy Pentland, you know, pretty much whatever you want. So take advantage of it. Let's get some of those questions. 

And right now, we have a question from Anthony Scriffignano, who says: "Can you talk about how humans are adapting the way they formulate questions to accommodate AI, and how that adaptation may be subtly influencing innovation?"

Sandy Pentland: The way I see, the way the data sort of show people using it is, yes, they're using it the way, to do this sort of, exploration and battle testing of ideas. So normally, people are used to looking things up or asking Google or something like that, and now you have a much more interactive medium to do that against, and so that changes things.

In the experience I've seen, that changes things for the better, because what you're doing now is you're training the person to be, in some sense, skeptical. They're asking about questions. They're prying, "Well, why do you think that?" Or, "Why is this true?

Or is there anything." So they're being much more engaged with the problem and much more questioning of it, because the medium, the AI medium, is much more responsive in this sort of way. And that's really the right way to be using it, because you don't want to trust the answers. It's not a matter of hallucination, it's a matter of, does it have all the data? Does it know what's going to happen in the future? No, it doesn't. So you want to be in there and use it as a way to query the world, to query what you know. And if you have that sort of, you know, exploration mindset, that's different than the way people usually do things.

Culture is co-created, not commanded

Michael Krigsman: We have another really interesting question from Twitter, from X, from Arsalan Khan, who says: "Culture, in some ways, is a reflection of executive leadership and the incentives and disincentives. If AI fails, should we just blame the executives?" Or I think also, more to the point, how do we assign blame between the executives and the AI?

Sandy Pentland: Culture is actually something that's co-created, and leadership has outside influence because they have control of things like incentives and so forth. And plus, people assume that they have more data, more vision, broader vision, than they might have. One of the things about AI is then you can have individuals, not just the leaders, have very sort of broad vision, which means the discussion is not as top-down as it used to be.

I think you need to regard culture as co-created, where the leader is sort of the conductor of the orchestra, but actually the sound comes from the orchestra, and it's that that is the final sort of product. So I think that notion of blame is probably the wrong way to conceptualize it.

Yes, people will do that, but the way to do it is, are we making good music together? Okay, and if the leader is screwing things up by jerking people in different directions and going against what other people think and not explaining so that they can get on board, then the leader needs to change what they're doing or get out of the way. I don't know it's a matter of blame per se, but it's a matter of, you know, again, think of the conductor of the orchestra. If their style doesn't match the style of the orchestra, you got a problem.

Why humans still out predict AI

Michael Krigsman: But what happens as AI gains greater, quote-unquote, "intelligence"? Does the AI, and of course, the AI is trained on all kinds of different sets of data, and so what happens to the culture that is then co-created by the human masters and the AI team? You were talking earlier about AI buddies.

Sandy Pentland: AI is trained on the past, not the future. It lives in a box, not out in the world, which means it's less sensitive to context and current events. It doesn't really have a notion of causality, what causes X. It has notions of what people say, but we all actually are able to predict the future, at least in the short term, better than the AI. And particularly in groups, this is just absolutely quantitatively true.

So I think, you know, one thing to keep remembering this is, what do the quants do? What do the hedge funds do? Do they just set up the AI and walk away? And the answer is, if they do that, they lose a ton of money. Why is that?

Well, because of the back testing, back view, because of the lack of context, because of, you know, the notion of knowing what causes what, and those are the hottest topics in finance: causality, things that the AI can't really do at all. You can't do it without experimentation.

So anyway, to go back to the question, if everybody has this sort of, "Okay, we're querying the future, we're querying the facts. If you have new facts, bring them in and contribute it, that'll be great," you know, the leader can provoke a discussion about longer term things.

So this is one of the things that AI allows you to do that we don't do enough of today, which is we often talk about the day-to-day things and what are the bugs that happen, but a leader has to be able to say, "Yeah, but we're going in four or five years to this place," and actually, they ought to be thinking about twenty years, forty years. Other parts of the world actually do this.

So for instance, if you look at investment funds in China, 40-year time horizons. If you look at investment funds in the Middle East, 40-year time horizons. They're asking: we're on this journey, which directions are we headed? And people need to have that in the culture.

They need to understand that, you know, we're walking in this direction, and, yeah, we're playing with today's problems and so forth, but the long-distance sort of future is over there. And, you know, we're all gonna try and make choices that head in whatever that direction is.

What would be a good example? Say, you make products of various sort or services. Well, one of the things you know is that the population is getting older on average, much larger aging. Okay, well, if your services are relative to helping people age in place or people getting more mature, then you ought to think about, okay, so 20 years from now, these are gonna be the demographics. The market's gonna be huge, maybe plus or minus you, what you feel about that, but that's an incredible opportunity, and you know that's gonna happen.

And the question is, are the directions you're taking in that direction so that you can capture that sort of opportunity, or are you marching in another direction? It's not gonna really change your day-to-day so much, but it is gonna change the direction you step in, at least each little decision that you make.

And so the long range, that's this exploration, which AI is pretty good at. The discussion of what direction to go, well, that's like this deliberation.io thing, where it's groups of people, you know, battle testing ideas with each other. And then the AI buddies is, what's going on right now, and what can I do that's gonna be the best decision?

The distributed workforce of the future

Michael Krigsman: What are the problems and the downfalls, the potential downfalls of doing so?

Sandy Pentland: There's so much we don't know that's going on, particularly in an organization. You know, what is everybody else doing? And I'll tell you a little story about this later about what some of the world's largest organizations are doing about this. But when you do that, when you query the AI, you have to have a query mentality, an explorer's mentality. You really shouldn't believe what it says without having some sort of check, either internal or external. 

It's a brainstorming buddy. It's not always correct. It doesn't know about the things that are happening today. It doesn't have an understanding of the human sort of side of the context.

But, and this is the story that about what some large organizations are doing, they're realizing that this sort of AI buddy giving you a sense of context for what everybody in the organization is doing is perfect for work from home. A big problem with work from home is you don't know what the sort of cause of the day is and what other people are doing, or distance offices, offices in other countries. So having offices that are in different countries that talk to the actual customers there person, you know, on a more personal basis is great, except that they don't know how everybody else is doing it, and there's no good way to spread ideas. This is what AI buddies are perfect for.

It's keeping everybody in the loop, and what I just said was, for instance, here's one example of a leader of a large, multinational, three hundred and fifty thousand employees. What he says is that he's going to move down to 100 and 50,000 employees that are in-house, but he's going to double the size of the other sorts of employees that, so he'll actually be hiring a hundred thousand more people, but they'll be based on projects. They're not in-house employees. They're things in different countries, working with different sort of customers and so forth.

And so what he's describing is a company that's much more distributed, that's much more relating to its customers, that has these electronic AI sort of aids to keep its actions coordinated and keep everybody on the same page. And I think that's sort of the biggest story that's out there.

Companies, rather than having these big sort of hierarchical structures in silos, are going to be much flatter. Teams are going to be much more agile. The teams are going to be smaller. From the view of sort of information and context, silos in companies, in layers of management, are exactly what you don't want. They're sort of a recipe for A doesn't know what B is doing. And the AI can help you sort of blend those and make people much more aware.

So many of the leaders that I talk to are envisioning a future where most groups are of 5, 6 people of different backgrounds so that they can talk to each other about different perspectives using AI, and they're relatively autonomous. They're not strict orders from on high.

The organization is, in some ways, much more like a venture capital, or actually more accurately, like an incubator, where, yes, the organizations get funding from the top. They have to report back to the top, but the top gives them a great deal of autonomy, and they're able to go out and then really act.

The thing that makes that possible is that now 6 people can control a business, set up and control a business that's worth billions of dollars a year. You can set up a finance company, a payment company, with a small number of people, and that's amazing! You never used to be able to do that before, or a logistics company, or any of a number of other things where you're getting teams together very quickly, producing things. Software becomes very cheap.

So, I mean, I'll just give you one story. It's from a friend of mine. This was a doctor in a hospital, no technical background. He saw that the hospital was just. You know, people were not coordinating with each other, and of course, bad health things come. So this doctor, with no technical background, got a couple of AI tools, a friend or 2 that knew a little more about it, and in 6 weeks, they built an operating system for the hospital that the hospital ended up adopting and having much greater results and much better efficiency. So look, if a small group like that can do something as complicated as a hospital, and that's a real example, then think about what's gonna happen in the future.

AI is about bringing lots of data together, making some sense of it, and then causing things to happen, and if you have a good group of people that are working together and sort of trading perspectives and things like that, you can make magic happen.

Rethinking the social safety net

Michael Krigsman: Folks, I encourage you, ask your questions. Folks on LinkedIn, all these questions are coming from Twitter so far. Where are you? You guys are watching. Ask your questions. And, oh, by the way, subscribe to the CXOTalk newsletter, so we can notify you about upcoming shows. Just go to cxotalk.com and subscribe. You'll be very happy you did that.

Anyway, Sandy, you're describing what we could call an agentic AI future, where we have agents and small tools doing hundreds, thousands of little tasks, and the efficiency gains that come from that, and the rapid adaptation, but along the way, won't there be a tremendous amount of job shifting, relocations, and job displacements, which will translate to a very painful intermediate period of time?

Sandy Pentland: If you get this sort of change to lots of small groups doing things, maybe they have the same brand, they're all PricewaterhouseCoopers or something like that, but it's actually lots of independent things that are funded and have some cooperation agreements, then there's gonna be a lot of churn. The main thing that will happen, from a business point of view, is incredible agility and ability to do things and turn on a dime. From a worker point of view, an employee point of view, that means you're gonna be changing what you do all of the time, and unfortunately, in American society at least, we've tied things like healthcare and retirement to employment. When we did that back in the 1950s, when lifetime employment was not a joke, and it's a mistake.

We need to change it in some way. An idea I have and have been promoting, and it's sort of interesting, is, if when you set up a company, you were required to give 10% of the shares, and remember, at the beginning with a company, the shares are worth 0, into a sovereign wealth fund, then, and in return, you would get guarantees about future taxation and stuff, people would go for it.

Now, if we had that in place, say, in 1990, we'd have a sovereign wealth fund worth something like $10 trillion today in the United States, and we wouldn't be having a lot of arguments about social safety net, because $10 trillion can support a lot of people. Just from the interest.

So there are ways of distributing the opportunity, the success, and the failure in a way where everybody has a good minimum. We're not raising taxes. What we're doing is we're having everybody own a piece of the pie, and I think that's really interesting.

I was just, I'm here in the UK, and that idea is really interesting to the government here, 'cause they have the inequality problem, they're facing all these sorts of problems with AI, and they have the problem of: how do you keep people safe? In the UK here, they still have this history of guilds, so you could imagine having professional societies and guilds, and those are the things that run a fund for the people like you.

So if you're a doctor, you join the doctors' guild, and you put a bit of your income in there every time, and then all that goes to all the doctors. Many different ways to do this by aggregating opportunity, aggregating risk, the way that, for instance, investment funds do, and which we don't do today because we haven't really changed the social contract since about 1950, 1960.

Reinventing education for an AI world

Michael Krigsman: I have to say, we started this conversation talking about shared wisdom and community intelligence. We moved, and much of that also improved and driven by AI. We then moved to the impact on the way organizations are constructed, work is created, and you've now broadened this, if we can follow the chain, essentially saying AI is going to drive really basic changes in the organization of society.

Sandy Pentland: The place that it's most visible is in education. I mean, we have this education system, part of it, you know, it's based on a template that comes from 1,000 years ago and really hasn't substantially changed, because knowledge wasn't very dynamic, and the world was pretty static in terms of, you chose a career, you stuck with it your whole life. Lifetime employment was a real thing up until not too long ago.

And so what are we gonna do with AI in classrooms? And let me give you an example of something that I was part of. There's a small country in Latin America, Costa Rica. They realized that they were gonna be growing coffee beans forever unless they could up their game and become, have a population that was technologically sophisticated, and they made an investment. They gave every school kid a computer.

This is back in the seventies. And they insisted that they speak the technical language of the world, which was English. So this is a Spanish-speaking, coffee-growing region, deciding that every kid is gonna be tech-savvy and be able to communicate in English, and as a consequence, they got Intel to build a fab plant there, they got software there. They're doing quite well, thank you, because they upskilled the population this way.

What would be the analogy in the United States for us as well? Why don't we give kids in 6th grade laptops with AI and ask them as groups, not individually, to build software that does things that they think are important, you know? Takes care of old people, makes traffic better, and what they're gonna learn is they're gonna learn how to work with each other in the presence of these IT tools.

And we see that, for instance, I helped shape the Media Lab at MIT. That's what it is. It's not an educational program in the traditional way. It's an apprenticeship program. You learn by building things and by sort of pressure testing them against company sponsors, against the real world, and various sorts of things, and in that, you don't just learn about the tools, you learn about the business, you learn about the context, you learn about the social issues of what you built.

And you need to have all of those, not siloed into a degree in X or a degree in Y. They have to come together, and you have to be able to use the technical tools to be able to essentially dance between them and create a new future. And we've done pretty well.

I'll brag, humble brag or brag a little bit. We had, I ran a class on entrepreneurship, where the challenge was you have to. A small group, it uses these sort of AI tools, and these are older AI tools now, that creates a business that touches a billion people in the world. And those billion people have to be outside of the rich countries. They have to be in the poor countries. And guess what? We have a couple people who did it. It's like, that's pretty amazing.

So these kids can really do things if they learn to combine the problem area with the tools, with the social organization to work in a team, and the passion to go out and make it happen. And in the end, that's what we need more of.

Michael Krigsman: I will mention the CEO of Zoho, Sridhar Vembu, who's been a guest on CXO Talk a number of times, has a concept called transnational localism combined with what I've called, when I've spoken with him, social capitalism, using Zoho as an agent of change to support technology upskilling in local communities. And I will say that they have put their money where their mouth is. I mean, Zoho has invested in this in a number of communities around the world. It's pretty amazing, actually.

Sandy Pentland: It needs to be the shared wisdom that that's the way you do education, not drill and kill. And currently, almost everybody does drill and kill. So that sort of notion of having a creative education, an education that actually builds things and brings in lots of disciplines and lots of tools is, I think, sort of the key thing. It does get us, if you want to go there, into what people doing outside of the US, because what they're doing is very different than the way the US thinks about it.

Data is the real battleground

Michael Krigsman: We have a question that came in early from Chris Petersen on Twitter, and I know there's some questions on LinkedIn, and we'll get to you guys next. And Chris Petersen says: "Do you see LLMs, GPTs, GenAI fulfilling the promises made by their boosters, or will AI pivot again to include other, maybe older forms, to round out their actual capabilities?"

Sandy Pentland: All of the sort of modern technology, the LLMs and GPTs and stuff, are really pretty good at being speech interfaces, summarizers, librarians. There's a number of things that they do surprisingly well. They're basically looking for long-term correlations and relationships that are so vast in terms of the data and the distances, that humans just can't see them, basically. But they're not reliable in various ways. These are correlations. They're looking backward and so forth. So there's really a great need for some of the older tools to be part of the mix.

A key thing is something that we're running called loyalagents.org. It's with Consumer Reports.

And the law school at Stanford, and what we're doing is we're saying, "Look, if you're going to have an agent buying things for you, it had better represent you and not some other company." I mean, this is a basic for doing, handling money, and they don't do that. Also, it ought to be auditable. I mean, if somebody says, "Oh, you're self-dealing or you're biased," you have to be able to audit that and answer it.

So those are places where more traditional machine learning or expert system type of technologies and other things like that have a real place. If you take something like law, which is a really hot area right now, law is a big expert system, okay? But finding which pieces of law are likely to be interesting is a good LLM problem.

So it's a mixture of the 2 things, to be able to ask, "Is what I'm doing legal right now, or am I going to get in real trouble?" Same thing with payments of various sorts. Is this something that is likely to end up poorly or something that ends up better?

And so I think that there's lots of room for growth, but in the technology and evolution, the main thing that I see as sort of the future of this is, yes, there'll be this evolution and so forth, but all AI is driven on data. All AI is patterns and data, and so increasingly, access to data is going to be the thing that the battle is about.

And you said we wouldn't talk about international, but I'll tell you, that's what the international people are doing. They want to run all the systems in the world, so they have access to all the data in the world. And I think that that's something you really need to think about, is if you have the data, the technology will get there to be able to use it in ways that are interesting, but if you don't have the data, you can't get there.

An interesting statistic, something that people ought to pay attention to, there was just a study of all the open source LLMs, and what they found is, for a given level of performance, the cost of doing that with the current state-of-the-art LLMs fell by 50% every 3 and a half months, and by a factor of 10 every year. So that means in the last 3 years, the cost of a given amount of intelligence has dropped by order of 1,000.

So when people talk about all the electricity, when they talk about all the cost and stuff like that, they're forgetting that this stuff is moving faster than anything that has ever happened before in terms of technology. And so you're already seeing things that run on big laptops, sort of experimental, high-end phones, things like that.

That's really interesting because that makes it really ubiquitous. It also brings the "who owns the data" question really to the fore.

But one of the things that's really interesting, I have a friend that runs the nation's largest supercomputer center, and, of course, they do all this sort of AI-type stuff, and he says that the moment they were able to get these LLMs to run on super high-end laptops, their productivity tripled. And the reason was, they didn't have to do all the cloud setup and configuration and fight for time, and so they just could do it.

And that's something to think about. So what happens when personal AI is available? So control of data that's proprietary or personal, it's on every sort of device.

That's going to be a really different world than what we're thinking about with, you know, Claude or something like that.

Where to start redesigning work

Michael Krigsman: Let's take a pragmatic organizational question from LinkedIn, from Kapil Chaudhari, who asks an important question. He says: "Leaders understand that AI value requires workflow and process redesign, but many hesitate because they lack concrete examples or frameworks. What practical patterns or starting points would you recommend for organizations trying to reimagine work with AI?" And I love this question because it's really starting to build a bridge between the vision, Sandy, that you've just been describing.

Sandy Pentland: For most organizations, you want to start with things that don't redesign the process. They just make the process better, less error-prone, more consistent. It improves the culture about it. But of course, the real gains come from redesigning the process.

And I think that a good way to sort of think about the process is: where is the data? If there's data that aren't talking to each other, then that's an opportunity, because AI can combine the 2 datasets and find, oh, these things are closely related.

If you find that there are silos that have to do with each other, then the right thing to do is to begin to bring talks between the people who represent those silos and the AIs, and ask, "What are the opportunities for reorganizing this?"

And this is where the human dimension comes in, is you have to make it clear to all the people in the room that if they can reinvent a great process, then they're going to be rewarded. They're not going to be out of a job. So talking about getting rid of people is probably not the right way to start a conversation like that. You want to say, "Okay, well, we have these data cells, these data streams. How can we put them together to build better processes?

And what sort of things can we cut out that have always been a thorn in our side, and move those people into these other sort of processes?"

It's often sort of difficult. I was on the data board for AT&T. You know, they have this problem that, you know, they're a hundred-year-old company, everybody was set in their ways, and they viewed that they had to now deal with people like Google, who were going to be competitors.

And so they put in place also upskilling programs that resulted in, you know, people getting applied degrees, that gave people early retirement if they didn't want to do that. And they tried different organizations in different parts of the company to see what worked with their culture.

And I think all of those are sort of practical things that, realistically, you're going to have to do. You have to experiment. There's no one answer to these things. It depends on your culture, your business area, where you reside, things like that.

Fighting misinformation at scale

Michael Krigsman: This is an interesting question from Lisbeth Shaw, who says: "Social media has a history of magnifying and spreading misinformation and disinformation. Generative AI has a history of sketchy information. How do you prevent corrupted, shared wisdom and community intelligence in this environment?" And I will just mention, for people listening, that we have coming up, I think in April, 2 researchers who are going to be discussing the impact of AI swarms on misinformation, disinformation, and democracy, so this exact issue. So what do we do about it, Sandy?

Sandy Pentland: There's at least 2 major sort of things. One is, you need to have a little bit of a reform for the way the internet works, where you can tell that things are bots. And so we run a group that's in the IETF, which are the ones that set the specs for the internet, to make it easier to identify things and harder to do bad things. And we feel pretty good about ourselves 'cause we're just finishing a spec that's going to cost North Korea about a billion dollars a year, 'cause they steal things. They're not just disinformation, they're stealing things.

And little changes about, where did this data come from, so you have provenance, make all the difference in terms of preventing things like this.

So, you know, it's an active area of research. It's also something that people are beginning to realize is the highest sort of priority, because we're seeing, you mentioned some bad things. Worse things are coming.

So I was just at a meeting with the heads of some of the very largest financial management companies in the world, and they're scared to death. Because just imagine that if all of this stuff is about fraud and stealing money and so forth, oh, my God.

So one is, I'm hopeful that changes will happen there to make it harder for the swarms to attack and easier to defend against them. But that doesn't cure the main thing with social media, which is you get these eruptions of beliefs that are false because it's all based on clicks.

You know, more clicks, more money. Okay, you're going to say something crazy, you get lots of clicks. Okay, so the crazy people are the ones that get all the clicks, and the more crazy you are, the more clicks you get.

There are ways of doing this. So, for instance, if you look at a thing called Polis that Taiwan uses for government conversations, government, "How do we run our country?" They have ways to sort of visualize what the range of conversation is, and what that does is it lets you identify the crazy people and say, "Well, that's a, yeah, he's saying that, but he's one of the crazy people."

We have a version of that called Deliberation.io, all open source. You can look at it, studies, blah, blah, blah.

It does an amazingly good job at this stuff. But the big social media companies have begun doing something called Community Notes. So Community Notes is a very simple sort of AI system, and when somebody makes a comment that's one of these, oh, my God, types of things, people can begin adding comment, their sort of judgment to it.

And when you get comments from a broad enough audience, not just the echo chamber, but a broad audience, you begin to see notes attached to these comments that say, "Yeah, but nobody else believes this." And it gives you these sort of warning signals, and it actually reduces things quite a bit.

The part that's missing is that the people who do this all the time don't have a reputation score.

You know, in the real world, when we talk to people, we know who to listen to and who not to, 'cause that guy always says crazy stuff. Or she's always a little off or not, whatever. We don't have that in social media, and that's one of the things that is, I think, a reform that needs to happen.

So social media needs to be a little bit more like face-to-face communication, reputation, community intelligence, shared in wisdom about it, is what the Community Notes does, and that can help a lot. You know, the things like the Taiwan Polis or our Deliberation.io is really sort of the way to do it because it doesn't have this instability where crazy comments get lots of attention.

That's the thing that you have to watch out for 'cause that's the thing that causes all this sort of BS going on.

Michael Krigsman: But of course, you're fighting the core business model as you describe, which is clicks and attention from some of the most profitable companies on the face of the planet.

Sandy Pentland: Well, the studies that have been done so far is that things like Community Notes, which is actually deployed. It's actually out there.

Michael Krigsman: Facebook actually has that. It's a minor, tiny little feature.

Sandy Pentland: Yeah, yeah, yeah. Yeah, yeah. So they haven't fixed it all, but on the other hand, that little thing does have a significant effect, and it doesn't reduce their profit. So. 'Cause people get involved in commenting with each other.

The right way to do it is to have something that's a different sort of platform that doesn't have that explosive characteristic. I don't know how to do that business problem, but one hopes that some of us, some of the creative sort of people will.

We've seen a number of things that are promising there, for instance, a post-processor. This is like an AI agent. So it looks at your stream, and what it does is it randomizes it, so you don't get lots of, you know, the sky is falling sort of comments all at once.

Or you can adjust it, so you don't hear, certain levels of emotion are downrated. So there are things that you can do post hoc, out of the control of the companies, and we'll just keep working at it.

I think that one of the things that's really interesting is that a lot of. Interesting and scary, is that there's a belief that having AI agents helping you do searches, and find things, and understand things, will reduce the amount of advertising on the web dramatically. And because of legal constraints, if you're actually advising people about purchases and stuff, then some of the legal constraints actually bite, where today they don't.

And so some of the things that drive that click hunger are gonna be changing.

Now, the danger is, of course, if you have agents doing all this stuff, you could have crazy things. And we've already seen a couple of agents that are absolutely crazy, and so we need to be able to regulate that sort of thing. This is the same sort of thing that all the big financial institutions are scared about. If you have agents that can do crazy things, and the poor user may not know, or the company may not know, then you can get huge amounts of chaos in the financial markets. That's not good.

Michael Krigsman: Steve Tout on LinkedIn says that he is doing work. He says, "We need verifiable data sources, and OASIS has a working group on this topic called Data Provenance Standards," and he invites anybody to connect with him if you're interested in this space. And on LinkedIn, he's put some URLs. So if you're interested in data provenance, check out the LinkedIn comment from Steve Tout.

Sandy Pentland: It's really a key problem and something that people are working on. We're pretty involved in that, too. You know, we'll see how all that comes together. The thing that I think will drive it is when people see financial things happening, you know, a million dollars stolen because of data you didn't know where it was come from, then we're finally gonna get provenance.

The global race for data dominance

Michael Krigsman: You started talking several times about data and what various countries are doing internationally. Can you give us some overview of that? Tell us what's going on there and why it's so important.

Sandy Pentland: Well, I'll just start with a little story. So I was at a dinner where some of the San Francisco's leading intellectual VCs, you know, talking about why they're investing and all this sort of stuff. And I was sitting with the head of Singtel, which is the biggest sort of telco in the Indo-Pacific, and Temasek, which is the sovereign wealth fund of Singapore. And they were having a hard time not laughing out loud because they think that this notion of frontier models and intelligence that is gonna run everything is just crazy.

What they see is that if you have cheap, accurate models that run every process, every business process, logistics, ordering, health, et cetera, if you make those agents, then you have all the data in the world.

And if you have all the data in the world, you can be able to predict where to invest, where not to invest, what processes to put in. You are in the catbird seat. And it doesn't take, you know, genius-level intelligence to do that. It is something that is intelligent, it's cheap enough to be deployed throughout all the countries that are engaged in international trade or have other sorts of business processes like that.

So that's what China is primarily focused on through Belt and Road, and they've been doing very good job at it. You might want to look at versus who are the biggest trading partners in Africa.

But also, India is very focused on that. They have a thing called the Citizen Stack, which is a generalization of their India Stack.

It's open, free, public software that provides identity, contracts, payments, medical information, free. Government insisted all the banks and hospitals begin to accept it. Started by a small NGO, but they made a modular version of it that can be used by other countries. Other countries can configure it.

And so they have, at the moment, 12 countries signed up, 26 altogether, that are thinking about it and doing parts. They've signed up one point four billion customers to this. Compare that to some of the other countries, and you probably haven't heard about it.

But now what they have is they have this data structure that covers the most fastest-growing part of the world, and it covers all the sort of basic parts of the economy, and you can bet people are building AI on top of that.

So maybe you want to pay attention. It's not gonna be the frontier stuff. It's gonna be the stuff that, you know, helps the schools run, make trade happen, keep people healthy, stuff like that. But what they'll end up doing is they'll end up owning, in the sense of being able to manage, see, and take advantage of, data flows from the entire Indo-Pacific.

China has Belt and Road, which is, you know, sort of the Asian to European sort of thing. Again, they don't have to see the data; they just have to see the metadata. They have to know where things are flowing, and then that tells you where you're supposed to invest.

I work as an advisor to the UAE's sovereign wealth funds, and yep, that's where they're headed.

You know, this is the game because as AI gets smarter and smarter and smarter, the scarce resource is data. Do you know what's actually happening that this thing can then reason about? And unless you have that data, you're just in the dark.

Preparing for AI-driven cyberattacks

Michael Krigsman: Sandy, we're just about to run out of time, but can you tell us the. Describe the implications of this for us?

Sandy Pentland: The thing that people ought to be thinking about is, how can they have visibility into larger parts of the economy through their, what they're doing? I wouldn't say necessarily owning data, but having metadata about where things are going and who's doing what, that type of stuff.

I think also, one needs to remember that the thing that will actually drive most of this are the bad guys. The cyberattacks, the people committing fraud, things like that, those are just beginning to ramp up in a way that's incredibly. It's really sort of hard to imagine.

But as we move towards this, this is something that has new attack surfaces. It's handling actual, you know, trade, money, things like that. It has real-life implications. It runs things like hospitals.

And it took us a long time to sort of figure out how to semi-bulletproof the Internet, and these new sort of agentic systems are gonna take time to bulletproof, and in the meantime, we're gonna see a lot of disasters. So be safe. Know where your data comes from. Monitor things, you know, to know what's happening, just the way you would monitor money. You know, if a million dollars disappeared from your account, you'd know about it in an instant. You need the same thing with these sort of agentic systems because they're actually the controller of things that, in the near future, that are going to be the things that are hit.

Sorry to be negative at the end, but, you know, it's getting hit that drives progress sometimes more than having opportunity.

Why open source AI matters

Michael Krigsman: Can you just share your view on the open source AI? And I'll just mention, folks listening, we have the CTO of Mozilla coming up on a show talking about this, so subscribe to our newsletter, cxotalk.com. Just very quickly, your view on open source AI and its importance. Really fast, please.

Sandy Pentland: Well, it's really a key thing because it makes people willing to adopt it. It's free, it's under your own control, you can see it, you can specialize it. And then the question to ask is, who controls the data?

Michael Krigsman: Always comes down to the data.

Sandy Pentland: That's where you get insights, and that's what matters.

Michael Krigsman: There's so much to talk about, and we barely scratched the surface of all the questions that came in, so I hope that you will come back, Sandy, and do this again another time with us.

Sandy Pentland: I'd love to, yeah. More to talk about, for sure. Things to move forward with.

Michael Krigsman: Sandy Pentland, thank you so much for being our guest. Everybody who's watching, you guys are amazing. Thank you for the people who asked questions. 

Check out our website, go to cxotalk.com, and subscribe so we can send you updates on new shows, and we'll see you again next time. Thank you so much, everybody, and I hope that you have a great day.