Digital Transformation and Data

Mike Flannagan, SVP of Analytics, SAP
Mike Flannagan
Senior Vice President of Analytics
Michael Krigsman, Founder, CXOTalk
Michael Krigsman
Industry Analyst
Dion Hinchcliffe, Chief Strategy Officer, 7Summits
Dion Hinchcliffe
Chief Strategy Officer

Digital transformation means business change based on innovative data sources and technology. This episodes examines how data, predictive analytics, and machine learning can power digital transformation and new business models. The point is accomplishing real business outcomes that make a difference. Our guest is Mike Flannagan, who runs Analytics for software giant SAP. 

As Senior Vice President of Analytics at SAP, Mike Flannagan is responsible for Product Strategy and Product Management with a focus on building products that deliver business insights to anyone, anywhere, on any device with a world class user experience. Mike also leads the Customer Success team, where SAP works closely with customers to ensure they derive maximum value from SAP’s products.

Mike is active in the Analytics startup community, serving on the Board of Directors for MammothDB, as an advisor to several startups, and as a angel investor. Prior to joining SAP, Mike was Corporate Vice President and General Manager of the Data & Analytics Software Group at Cisco Systems. Mike holds several patents, has co-authored several technical books, and is a frequent speaker at industry events.

Download Podcast

Digital Transformation and Data

Michael Krigsman: Welcome to Episode #233 of CxOTalk. I’m Michael Krigsman, and we are streaming live from SAP’s big user conference called Sapphire Now. And before we go into this discussion, I just want to say “Thank you” to Livestream because Livestream is our streaming platform; and man, those guys are really good. And if you ever need a streaming platform, go to Livestream. They’re really good. So, thank you, Livestream.

So, we’re here at Sapphire Now, and I have the privilege of speaking with Mike Flannagan, who is deeply involved with analytics, with data, and with something new that SAP announced called Leonardo. Mike, how are you, and thanks for being here!

Mike Flannagan: […] Thanks so much for having me!

Michael Krigsman: So, Mike, you're deeply involved with analytics and with data, and with Leonardo, so tell us about your role and what do you at SAP.

Mike Flanagan: So, officially I’m the Senior Vice President of products for analytics. And now that we have launched SAP Leonardo, I have also taken on the role of Head of Products for SAP Leonardo. And, we had a big announcement this morning by our CEO Bill McDermott.

Michael Krigsman: So, I really want to dive into the issues around data and analytics, but very briefly, tell us what is SAP Leonardo?

Mike Flanagan: So, it is officially a digital innovation system, but the idea behind SAP Leonardo is fairly simple. Everybody struggles with business problems, particularly now with the pace of change and the need for transformation of digital business. If you're in retail, problems that you have are not that dissimilar from problems that your peer companies have. And the solutions to those problems from a methodology standpoint and a technology standpoint also have a lot of commonalities. So why does every company have to feel like they’re reinventing the wheel? And Leonardo is intended to help accelerate digital transformation for companies by leveraging the SAP’s experience with other companies to help them solve the same problems using the same methodologies and approaches. Obviously, there’s some customization that’s involved in the company, but you start with a nucleus that is able to accelerate solving the business problem.

Michael Krigsman: So there’s this combination of technology and business process that kind of move together?

Mike Flanagan: Well obviously, nobody in the C-Suite bubble stands up in the morning and says, “I want to go buy some digital transformation.”

Michael Krigsman: Exactly.

Mike Flanagan: They’re thinking about, “How do I improve revenue growth; how do I improve bottom-line profitability; how do I improve customer experience,” so, when you look at those things, you can break them down into a set of fairly digestible business problems that need to be attacked. So, if you can very quickly move from the first problem to the first solution, and then you attack the second problem and second solution, you can move your company along a maturity curve until you become a fully digital business from a post-transformation … all the way across. But if you start saying, "I need to go change everything tomorrow,” that seems like almost an impossible task and a bottomless pit of money. So, it’s important that customers be able to take a little step and see the results and get the return on that investment, so that if they feel confident taking the next step and continuing […].

Michael Krigsman: Mike, I think we should begin with a discussion of data. And we heard in one of the keynotes this morning the phrase “Data is gold,” and we hear similar kind of sentiments all around the industry. And so, with digital transformation, let’s begin with this notion of, what is the relationship between the data and the ultimate digital transformation that takes place?

Mike Flanagan: There are all kinds of great analogies in the market; that data’s the new gold; data’s the new oil; whatever you prefer, data’s a very valuable asset. And, if you think about your data, that you think about your human capital, you think about the way you think about your real estate investments, you start managing it as an asset that has a lot of business value. Then, you start realizing the transformative power of doing things with that data that you couldn’t do before. And then, of course, everybody’s talking about IoT or the industrial internet, and that really is about opening up a whole world of data that you didn’t have before with sensors and wearables, and those sorts of thigs; and the transformative power of that data become exponentially greater because so much more data from which you can draw insight [becomes available].

Michael Krigsman: So, talking about collecting data from many new sources that even a few years ago were really hard to imaging; can you give us examples of some of the new data sources that are available to us?

Mike Flannagan: Sure, absolutely! I mean, and of course, I think it’s worth noting that is it’s not just new data sources. If you’ve been running your business for a hundred years…

Michael Krigsman: Good point.

Mike Flannagan: … you have a lot of really valuable enterprise data. I think the power of things like industrial IoT is adding to that some data from new sources, and so you think about data from sensors. We’ve got examples of train companies who outfit the brake systems of their trains with sensors so that they can measure break wear. In fact, my car has sensors on the brakes. It doesn’t send me an email, but it gives you a little display on your dashboard. Everybody can sort of relate to that little e example. Now imagine you’re managing like Trenitalia does to thirty thousand locomotives, and you’re trying to minimize the amount of time that you’re out of service for maintenance, both to decrease your downtime costs, but to improve your customer experience by having trains running on the tracks. The ability for them to just add some sensors to monitor a little bit of data about maintenance really gave them the ability to transform the business process around predictive maintenance.

Sensors are really one example. Wearables are a new data source. And you know, I think if you consider those types of data sources, you could imagine what the future might hold of all kinds of different wearables, embedded sensors… Video is becoming a really powerful new data source; deep learning starts becoming a more mature technology. So, it's an incredibly interesting time for data people.

Michael Krigsman: And these data sources have the power to shape and mold processes. I mean, just, for example, last week on this show, I spoke with the Chief Marketing Officer, the CMO, of Aetna, a huge insurance company. And, he was talking about how they can take wearable data, just as you were describing, and feed that back to patients in order to increase patient wellness. So, can you elaborate, then, on the linkage between having these data sources, and changing processes and even changing business models?

Mike Flannagan: Well, you know, it’s interesting. In the enterprise world, we talk a lot about the business outcomes. In the Aetna example, what you’re talking about is patient outcomes; human outcomes.

Michael Krigsman: Exactly.

Mike Flannagan: If I can improve as a doctor; if I can improve the outcome of interacting with a  patient to extend their life or extend the quality of their life, I mean, that’s really exciting. You know, it’s interesting to have business outcomes with more profit and more revenue. But you know, when we start injecting some of the human discussion about the power and the potential of this data, we start realizing we can really change the world. We can change society, we can change the quality of people’s lives, and all of that is starting to be made possible by these new sources of data. They give you new insight into people.

Michael Krigsman: And, before we go on to the next phase in, shall we say, the life cycle; so, we collected this data, how do we then start to use it? Can you give us an example of existing corporate data that we can find new uses for today?

Mike Flannagan: Sure. So I think if you look at loyalty card data in retail, there’s a lot of information there about purchasing history, purchasing preferences, which stores do you tend to frequent, those sorts of things. There’s a lot of rich data there. Historically, it’s been used for things like sending you coupons. But, there’s a lot more that can be done with that, particularly if it’s augmented with some new data from new sources. And, so I think there’s a lot of value in the dataset that already exists there, and as you start thinking about how to augment that with new data, the power of both really becomes much greater than the sum of the parts.

Michael Krigsman: Okay. So, we've not got our existing corporate data, that we can make use of in new and better ways because we can now aggregate it; and we have things we can do with it that we couldn't do before. And so, what can we do now with that data that historically, we could not do? Because it seems like that's the thing that unlocks the power of that existing data.

Mike Flannagan: Yeah. I think there are advances in analytical techniques, things like machine learning; you know, lots of industry buzzwords; excitement around machine learning, these days. The power of machine learning is that it really gives you the ability to go back into data that may be two, three, ten, twenty years old, and take all of that history that you have about customers and store operations, and a variety of different things, and turn that into training data, right? To teach the machines what a customer looks like. What does a good customer look like? What does a bad customer look like? What does fraud look like? Those sorts of things require processing the quality of data from which you learn that a human would be incapable of dealing with, right? So it has to be about using the power of machines.

And then, obviously, there are examples in manufacturing here you take that learning and turn it into artificial intelligence; things like robots. But, there are also examples in customer service, for example, with chatbots, where now I want to ask a few questions to my bank, and instead of having to have a teller answer the questions, I can just go online and chat and get automated responses that are amazingly accurate for the questions I’m asking.

Michael Krigsman: Okay. So, there are all these things that we can do with the data, but how do we prepare that data? How do we prepare …. So, we’re collecting that data, we’re doing something with it, and then it can be used in the applications you were describing?

Mike Flannagan: I think there are a couple of different ways to answer that questions, but one that I think is particularly of interest for a lot of our customer is, when you talk about leveraging a huge population of data from which to learn, there are concerns about privacy, and there are concerns about data protection. And so, one of the things that is, I think, important in every conversation about large datasets is how do you anonymize that data? How do you protect the personal information that is contained in that data, how do you make sure that your policies are such that you only is that data for its intended purpose?

That having been said, part of preparing the data is sometimes normalizing the data so that things look common across a large dataset. Also, anonymizing that data. And so, when you take an aggregate, you can use that data for, let’s say, benchmarking. The power of the average price that I should expect to pay for a bar of soap: I can collect data from hundreds of different sources. Some of them may express it in dollars, some in Euros, some in different currency…. I have to normalize that data so it’s all a common currency, and then I aggregate that data. It doesn’t really matter whether the data came from Retailer A, or Retailer B, or Retailer C, so I can anonymize that part of the dataset. And, what I’m really looking for Is, “What is the average price in each city, each country, per bar of soap?” And then the value of that is a good benchmark for retailers that market to use, but you’re not using any data that’s specific to a retailer in a way that’s identifiable for the […].

Michael Krigsman: And where does that normalizing and anonymizing take place? Does it take place inside the customer walls? Does it take place on the platform side, like on the SAP side? How does that … What’s the mechanism for that? And then for the benchmarking as well that you were just describing?

Mike Flannagan: It really depends. So, one example of aggregated, anonymized data that is being used for benchmarking is in SAP Fieldglass. It’s an application that we make available to customers to deal with the contingent workforce. And, if you look at Fieldglass, we see hundreds of thousands of transactions every year for people who are looking for jobs, and people who are hiring for temporary workers. Inside of that application, we can now aggregate and anonymize the data so if I say, “What should I expect to pay for a salesperson in these three cities?”, we have that data. And we can make that available to customers as live insights in real time. When they’re thinking about what is the right labor rate to offer for this role, they can see what is a common labor rate that will get them a well-qualified, talented individual to fill the role in a reasonable timeframe.

So, that kind of data would be aggregated and anonymized and injected back into the application by SAP at our level. But, we have an example here, actually, at the Sapphire conference. Our SAP data network folks are talking about something they’re doing with a very large elevator company, and that data is that customer’s data. So in their case, aggregated, anonymized, and used on their premises in their systems.

Michael Krigsman: And I have to assume that this benchmarking capability, either real-time, in order to look up … So I want to hire somebody and what are the labor rates, for example, for this type of position? Or, historical, “I’m thinking of doing something and I want to know how did we compare in the last six months to our competitors?” So, I have to assume this is extremely valuable and this is what customers want.

Mike Flannagan: It seems to be for sure something that we’re getting more and more requests to make available. What I think is interesting is not so much … I mean, [it is] certainly interesting; the raw benchmarking … What I think is more interesting and what we hear more of a customer saying, “If you could do more like this, it would be great,” is … So I know that I have a certain budget, and I know I have a certain set of needs, and that set of needs materializes for me five skills that I need from an individual. But when I go look at the benchmarking data, the five skills that I need in the market that I need them in, twice the budget that I have available. Well, that’s not very useful. All you’ve done is tell me that I can’t hire what I need, and so now what?

Michael Krigsman: I can’t afford the thing I want to buy.

Mike Flannagan: That’s right! And so, the more useful thing in that scenario, I think, is to be able to say, “What if I could compromise and only get three of the skills that I really need?” Maybe I can teach those other two once the person’s on board. And if that fits my budget, then that becomes sort of a win-win, right? I get somebody who doesn't do quite what I need; the data they join, but I get them for the labor rate that I can afford, and I get the opportunity to teach them the things that they need to come up to … That kind of benchmarking also gives you the ability to say, "Well what's my next best option?"

Michael Krigsman: And where would this type of calculator be built? Is this built into the HR application? Are they doing this new, dare I say, in a spreadsheet?

Mike Flannagan: [Laughter] So, in a spreadsheet is typically how this kind of stuff is managed. We go out, we take the big salary survey, and we pre-populate a central repository; generally a spreadsheet, of benchmark labor rates. That is what we are helping customers move away from. If you want to run a live business, that’s not very real. And so, the Fieldglass application …

Michael Krigsman: And prone to errors .. I didn’t mean to interrupt, but there’s a lot of problems with spreadsheets. But anyways, I didn’t mean to interrupt. I’m sorry.

Mike Flannagan: Oh no! Absolutely right. But, I think the key here is that we’re injecting that information back into the Fieldglass application so that it’s right there in the workflow when a customer is trying to populate a new template for a new job, for typically a job posting. Being able to do that means it’s not […], it doesn’t sit off to the side of your core business application. It is part of your core business application.

Michael Krigsman: And therefore, it’s a core part of your … So this type of analysis, then, becomes a core part of that business process as well.

Mike Flannagan: And that is the key to moving analytics from what it has been up to this point, which is something that is useful for ten or fifteen percent of your total employee population to something that is used by one hundred percent. I have to put that sort of intelligence into the business process. It can’t be a separate thing.

Most common example of this with the consumer is probably Netflix with their recommendation engine, or Amazon, real products that go with this product; those are recommendation engines powered by machine learning. And they’re extremely powerful not because of the intelligence behind them, but certainly for that, but because they’re in your process. While you’re looking for a movie, you’re seeing the recommendations about movies you might like based on your previous choices. While you’re buying toilet paper, you see bar soap that most people might buy at the same time. That’s useful because it’s in your process. And so, it’s very easy for you to use it, and very easy for you to see the value.

Michael Krigsman: The key, then, is by building these, let’s say features that are backed by this data, from a user interface standpoint, the teachers probably look pretty simple on the surface. You know, checkbox, this, this, that to make a few selections. But, the key, there, is by building it into the application, it means it’s now central to the activities you’re performing also known as the process.

Mike Flannagan: That’s exactly right. Nobody wants to have to teach all of their employees how to be data scientists. You don’t want them interacting with complex statistical algorithms. And what you want is you want them to do the work that their experts are doing. You want our HR people focused on HR. You want your finance people focused on finance. But if you can make all of those applications that they’re using in their day-to-day lives more intelligent, more capable of helping them run the business correctly, then it’s great and it’s embedded into the work that they’re already doing so there’s no learning curve for them.

Michael Krigsman: Okay. So now, we’re going through this story, and I want to remind everybody that you’re watching Episode #233 of CxOTalk. And, we’re speaking with Mike Flannagan, who is responsible for analytics inside the important Leonardo product at SAP. Did I say that right?

Mike Flannagan: All right.

Michael Krigsman: [Laughter] So now, we’ve got the data. It’s been anonymized, it’s been aggregated, so now we can benchmark against it. We’ve got the user interface to that data, and a nice friendly way inside the software application; so it’s a core part of the process. And that application is being fed from that data store. But, you mentioned this kind of magic term, “machine learning.” So, how does machine learning and other, let’s say, I hate the term “artificial intelligence.” It’s become like “digital transformation,” it’s sort of a catch-all phrase. So, how does machine intelligence, machine learning change the way you now can use that data? Where’s the magic?

Mike Flannagan: Machine learning is a buzzword, and everybody’s talking about machine learning. Machine learning is a fairly horizontal capability. What makes it interesting is data that will train a model that can then be used to make better decisions.

Michael Krigsman: Just to elaborate: When you say “train a model,” for businesspeople out there, what does that mean?

Mike Flannagan: So, think of it like hiring a new college graduate and needing to teach them how do to a job function in your business? The more they do that job function, the more they learn how to do it. The more they do it right, and somebody says “good job,” that positive reinforcement makes them do it that way. The more they do something wrong, and somebody corrects them and shows them the right way, the less likely they are to make that mistake in the future. Fundamentally, the same concepts as machine learning. If I want to take the very large dataset of historical data to predict what might happen in the future, what I’m looking for is what are the things that have happened in the past and what are the things that happen in the future, and where do I see very high levels of correlation between the past and the future?

So, for example, Salespipe One data may be highly correlated to next quarter’s revenue. The higher my pipeline is, the higher my close rate is on that pipeline, the more revenue I’ll have next quarter. The machine starts learning exactly how correlated that his; exactly how good of a predictor of next quarter’s revenue is this quarter’s sales. And, the more it learns, the more data you feed it, the smarter it gets. The more accurately it can predict things; obviously, it's not a crystal ball, there's always an opportunity for unforeseen things to change the future.

But machine learning; one of the powerful capabilities is about learning from the past and being able to automatically apply that learning to estimate the most likely thing to happen in the future.

Michael Krigsman: So, we’ll definitely dive back into that, but I also want to welcome Dion Hinchcliffe. Dion is an industry analyst like myself, and truly one of the most influential analysts among CIOs; and also hosts his own CIO-focused show on CxOTalk.

Dion Hinchcliffe: CxOTalk, absolutely. Well, thank you. Thanks, Michael. Hi, everyone. Hi, Michael.

Michael Krigsman: So we've been talking about … You were just describing machine learning, and maybe since Dion is here, where do IT and the business … Where do IT and the business fit together in this whole landscape?

Dion Hinchcliffe: […] one clarification. So, I’m working with CIOs and the C-Suite in general. There’s a lot of excitement around what machine learning and artificial intelligence can do. The question is, that I’m getting more and more now, is, “now I’m going to hand over my data to these learning algorithms. What stops you from learning so much about my business?, and then that knowledge gets transferred inevitably to the products of my competitors and other businesses. So, how do I know that all that stuff it learns stays with me,” right?

So data is the new gold, the new oil, as you guys were talking about. How do organizations retain the control?

Mike Flannagan: Well, I think the Number One thing in my mind, as you start asking that question was, it starts with something that we talked about earlier; to recognizing the value of the data that you own; recognizing that your data is an asset to be protected. You don’t take the buildings that your company owns and leave all the doors unlocked when everybody goes home. Same idea with your data. You have to realize what data is valuable, what data is important, what data is proprietary. And, take the appropriate steps to protecting that data. And, sometimes, that means that you need to think very carefully about the aggregation, anonymization process, to make sure that it can’t be reverse-engineered; to make sure that somebody can’t de-anonymize data, for instance.

Dion Hinchcliffe: It's surprisingly easy to do since everything gives off data now, right? So, there's a lot to correlate with. Is all this data insecure, or is it underappreciated in terms of its real value?

Mike Flannagan: I think it's underappreciated. I think most of the companies that I've talked to recognize that there is value in their data, but if you ask them to put it on a balance sheet, to put it on a bottom line, they couldn't tell you exactly how to value their data. And, that's a problem; because I can tell you exactly how to value my real estate assets. I can tell you exactly the value of every employee in my enterprise, but I can't tell you how much, what is being called now one of the most valuable assets of every enterprise is actually […].

Michael Krigsman: So Mike, given this, what are some of the metrics that an organization that is undertaking a program of digital transformation, at least when it comes to data, what are some of the metrics or the KPIs that they can use to evaluate their progress? How are we doing?

Mike Flannagan: This conversation sounds very, right now, at the moment, and new, but a lot of the answer to that question, I think, has been the same answer for thirty or forty years, which is a lot of companies have a Garbage-in, Garbage-out problem. If your data's wrong, it's not valuable at all. And if you use incorrect data as training data for machine learning algorithms, it's about to predict the future? Your predictions are all going to be wrong. So, a big KPI is data quality. How good is your data? How accurately is it inputted in your systems? How well do you take data that's incorrect out of the system and out on a process?

So, I think that's a key starting point. Because, if your data's not right, all these advanced technologies; all of these new techniques from learning from data will not benefit you in a way […].

Michael Krigsman: And so, this is a, shall we say, part of the … Is this – the correct terminology – part of the software implementation process?

Mike Flannagan: Well, it’s part of a couple of things. Obviously, data quality, there is software that helps with the process of data quality, but the other thing is business process; making sure that you have a good process for data being entered correctly, validated …

Michael Krigsman: On an ongoing basis.

Mike Flannagan: On an ongoing basis.

Dion Hinchcliffe: The challenge, though, that we’ve heard from here is that speed is paramount these days. I surveyed 54 CIOs, top CIOs around the world, many companies, about how fast you have to move. And they all reported they’re under very strong pressure to move much more quickly. How can they take these quality measures when everyone’s been asked to execute and deliver as fast as possible?

Mike Flannagan: Yeah, I mean, this is the problem. Sometimes you have to slow down to go fast. If you have a data quality problem, and you don’t slow down to fix that, all of these technologies that are going to help you go much faster are not going to help you go faster, unless you’re going faster in the wrong direction.

Dion Hinchcliffe: That’s foundational.

Mike Flannagan: It’s foundational. So, you really have to consider transforming the way you think about data from its origination all the way through to its ultimate delivery of value. And, if the origination of the data is flawed, then the whole rest of that supply chain, if you will, becomes flawed. Machine learning and artificial intelligence, it all sounds very new, but most of the advice that I’ve just talked about goes back for a long, long time. The fundamental processes haven’t changed. What is exciting now is that there are technologies that if you get those fundamental processes right, can help you go at an incredibly accelerated pace.

Michael Krigsman: Now, we hear about data scientists being so in demand, and you need to be prepared to hire data scientists. I think most businesspeople, they hardly know what a data scientist even does.

Mike Flannagan: Most data scientists hardly know what a data scientist does, but if it’s on your resume, your salary goes up! So, everybody’s a data scientist!

Michael Krigsman: So, how should businesses relate to, let’s say, let’s call it; and to all my data scientist friends out there, I apologize for this; but how should businesses relate to the data scientist problem?

Mike Flannagan: So, I think, you know, if you think fundamentally about what your core business is, and you make some decisions from there about how far away from your core business do you want to learn something, versus where do you want to procure something? In my core business, it’s data […]. Having an army of data scientists in-house makes absolute sense. If I’m a retailer, I think there’s a reasonable question about how much of that do I want to do in-house?

Michael Krigsman: But you need data science, so all of this is about data science. So, what should we do; we businesspeople?

Mike Flannagan: Well, the question is, do I want to hire them and own it in-house? Or do I want to work for the firm who does that as their core competency?

Dion Hinchcliffe: Data science as a service, right? […]

Mike Flannagan: And I think the question is a core-versus-contexts question, just like everything else. Do I want to own in-house janitorial services? Or, do I want to hire a janitorial services firm to come do that? I think you can apply that to lots of different areas of your business about what is the core, and what things do you need? But they’re not the business that you need to be […].

Dion Hinchcliffe: You know, we heard some great things about SAP Leonardo today, and you guys probably already talked about some of this, but it seems like the packaging around that is really designed to say, "Alright, so this is part of the N-to-N value chain that most organizations have to realize. Data is at the center, and the value it attracts is going to come from an increasing layer of technology; so blockchain, to machine learning, to data intelligence, and so on. If someone wanted to understand what SAP Leonardo does, how do you describe that in one sentence?

Mike Flannagan: I’d say, first of all, there are a whole lot of technologies that are in Leonardo that a hundred other companies will sell you as well. So, the technology element is interesting and it’s differentiated. But, this isn’t a technology solution. It’s a business problem solution. So, if you look at Leonardo, the idea is once I solve a business problem, there are common elements of my problem that apply to lots of other companies and lots of other areas. So, we talked, I think, in one of the keynotes about an example that I mentioned here, which is trains being outfitted with sensors for the purpose of predictive maintenance. Eighty percent of what was done there would be interesting to a transportation company who owned trucks, or a mining company that owned heavy machines, to be able to do the same sort of predictive maintenance to minimize downtime and improve their operating costs.

And so, taking those common elements, and packaging them as industry-specific accelerators, so that you as a CEO could identify a business problem and figure out very quickly how to get them identifying that problem and implementing the solutions. It’s about accelerating that process in between.

Dion Hinchcliffe: So, it’s a combination of technologies from the SAP portfolio that is aimed at a specific industry or vertical issue.

Mike Flannagan: And it’s combined with a few services because what I've done is taken a hundred percent solution for this customer, and generalized seventy or eighty percent of it.

Dion Hinchcliffe: And they get […], right?

Mike Flannagan: We then need a few services to tailor it back to the next customer and the one after that. And so, being able to do that lets us move from problem identification to implemented solution about fifty percent faster.

Michael Krigsman: So you’ve kind of systematized or productized some of the common technology elements, and some of the common process and deployment aspects of it.

Mike Flannagan: Exactly right. So, there are business problems for seventy or eighty percent of common. There are, if you look at those problems, technology solutions that are always going to be seventy or eighty percent common; and it's taking those common elements of technology, putting them together with the common approaches, the methodologies that are used to implement them and [know] exactly what do I do with that sensor data to get it to reduce the operating expense, and packaging that as an accelerated order.

Michael Krigsman: In the old days, we used to call that "packaged solutions." You must have industry … How do you break it out? I mean, for industry; for banking; it's going to be attributes of this ... The underlying technology may be the same, but certainly, the process aspects are going to be very different than, say, retail or …

Mike Flannagan: And these accelerators are packaged by industry. So, the recognition is certainly that business problems are fairly specific to industries. There are some that can be generalized horizontally as more technology problems or process problems, but the business problem if you really want to make it that repeatable, it has to be someone specific to the user.

Michael Krigsman: Probably the farther down in the technology stack you go, the more commonality there is. And as you get closer up to the process and to the activities that people perform, and how the data is ultimately used, I would suppose there it becomes much more differentiated industry-by-industry.

Mike Flannagan: That’s right. And I think as a consumer, you could argue that if you need to go buy a toothbrush, the difference between Walgreens, CVS, Target, is not distinguishable for you. But if you get into actually how they run their business every day, obviously each retailer has things that are specific to them that are different from the retailers. And those have to be considered in […].

Dion Hinchcliffe: So, a lot of talk these days about blockchain. We’re seeing more and more types of data used just to […] transact […] the blockchain. And now we’re hearing things like “identity,” or things like SKUs or unique customer IDs; all sorts of things are being thrown in there. What’s the blockchain story, in terms of the data and the analytics? Now we have to talk about blockchain analytics, I guess? The whole new generation of data? What’s your view on all of that?

Mike Flannagan: Gardner has; I'm a big fan of research, and Gardner has something that I think's appropriate and it's called the Hype Cycle. And, it's a curve that all technologies; new technologies start to ride and at some point, the expectations are, this technology can do everything, it can solve every problem, it can slice it can dice. And I think that may be a little bit of where we are with [Cooptator] right now. There’s a lot of potential.

What I think is really interesting is, which ones are going to land on real business value? Which ones are really going to change a business model or business process? And I think some of that’s still going to be worked out. But I love the fact that there’s so much potential, and there’s so much conversation and people are trying things. The key, I think, is fail fast with any technology that we’re experimenting with.

Michael Krigsman: Mike, we just have a few minutes left. And, what advice do you have? You’re working with a lot of customers. You see a lot of different businesses. And what advice do you have for a businessperson who is looking at all of this and hearing about machine learning, and all of this stuff: they’re trying to figure out what to do. So, what should they do?

Mike Flannagan: My Number One piece of advice, and this is a little bit shamelessly associated to… The approach we're taken with SAP Leonardo is [to] take one small step at a time, get business value from that step, or fail fast and move on. Don't try to solve every business model, every business process, every business problem all at once with some giant tens of millions of dollars transformation project.

Michael Krigsman: Start small.

Mike Flannagan: Start small, find some quick wins, deliver some business value, and then do it again. On the things that don’t work, fail fast and fail cheap, and move on. And I think that’s probably the most powerful advice that I can offer, and that’s the design principle behind Leonardo.

Michael Krigsman: And I know, Dion, we speak with lots of CIOs. It’s certainly great advice for any CIO.

Dion Hinchcliffe: Yeah. Totally agree. They get in the list as quickly enough and building the skills that are doing that, it allows you to tackle and move […].

Michael Krigsman: Alright. Well, this has been a fascinating conversation. We have been speaking with Mike Flannagan, Senior Vice President at SAP, and I’m so thrilled that Dion Hinchcliffe, industry analyst, focused on CIOs, has come join us. And of course, Dion has his own show on CxOTalk focused on CIOs. Thank you, everybody, for participating today. Mike Flannagan, thanks so much! It’s been great!

Mike Flannagan: Thanks for having me.

Michael Krigsman: And Dion…

Dion Hinchcliffe: Thank you.

Michael Krigsman: Thank you, everybody, have a great day.

Influencer Marketing with Video

David Hoffman, Documentary Filmmaker
David Hoffman
Documentary Filmmaker
Mark Fidelman, CEO, Fanatics Media
Mark Fidelman
Chief Executive Officer
Fanatics Media
Michael Krigsman, Founder, CXOTalk
Michael Krigsman
Industry Analyst

Marketers today must understand how to create and promote video. Our show this week brings together a documentary filmmaker with a top influencer marketing expert to explore the new world of video content.

Mark Fidelman is the CEO for Fanatics Media, a digital Marketing Agency with a focus on B2B and B2C Influencer marketing. Mark has been named a 2016 Top 20 influencer of CMOs by Forbes Magazine, a Top 25 Social Media Keynote Speaker by Inc Magazine, and a Huffington Post Top 50 Most Social CEO. Mark writes the Socialized and Mobilized Columnist on Forbes and is the author of the book SOCIALIZED!

David Hoffman practices a simple but profound idea. Hoffman says” it's not what you say that your audience hears. Your audience hears a combination of what you say and what they already think/feel. Therefore to communicate effectively, you must deeply understand the target audiences who you are trying to reach. He is an 8-­‐time Emmy Award winner & Tribeca Disruptive Innovation Foundation Fellow who has consulted to executives on audience engagement at more than a dozen Fortune 100 companies including AT&T, GE, Google, Verizon Wireless, United Technologies, Merck, Amazon & Sony. His start-­‐up clients include Mesosphere, Yerdle, Cherokee Uniforms, Liquid Robotics & TEDMED. He has worked directly with leading entrepreneurs including Jay Walker, Jeff Bezos & Megan Smith. Check out his YouTube channel.

Download Podcast

Influencer Marketing with Video

Michael Krigsman: Welcome to Episode #230 of CxOTalk. I’m Michael Krigsman, industry analyst and your host for this show. Today, we are going to be speaking about video: video in marketing, and video in business. We have two people [here] who are amazing.

Before I introduce them, I want to give a major shout-out to Livestream. Livestream provides our video streaming platform. I have to say, I used to use Google hangouts, like a year and a half ago, and it didn’t work. And there were bugs, and there was no support, and I hated Google Hangouts, and I went to Livestream and things just work! And so, I want to say “thank you” to Livestream. And not only that, if you go to … no. Wait, wait, wait … If you go to Livestream.com/CxOTalk, they will give you a discount.

And without further ado, our two guests. I’m going to introduce David Hoffman first, who is a … not sure what the title would be. It’s a strategic communications consultant, and he’s a guy who happens to have made 175 documentaries for television? He’s a guy who knows a lot about video and business. Hey David, how are you?

David Hoffman: Hey, Michael. Hey, Mark. Good to talk to you. Thank you.

Michael Krigsman: So, David, tell us briefly what you do. Who are you and what do you do?

David Hoffman: I’ve been doing the same thing my whole career, which is communication. But, what’s communication? My view is, it’s not what you say that the audience hears. The audience hears what you say and what they already think. So my whole world is about engaging audiences. Engagement means I’ve got to know who the audience is, and I’ve got to know how they’re going to react to any message in any media. These days, primarily video. That’s my world. But, whether it be video, audio, a book, a podcast, or a meeting, if you don’t know the audience, you’re not going to have a successful communication. That’s my view.

Michael Krigsman: All right. Well, clearly, we’re going to be talking a lot about that. And, our second guest is Mark Fidelman, who runs a marketing agency … I’m not sure if that’s the right term, “marketing agency” or “digital” …

Mark Fidelman: Digital marketing agency.

Michael Krigsman: And so, Mark, hi! Tell us about yourself!

Mark Fidelman: I’m Mark Fidelman. I do run Fanatics Media. Two focuses for us, and very happy to be here, Michael and David, because I love talking about these subjects; the first one being influencer marketing and the second one being video marketing. So, I look forward to our discussion. I think the audience is going to learn a lot.

Michael Krigsman: Well, I look forward to our discussion, because I produce one, a whole lot of videos, and I want to learn from you guys so that I can do a better job. And I think we need to begin with the idea and discussion … Why video? Why should business people care about video? Why do marketing people care about video? Thoughts? Anybody?

David Hoffman: Sure. Video is about emotion. What do you do when you read the Wall Street Journal? You skim it. What do you do when you read a website page. You skim it. We all skim. Nobody reads anymore. Maybe you read at night in bed with a book. So, video is about emotion. Interesting statistic: the eyeballs last longer. Any other form of media that’s on the web doesn’t hold you like video. So, if the video has a good story, and speaks to a target audience, or an influencer as Mark said; I’m interested in hearing what he thinks about that; then, I say video is the most powerful way to affect an audience to make a change.

Mark Fidelman: And, I like to give one example, Michael, if you don’t mind. And most people here probably know Gary Vaynerchuk, and if you don’t know, Google him. This is a kid, and I still call him a “kid” because he’s much younger than me, who started doing videos on YouTube eight or nine years ago for his wine business with a cardboard table and some wine on top of it. It was a $3 million business ten years ago. Fast forward five years after that, the wine business is a  $60 million per year business in five years, all because he did a video a day comparing wines. If that doesn’t tell you the power of video in business, I don’t know what else can. And that was five years ago.

David Hoffman: And that’s been millions, and millions of viewers, and thousands of guys making millions of dollars, as Mark knows. A key thing is this: when the person watches the video of a searcher that’s coming from YouTube … So this guy found searchers that search wine, spot him, and say, “Whoa! I want to see that guy!” Or, is it your friends, like on Facebook. Two very different states of mind. In this case, I’m searching for “dog walker,” not really the person whose video I’m watching. In Facebook, they already know me. I want a dog walker. They can hit me with a dog walker video, which is why Facebook’s sponsored posts are just off-the-charts successful when you’re targeting an audience. Amazing!

Mark Fidelman: Yeah. And Michael, you can’t do this with text or even audio. Gary could not have built that business with any other medium but video. You had to see it, you had to see his reaction, you had to see him. That’s what made it work, and he did it in a time where video was barely even scratching the surface of what content’s out there. Now everybody’s predicting it will be 75% of web traffic by the year 2020.

David Hoffman: Let me just say one other thing about what Mark just said. Interesting. Which is, people say, “Video should be 30 seconds. Video should be one minute. No more than 90 seconds.” Let me tell you, I made a video on cancer for a guy that wrote a book on cancer. Nine minutes each clip; everybody watches it. It has like a 90% viewership. It’s not about time. It’s about the audience! The audience cares, and this kid’s telling me about a wine that I didn’t know about; and here in Northern California, people really care about wine. I’m going to watch the whole thing and I’m going to click on his other videos. It’s enormously powerful when you’re speaking to the audience and trying to effect something, and you’re a good character and you’ve got a good story. I don’t know if Mark agrees with that.

Mark Fidelman: I mean, absolutely. I kind of look at it a little bit slightly differently, because not all of us are @garyvee. The longer the video, the higher the production quality or the higher the individual charisma has to be. But, that’s not always true, because if you look at the quality of video with Gary’s videos, it wasn’t there. It was just him, and his personality, and his charisma that drove it. He was kind of the Howard Stern of the wine business. You just wanted to watch him, and you just wanted to hear what he was going to say next. But, he really knew his stuff, too! So there was an educational component in it.

David Hoffman: I’m going to have to disagree with Mark about that. The example I’m going to use goes way back to my youth. I don’t know how many of you watching this know Carvell…

Mark Fidelman: Yeah.

David Hoffman: New York City ice cream. And Tom Carvell produced these commercials, and the commercials said, “Hi! I’m Tom Carvell! I’m here in Riverhead! Hi, Joe! How you doin' Joe? You make ice cream. Is it fresh?" It was the most primitive and enormously powerful. The same with Crazy Eddie, in New York. I learned from that, and I have found experimenting that my most popular videos for clients and for me … Sorry for the phone, they're all after me, only for me [Laughter] … The most popular videos are not the ones that look the best, in fact, the production value isn’t it. Production value makes me think it’s made by a company, and its public relations, rather than what I call, “User-generated style.” And I like “user-generated style.” Michael’s talking to me right now, and I’m talking to you right in this little box just being me, and Mark’s being him. That’s the most powerful communication in my view.

Michael Krigsman: I have a question for either one of you. This issue of charisma: How do we create videos that cut through the noise? What can we do? And by the way, we're starting to get some questions from Twitter, but as somebody who creates video, I need to know the answer to this question. How do I cut through the noise? Especially now. There are so many people producing video, and it's so easy to make video – most of it bad. How do you cut through the noise? Tell us, please.

David Hoffman: Okay. Old man goes before the younger guy. But Mark, you can interrupt. No problem there! Here's what to think about: Don't think about noise. Think about the audience. Mark probably agrees with me about that. When I get down to … I have a video called … It's about the United States equestrian team made by the team, so I made it. And in that, one of my keywords in YouTube was "falling off horses." Ninety percent of the people who clicked that clicked my video in Google AdWords and watched the whole thing waiting for falling off horses. So, it's about: Who is your audience? There is no noise if I have a certain cancer, and 50 thousand people have that cancer, and you've got a drug that's going to help me. You may not have the drug I use, but I'm going to go to you. Don't think about noise. Noise is if I'm selling Pepsi. If you're not selling Pepsi, but you're selling a brand, let's say BMW, it has a target and they have targets within the targets. Is it a woman? Very different. They're still making their ads on television for men. I can't believe that! Is it a Millennial or a Baby Boomer like me? I think, Mark, are you a Baby Boomer or a Millennial? I think …

Mark Fidelman: No, I’m not either. I guess what they call “Generation-X” which they never came up with a name for and we’re still insulted by. But, just to play off what David said, you know, we work with a lot of different influencers especially on YouTube, and some of them put on makeup all day long, and they get 2-3 million views every time they put on makeup. Who would have guessed? Another brand, or another influencer called “healthy junk food.” I mean, that’s an oxymoron, but all they do is go out and create big versions of different fast food recipes and they knock out of the park. Over 400 thousand views every time they put out a video.

So, Michael, I don't know … There's not a single formula for success in breaking through the noise. There is a lot of noise, but I agree with David. You've got to find your audience and if you experiment around a lot, pretty soon, you’ll be able to develop that audience and speak to that audience. But I still think you’ve got to work on charisma. Charisma is just a function of, “Be who you are.” There’s a guy, he does finance and I’ll look him up in between questions here, that’s he’s kind of dry. He speaks at the camera, he talks about finance, but this guy’s phenomenal. He gets 3-400 thousand views. And he’s talking about financial subjects like stocks and bonds and all these obscure subjects that you and I know nothing about, but he makes it plainly visible; and he demonstrates value by just talking in plain English about what these things are. So … I haven’t found a formula yet, but I could do a brief “find your audience” and be interesting to that audience.

David Hoffman: Let me think [about] something about what Mark just said. Really powerful, yourself. Make mistakes. When Reagan made a mistake in his press conference by saying, “Look, I don’t know the answer to that” to this member of the press, his ratings went higher than they had ever been. I think 76% of the country supported him. Be real. The issue on the web with video is authenticity. If that authenticity comes from a very well-produced, beautiful musical comedy, okay. If comes from you talking at the end of the day, you’re tired, you’ve got a little bit of circles under the eyes, you’re real. You work with the audience.

Mark also has a part of his business I really am fascinated by which is the influencers because what influencers do, and there are a lot of statistics on this; if someone’s going to watch thirty seconds, but it’s coming from an influencer, they’re going to watch four minutes! Now the influencer has enormous power.

Mark Fidelman: They do. And, the person I was thinking of, David, was Scott Galloway. So anyone watching just Google, or go to YouTube and type in "Scott Galloway." He doesn't look interesting at all, but I assure you, he'll captivate you in the three – ten-minute videos that he does. [There is] a lot of good B-roll stuff, but phenomenal use of video for what he is, which is kind of a stock analyst that makes things interesting because of the way he presents it. So, you know I …

David Hoffman: Pardon me, Mark. I’ve done thousands of presenters in my life. My job has been to draw them out to become real. And, I put them in front of the camera. First of all, I make sure they’re looking at me. The camera. This is video on the web! Don’t look off like it’s the press. It’s not the press. The press is over here. I can see that. I’m here! I’m talking to you right now, right into the little green dot on my Mac. That’s real important.

Second, I start off by saying from behind the camera to the guy, “So, are you nervous? ‘Yeah, I’m a little bit nervous.’” Be nervous. It’s not bad to be nervous. You’re nervous! You’re on media! It’s good. It’s good to be nervous.

The other thing I say which will help you all, I think, who have to go in front of the camera. Think “future.” Video isn’t about the moment, it’s about the future. Somebody’s seeing this in the future. Therefore, what do you need to say where a month later, a year later, five years later, it’s going to have meaning for that person you’re trying to reach?

Mark Fidelman: Well said! Well said. Michael, I want to ask Dave: David, live video. What do you think the difference is between recorded video and live video, and can everyone cross that chasm?

David Hoffman: Wow. What a good question! My vote is some people are good live, other people are not. The introverts of our world tend not to be … Mark and I are more extrovert than Michael is. Michael has learned how to give his personality and rhythm, and add credibility without really attacking you or being as aggressive as Mark and I are. So my vote is, if you’re not good on Facebook Live or YouTube live, don’t do it!

Mark Fidelman: Yeah.

David Hoffman: It’s not the thing for you.

Mark Fidelman: So would you instruct them to kind of practice to see if you’ll get good, or there are just people that are good live like Broadway and there are other people that are good on screen because, you know, they need to practice their part fifteen times before they go on?

David Hoffman: A good question. You must have directed video in your life. A lot of times, we’re selecting characters from the phone. From the phone! We’re not seeing their face! So I use things like, “Hey! Do you like your work?” And if the guy goes, “Ehhh, no, no, no.” He’s not good in “live.” The live people have definite thoughts about everything, even if they change their mind. “I really like this burger. I don’t know what it is. But the burger, wow!” That’s a certain kind of person. He makes a definite kind of statement. So my vote is, you’re not so good on live. If you want to contemplate, it’s not a contemplative meeting.

Michael Krigsman: You know, when I started CxOTalk, the concept was live from the start. And, David’s right. I am actually pretty introverted, although I’ve gotten used to doing this. But, the idea for CxOTalk was when you do it live, it creates a certain level of tension because there's no safety net, and that means there's pressure on me and there's pressure on the guest. And when it works well, it means you get the very best, the best possible thinking of everybody because they're on their A-game. That's why I like live video.

David Hoffman: Well, you're running a program for an audience. And your audience, I asked you before we started, is a broad audience that cares about the subjects we talk about, and maybe some that care about video. There are two things looking at the video. I could be moderately good on video. My mother is terrific. One of the best political campaigns of this last season; I think it was South Carolina or North Carolina; a woman put her mother on to talk about her. So, always go to the most credible, authentic, charming character. Talking about you is better than you! Corporate leaders could do that. I mean, they've got to be human beings also. They can't just be reading the teleprompter. I wonder what Mark thinks of the teleprompter?

Mark Fidelman: I don’t like using it. I can’t use it, or I sound like a robot. And maybe back to what David said about live, it’s not a complete tell. But, if they’re good on stage and captivate an audience on stage, I’ve seen most of them make the transition to live video and they’ve been great. Sometimes, they’re not. You know? Looking into the eye of a camera isn’t a natural thing to do. You really have to get used to it. And I find if they can do that in front of an audience, they have a good shot of doing it live. But me, I don’t do a lot of live video, I just don’t think I’m that good at it. I’m better recorded and contemplating what I want to stay and making sure it’s structured in a way that I think the audience will appreciate and I’m not wasting their time. So, that’s kind of my shtick.

But I recognize that there are people that are very funny and very entertaining that can do this in 30-40 minute chunks. I couldn’t do it.

Michael Krigsman: We have a question from Twitter. And, Arsalan Khan says, “Most companies are thinking about video for their external audiences. Is there a difference if you’re creating video for an internal audience?”

David Hoffman: You bet. Really good question. Two examples, because examples help. When … Who was Jane Fonda’s first husband? Tom something-or-other? That guy? The Senator.

Michael Krigsman: Tom Hayden. Tom Hayden.

David Hoffman: When he was first running for the Senate, he said, “I knew there was a problem with American car companies when my father, who worked at a Chrysler plant, bought a Honda because he saw a Chrysler commercial on television that he didn’t believe.” I ask you, when Morgan Stanley says, as they did last month, “Morgan Stanley: For 140 years, we’ve been your trusted ally.” Does anybody believe that? Does the president of Morgan Stanley believe they’re my ally? You’ve got to be kidding! Imagine the employees. They’re laughing around the bar about that! When you’re speaking to your internal audience, you’ve got to go right for it. If you’re firing a thousand people, and you want to make a video that helps the other 19 thousands think they’re safe, you’ve got to acknowledge that you feel awful about this. I feel awful about this. The credibility factor gets even higher when the corporation is talking to itself because that’s where the people know the truth. We all know that.

Mark Fidelman: Yeah, I mean, if I were running a big corporation, and I agree with David, I would be doing a lot of probably live and recorded [film] at the same time. I do kind of a three-minute pitch to my employees and I make sure that I talk about the relevant items of the day, the week, the month, where we’re headed; you know, it’s such a powerful thing to do and you could distribute it internally pretty easily and connect with every single one of those employees.Too many people have told me today that from the boardroom all the way down to the front-line worker, there's this huge disconnect. Now, what better way is there than video to kind of make that disconnect less uncomfortable and have a better relationship with people because you're connecting with them on a one-to-one basis.

I have to agree; I think more corporations use it. I think they’re not because of legal implications, but for me, I think it’s the future. I think it will happen and I think it’s a great idea.

Michael Krigsman: Okay …

David Hoffman: I just want to say two things. One, I totally agree with what Mark just said, but I have a really interesting insight. I was doing a commercial for Sikorsky, and Sikorsky Helicopter is part of United Technologies. And who got most affected by this commercial? They had to sell a hundred corporate helicopters. They did that. It was the employees. Why? Because in the commercial, I honored the people who made it. And all of a sudden, the employees thought [they were] worth something. You know what? You give a guy a raise, 25 thousand bucks, 30 thousand bucks, it’s not going to change his life. You do something that his wife and kids, or her husband and kids, can see at home; can feel, “I’m proud of what my dad or mom does.” Enormous change. That’s really what video’s about.

So, the employees matter. Mark is absolutely right. It’s a really powerful tool, but better be believable.

Michael Krigsman: I want to know, how do you tell a story? Video is about flow. It has a beginning, it has a middle, and eventually, it comes to a close. How do we structure the flow in order to tell a story?

Mark Fidelman: I’ll jump in, but I think David’s going to be better at this than me, because of all of the stories that he’s told in long-form. But, I always like to look at the classics, and draw lessons from the classics; you know, how they invoke emotions, villains, good endings … You know, why do movies like Star Wars connect with us? It’s about the future, it’s about science fiction, but still, those stories are about other things, and if I’m a brand, I’m usually about other things other than real life. How are they making those connections with people? So, if I can do that in less than ten minutes, and tell that story, have emotion, and follow some kind of a classical story structure with a story arc, I think that’s a big win if you do it right.

David Hoffman: Story arc. He’s absolutely right about that. But I’ll tell you what really does it. First, start at the opening. This is such a male thing, but it works for females, and tell me what you’re doing. When females sometimes start stories to each other, particularly, they wind the story out. That doesn’t work in video because nobody goes back. So if I don’t start off by telling you what I’m going to tell you now is a thing I’ve learned that’s worked for a hundred people, now you’re set. If you don't want to hear it, and this [becomes] the other key thing, you have no "ear-lids." So, if you're listening to a podcast, you don't turn the video off, you're still listening even if you're doing other stuff. You're still listening, Your ears are amazing. They make the story of interest to the audience, tell the audience up-front what it is I’m about to tell you, and why I feel it might be of some value to you, and then run it in such a way that I’m surprised. People, kids, everybody loves, “Oh, look at that! Look at the […]”

Mark Fidelman: Yeah. Yeah, great idea. Yeah, I mean, I think of it as a mini-movie just like you do, David, except I have the foresight to know I’m not Steven Spielberg, so I always seek professional help when I need to. We’ve got comedy writers, we’ve got other writers to help tell the story, but … So, if you’re not a good storyteller, there are places to help. It’s not very expensive, and those resources are right at your fingertips. All you have to do is ask David, for example, or…

David Hoffman: Well, Mark, no, no. I looked at some of the videos on your website. Comedy is the God-damn hardest thing to do.

Mark Fidelman: It is.

David Hoffman: I could […] right now. And yours are good. Just that I laugh even once, or even smile, that is just not easy, and dumb commercial-makers, we all know this, on YouTube when you see the commercials up-front, they say, “Oh. Comedy holds them for the ten seconds.” How many time do you watch the commercial on YouTube, or how many times you click “skip it.” It ain’t easy. I saw two of yours, Mark. I liked them a lot.

Mark Fidelman: I know, and I agree. It’s not easy. So, if you are going to pursue comedy, and you’re not extremely funny, I would hire writers. But I don’t know what you think David. I tend to move away from comedy unless I’ve got a huge hit. There have been some brands. If you look at what Taco Bell has done, maybe a little bit about what Wendy’s has done, they’ve done some pretty funny, humorous videos. But in general, I kind of steel away from it until there’s a greater comfort level with it.

David Hoffman: Totally agree. Here’s why. How many people go to Taco Bell laughing? How many people go to Taco Bell because it’s cheap and they can feed the kids? How many people go to Taco Bell feeling a little bit guilty that maybe it’s going to make them fat? Who’s your audience? That’s what you want to do. I’ve run … You won’t believe this… I’ve taking YouTube ads that are three minutes long. You can click off in five seconds, and I’ve got fifty percent of the audience watching the three minutes because it doesn’t look like a commercial. It’s not trying to sell you anything. It’s pretty much the kind of videos you make, Mark. They’re not commercials.

Mark Fidelman: But how do you feel about conversions on those, David? I mean, do you find them converting? It’s one thing to show a funny three-minute segment, but are they really going to sell your audience? And if you’ve got a long-term perspective, perhaps. Maybe a short-term, it might not be the right thing for your clients. Do you agree?

David Hoffman: Absolutely agree, and I think it’s important to think about, are you … Because everyone says, I say, “Who do you want to reach?” They say, “Everyone.” It’s rarely everyone. Yeah, if it’s Taco Bell, it’s everyone. But if it’s a nuclear power plant device out of GE …

Mark Fidelman: Right.

David Fidelman: …or a heart device, that ain’t everyone. I really, in my work with GE, I helped GE focus on the web, on understanding how to target that heart device at the doctors and the hospitals they had to reach and nobody else.

Michael Krigsman: Let’s talk about YouTube. […] We have a question, again, from Arsalan Khan, who really raises a great point. He says, “Do video analytics help influencers become super salespeople, or is it just gut feeling?” This entire question of creating stories, when it comes to the web, it is about creating stories, or is it about figuring out how to get lots of people to view? Because a lot of people think it’s the latter.

Mark Fidelman: I think it’s both, really. I mean, I look at what the influencers are starting … I know we’re starting in terms of analytics. You can see how long they watch your video on average. You can even do it the individual level if you’ve got the right tools. We use TubeBuddy. I can’t recommend that tool enough for YouTube. YouTube’s analytics is really good as well. You can really start to understand what videos do well, what don’t and then kind of reverse-engineer them to find out, “Why am I not connecting with my audience here, and why am I connecting with my audience in other places?” I think the thing about influencers is they’re not afraid to put themselves out there, be authentic, and find that… The audience really finds them, but really talk to that audience about things that they’re interested in. I mean, there are people that, I think …

Unbox Therapy, for example. They get over a million views just on boxing tech products. And the reason is, is because he does it in a fun, entertaining, interesting way; but if he doesn’t like a product, he says so. He doesn’t hold back. He’ll lambast you. So, you’ve got to be careful what you send him because might review it, and it might be a bad review.

Dan Hoffman: I would say, you’re asking a question about analytics. I agreed with everything Mark said, by the way. Analytics, I use heavily. I use two types of analytics. I agree with Mark. YouTube’s pretty good. Really good. Seeing where did somebody drop off, where did most people drop off; X-number drop off from the first fifteen seconds. They're not your audience. X-number last the whole time of the video. Interesting! X-number drops off, 32% in. That's also interesting because they might have lasted longer. So, I do use analytics, but then I do something really strange. I call somebody who’s watching it. I ask people, “Can you watch that video? I like to record it.” Because, what people say gives you far more insight into the emotional reaction they’re having, which if you said, “Okay, David. What’s the most common reaction?” The most common reaction is, “He’s not talking to me! And I don’t know why I’m listening. I don’t know why I’m watching.”

That’s not good. That means either the person doesn’t understand the audience, or he’s got the wrong audience. In some videos I’ve done, I get a 5% viewership. 5% don’t click off in thirty seconds. But I want the 5%. I’m fine with that. Those analytics really help.

Mark Fidelman: Yeah, I have another example, Michael, I’d like to share that’s less about storytelling; I’d be interested in what David’s opinion is on this, and maybe even your own, Michael. There is a YouTube channel called “First we feast.” It’s an interview show, not unlike this, but it’s very unique. All they do is they set up ten different hot sauces, each getting progressively hotter. They bring in a celebrity, they bring in other YouTubers, and they place those hot sauces on ten wings; remember each getting progressively hotter, and after every time they eat a wing, then they have to answer a question. And you can imagine by the end of it, this thing is burning a hole in their tongue.

So, there’s episode in particular with Kevin Hart. This might be the funniest show I’ve seen on YouTube, but it doesn’t really follow a story, it just follows the career of somebody that’s being interviewed there. But, I think people are tuning in, because they want to see how this celebrity really breaks down and starts crying by the tenth wing, because it is a phenomenally entertaining show.

Michael Krigsman: So I’d say there’s a humanizing quality associated with that.

Mark Fidelman: Yeah, when you’re breaking down from being burnt by hot sauce, and you’re about ready to vomit, there is a humanizing quality about that.

Michael Krigsman: David, how can we humanize our videos, which gets directly to the point of authenticity that you raised earlier?

David Hoffman: I’ll make a list. I mean, I told you, start off by making mistakes. Mistakes immediately draw the audience to you. I’m sympathetic to that guy. He’s not perfect, or that girl […]; I’m not being sexist about this. And two, I would say tell stories from the heart even if they’re the … Wonderful idea; you eat this stuff and it’s got spicier and spicier, and then you start burning your thing, and the reactions are real. That’s got an arc to it; what we call a story arc. That’s pretty good. I like that. I was thinking of Pepsi and Coke. Remember when Pepsi went into the streets of Central Park…

Mark Fidelman: Yeah, the Challenge.

David Hoffman: The Challenge. And before that, there was Avis and Hertz, “We’re Avis, we’re number two.” Oh, boy! The audience went, “I want number two!” You wouldn’t have thought that. So, I say, resonate with the audience, which is the key for the people watching, by being, I’m not saying “real,” because that’s the wrong word. That means “honest.” You don’t need to be honest. Honest, sometimes good, sometimes not good. But real. Like, I’m going to tell you right now why I know my Taco Bell is made with better ingredients, but I know it. I’m just going to take it as a given. That guy I trust….If you want to see what not to trust, watch commercials on television. Everybody has got a DVR that’s speeding past them. The advertising agencies – can you believe this? – are still ignoring that that’s what their viewers are doing! They’re saying, “Well we have 750 million views.” They did not because people are DVRing. The commercials are unbelievably fake, for the most part, even when they’re faking being real.

Mark Fidelman: But David and Michael, what some of the … because I work with brands a lot, and maybe, David does as well. I'm sure he does. A lot of them are afraid to make mistakes. I mean, I'm doing a series of twelve episodes for a company now, and my God, everything has got to be perfect. Now, unfortunately, it's still coming across as somewhat inauthentic, but there is no room for mistakes. You can't make a mistake, or else there are serious repercussions. At least they're telling themselves that.

So, I’m wondering what David thinks about that, and I think, from what I’ve seen, that’s the maturity of brands that are producing their own videos; less so about influencers, because they know the influencers have already got a big audience. They don’t care if they make mistakes with their own audience. But if it’s coming from the brand itself, I’m finding they’re afraid to make mistakes, and therefore, not willing to take that step to authenticity that you were talking about, David.

David Hoffman: Totally agree. You have a business; a part of your business that I would use. I didn’t know it until Michael [brought us to] this talk today, but influencers can make mistakes. So even though the company has its lawyers, God, get the lawyers off the script. They wreck everything, right Mark?

Mark Fidelman: That’s right.

David Hoffman: And they wreck everything. They get the lawyers…And then remember there was a Time Magazine ad once, which had a two-paragraph thing coming out of an oil company or something. And every single word was crossed out and changed, and the answer was, "Don't get a group to write a script." But you know that. And the brands, oftentimes, pass that over to PR. Oh, God. Don't ever pass to PR. Pass that over to marketing. Or, would you pass it to the legal department? So, he's got this other business. Influencers; they can say the things, make the mistakes, be the real that you can't be. And to me, that's … I don't know how many agencies actually do what you do, Mark, but it’s a very appealing way to get around the silliness inside the corporation of making a mistake.

Michael Krigsman: I mean, personally, you know, for myself, I work also with large companies, and as well as with startups. And I think that that fear of making mistakes, which is the desire to appear perfect, and to present the image that we don't have any vulnerabilities, it's such crap because the reality is that, I mean, seriously, right? It's bullshit. The reality is that we're all alive; I mean, presumably, we are, since we're having conversations about this stuff…

Mark Fidelman: … Well not according to Elon Musk, we’re not.

Michael Krigsman: Well, there’s … you know. Most of us are alive.

Mark Fidelman: Yeah.

Michael Krigsman: Anyway, so we all know that being alive means that things can go wrong. Things can fuck up, and they will. And, it happens, and so we don’t need to smooth out. You know, we don’t need to smooth out.

David Hoffman: Well Michael, Michael. Just look at United Airlines. Can you believe United Airlines? That CEO ought to be trained by Mark and I and you, Michael, to be a human being. In my early life, what was the famous chemical company in Bhopal? They poisoned twenty- …

Mark Fidelman: Dupont?

David Hoffman: No, it wasn’t DuPont. It was the other big one in Connecticut. Michael? Anyway…Big, huge chemical company.

Michael Krigsman: Oh, Union Carbide.

David Hoffman: Union Carbide poisoned 22 thousand people, and Mr. Anderson, who had never smiled hardly ever, got on his internal and external media and said, “I think I’m going to cry.”

Mark Fidelman: Oh.

David Hoffman: That was the start of his talk, having poisoned 22 thousand people. The United Airlines CEO, the guy has no feelings! Those PR people had to train him to have feelings. What a disaster!

Mark Fidelman: They trained the feelings out of their CEOs. You know? That’s what they do. I’ve seen their training. It’s ridiculous.

Another good example would be Tylenol when they came out after somebody had poisoned Tylenol. That was I think a good response to it. We still see these corporate hacks, these executives that are toeing whatever legal line that the PR company has told them, and they come across as unauthentic, and people don’t believe them, and it just makes matters worse. And I think, David, you’ve highlighted what happened at United. It’s a case-in-point.

David Hoffman: You know, another one; I’m not going to name a name but it’s a giant sort of company. When you work with them as I have, everything goes through a group called “Public Affairs.” Public Affairs has to approve absolutely every out-facing commentary from Coke to keep the brand whole. As Mark and I both know, to understand a brand is a critical thing. And they provide great insights to that, but to communicate to an audience requires the audience you’re communicating to – which Public Affairs people don’t have a clue about. You put those two things together, and you make […]. You may power on video, in my view.

Michael Krigsman: Alright. We have about seven minutes left. This has been a very fast conversation. I’d like to go back to the notion of building an audience. And I realize that it all starts with understanding the audience, communicating to the audience, evoking emotions and being trustworthy. But, I also want to know the cheap tricks. I want to know how I can get more people to watch my stuff. Are there any cheap tricks? [Laughter] Or does it all come back to that same thing, which is “hard work?”

David Hoffman: I have a cheap trick, but I want to hear Mark’s first.

Mark Fidelman: Well, I mean, the most natural way of doing this is if you have other audiences, and most of the brands probably watching this do, whether it’s on email, Twitter, Instagram, what have you; the easiest and fastest way to build an audience on YouTube or Facebook video is to invite them to subscribe, and give them an incentive, do something that will get them to subscribe to your videos, because on YouTube, unlike Twitter, when you subscribe, most of the time, depending on whether YouTube's hiccupping or no, most of the time they get an alert that there's a new video been posted, "Hey, go take a look at this," and they go watch on their mobile; they could watch at their desktop. .. For me, YouTube is probably the number one place most people should be focusing, followed closely by Facebook for video. But if you think about it, YouTube has got a big advantage with SEO value, as opposed to Facebook where nobody goes to Facebook to search for anything. Whereas on YouTube and Google, that video will sit there forever, and to David's earlier point, if you're building videos that have longevity; if you're thinking five years out; then this video's going to pay off for you. A single video will pay off for you for at least five years.

David Hoffman: I’d say three things. And by the way, I never met Mark before and he hadn’t paid me. Influencers really work. I can’t believe it. When an influencer says something about something I’ve made, the views go from 300 that I really care about to 30 thousand in three days. Even if the guy’s got only 300 thousand subscribers, he knows how to deal with them. They’re tough. They can ask for too much money, they can be a pain in the butt, I don’t really know all the details of influencers, but I love influencers.

The goal on YouTube is sharing. You want the video to be shared. So, whenever I look for my weekly analysis on any client, I want to know how many people are sharing it. Am I reaching a guy who’s got cancer at 63, so he’s going to […] the other guy’s cancer, maybe his doctor. That’s very informative.

And, the third thing is, let’s be clear. Facebook is going to kick YouTube’s butt in one major area that’s unreal. Facebook is about friends. So it knows that David Hoffman is speaking to you now from his home. It knows it because I've made a Facebook post. And I've said, "I'm at home." "Oh, home. David must live in […] home, he must like this event, and it finds me." I'm clicking on one in ten Facebook ads right now. Why is it Facebook sponsored posts? Why? Because they know me! They're figuring out, "Oh, the guy's interested in new glasses. Oh, he wants to go to Cancun beach." It's fantastic! YouTube is about the searcher. "Wonderful," as Mark says, and huge, and China’s opening up. I don’t know what that’s going to do to the total YouTube fanbase.

Facebook is about my friends, and it’s about them knowing so much about me that if I’m one of fifty people in the United States who has a specific issue I want to deal with, they’re going to find a video that if the sponsors were right, that goes right to my heart. Amazing!

Mark Fidelman: I mean, I’ll challenge David a little bit. I agree Facebook is going to head in that direction. However, like Facebook has done to us marketers, and we probably deserve it, they make it very difficult to reach our own audience that we’ve built up unless we pay for it. So you’d better be prepared to pay for it, but yes. Second-to-none, Facebook is the best targeting that’s out there. But I think YouTube will quickly catch up. You know, I have a lot of faith in Google, as much faith as I have in Facebook. So, it will be interesting to see what happens, but I think YouTube is a primary place for the next couple of years because of the search value.

Now, if you’re producing Facebook Live videos, way better place to go than YouTube. So, depends on what you’re putting out there.

David Hoffman: Totally agree. I think that YouTube is the greatest network. The greatest video network the world has ever seen, so surpassing any other system that I ever had to find an audience that enjoys and reacts to what I’ve done. Amazing!

Michael Krigsman: So, in our last few minutes, I’d like to ask each of you to share your advice and David, let’s start with you. You’ve spoken about this, but you’ve done so many videos and some of your videos; like, you did one, was it BB kIng at Sing-Sing Prison? Was it Sing-Sing?

David Hoffman: Yup.

Michael Krigsman: That is so phenomenal! How do you do it? How do you do it?

David Hoffman: Well, from the corporate perspective, they know that sponsoring events that people like, care about, is a great thing to do. The problem is, and it’s a real challenge, how to connect a corporation’s values with BB King, or with Earl Scruggs, or many other people that I film. Not exactly influencers, Mark, in a modern, YouTubey world. But if it was to the public, celebrities.

Mark Fidelman: […]

David Hoffman: […] BB King for one purpose. The guy who paid for this. And I said, I can get your purpose in there, but this story is about BB King. Otherwise, what’s the point? It’s so popular on YouTube today, I made it when I was young. I made it when I was 25. It got millions of views on YouTube. It sells hugely, and the clients have long gone. The company’s long gone. That’s happened with my AT&T work, that’s happened with a lot of these companies. The company’s gone, but the videos are still running. That tells you something, Mark, about the life of these things. [Laughter]

Michael Krigsman: Alright. And, Mark, once we’ve got a video, and we want to market it, and we want to put it out there, what’s the best way to do it?

Mark Fidelman: Well, I mean, David stole my thunder, as usual.

David Hoffman: [Laughter]

Mark Fidelman: Influencers are the best for two reasons. One is organic reach, I mean, because they’re introducing their video to your audience. It’s all organic. Second, you can pay for it, and you can pay for it on Facebook and Youtube, and Facebook will be better targeted, but YouTube does a great job of targeting as well, so that’s another way. Very inexpensive. You can get it down to a target audience to two, three cents in some industries per view. I mean, that’s phenomenal! I mean, you couldn’t do this a couple of years ago. And then thirdly, is share it on your other channels and encourage your employees, especially if you’re in a big organization, to share it internally.

David Hoffman: Totally agree with what he just said. Didn’t say that. Really good. The employee base in enormously powerful. Their spread is so great that when you’re lucky enough to do something that you asked them to employ engagement programs, ask them to share. It skyrockets viewers that mean something. Not just viewers, viewers that mean something.

Michael Krigsman: Okay. And on that note, it’s time for Episode #230 of CxOTalk to draw to a close. And in the spirit of sharing, I am going to ask each one of you to like us on Facebook. You can do that now. Click the little YouTube button, and subscribe to us on YouTube. And, call five of your friends and tell them to do the same thing. No, seriously. Thank you so much to Mark Fidelman, and to David Hoffman for joining us on this very interesting episode of CxOTalk, talking about video. Next week, we are joined by David Edelman, who is the Chief Marketing Officer for Aetna, a big insurance provider. And we are going to talk about how changes in the insurance market, environment, and expectations affect a huge insurance company. Thanks so much, everybody. Have a great day, and we will see you soon.

Mark Fidelman: Thanks, Michael.

Digital Transformation of Healthcare

David Chou, VP, CIO and CDO, Children’s Mercy Hospital
David Chou
Vice President, CIO and CDO
Children's Mercy Hospital
Dion Hinchcliffe, Chief Strategy Officer, 7Summits
Dion Hinchcliffe
Chief Strategy Officer

Is digital innovation really happening at traditional healthcare players or are so-called "healthtech" startups really leading the charge? As regulated industries finally undergo digital transformation in a major way, CXOTALK brings back well-known CIO David Chou to the show for an update on what he's seeing take place in the healthcare industry today. The latest CIO topics will be discussed as well, including the need for more IT agility in 2017 plus an updated look at decentralized digital transformation using the change agents model.

David Chou is the Vice President / Chief Information & Digital Officer for Children’s Mercy Kansas City. Children’s Mercy is the only free-standing children's hospital between St. Louis and Denver and provide comprehensive care for patients from birth to 21. They are consistently ranked among the leading children's hospitals in the nation and were the first hospital in Missouri or Kansas to earn the prestigious Magnet designation for excellence in patient care from the American Nurses Credentialing Center.

Download Podcast

Digital Transformation of Healthcare

Dion Hinchcliffe: Hello and welcome to CxOTalk, Episode #231. It’s Tuesday, May 9th. Welcome to the show! We have a very special guest, today, a person I've got to know recently, thanks to global travel: David Chou. He is the CIO and now also the CDO, Chief Digital Officer, of Children's Mercy, Kansas City. Welcome, David!

David Chou: Thank you, Dion! Thanks for having me!

Dion Hinchcliffe: You bet! We’ve had you on the show before, so we’re hoping to do kind of a catch-up on all the things that are happening. Digital is changing faster than ever. I’ll be helping to revisit what you’re up to, how things are, your new role as Chief Digital Officer; so, maybe if you can tell us a little bit about … give us an update on Children’s Mercy. You know, get [us] familiar with how big you guys are, any interesting stats, and we’ll take it from there.

David Chou: Yeah. Thank you. So, Children's Mercy is a pediatric standalone hospital in Kansas City. We're a $2 billion organization; we have almost 8,000 employees. [An] interesting stat is we’re an academic medical center. We’re building a huge research arm; a lot of investing there. And we are training facilities for a few surrounding colleges like the University of Kansas, University of Missouri, Kansas City; so, we pride ourselves on being that flagship hospital from a pediatric space, here.

Some interesting stats: We had over 20,000 procedures last year, over 14,000 patients; most importantly, we serve everyone. So, what does that mean when we say that we serve everyone? That means whether you have insurance or not, we provide care. So, we have an estimated total of about $119 million in uncompensated care for the state of Missouri and Kansas. So, that’s one of our missions that we pride ourselves on, is any kid who enters a hospital, they’re going to get the best treatment that we can provide in terms of care. And, that’s one of the many reasons why I joined the organization. When I saw the quality of care that this organization provides…It’s truly exceptional and it’s a place where I would take my kids to. And that speaks a lot when you work in the healthcare space, where I’ve seen organizations where I probably would have second doubts about bringing my own kids to the facility.

Dion Hinchcliffe: Now that's a great mandate and mission that you guys have there, and it probably gets you excited to get up in the morning to use technology to improve things there. Describe your role there now. It's shifted a little bit. You were Chief Information Officer, and now that role has expanded. What are you responsible for and how has that grown over the last few years?

David Chou: Yeah. So, you know, we added the title of Chief Digital Officer, because we’re in a digital world now. If you think about every organization, they are a digital organization. Whether you like it or not, doesn’t matter whether you’re in agriculture or finance or healthcare, you’re a digital company. So, that’s part of the reason why I want to make that stamp on the title, where in addition to leading the core technology in all of the various functions of the CIO, starting to have the organization transform and think about a digital experience. And that could mean a lot of things. But, it really puts that stamp in terms of my role. I would say when I first started here a year ago, a lot of the transformation was in the culture, building a team. They would say I would only be able to spend about 60% of my time focusing on digital and focusing on the business side of the house. Now, I’m at 80%. And, that’s where it needs to really be first to transform the department, the internal organization; that’s going to help us buy the best care for the citizens of Missouri and Kansas.

Dion Hinchcliffe: I’ve actually worked a little bit advising the board of directors of healthcare associations, and I kind of get a pushback. The people who are into healthcare are really caught up in the mission that you just described before, right? Helping sick people get better. Do you tell them that every organization’s becoming a digital organization? Every company’s becoming a technology company, and what do they say about that?

David Chou: It’s still a very new concept. Some folks who may give me their […] as well, come from a technology leader, of course; you want to be the center of attention. But it is true when you think about the biggest and best one that we had as an organization. Our biggest investment has been technology. It's not about the MRI machines; it's not the […] robots; it's actually our spending on the technology portfolio end-to-end. So, if you think about it, you’re spending hundreds of millions of dollars on technology, but [if] you’re not operating as a software provider or even a product vendor, you will never be successful. So, it’s a mind-shift, and you have to change that perspective. And, that’s something some of the things I always talk [about] to with my senior leaders. When I put it into perspective, the amount of money that we’re spending on technology, we have to really change our mindset.

Even more so when we talk about the CDO role – the Chief Digital Officer: lots of organizations have approached me; big healthcare systems where they have great CIOs, and the CEO would say, “You know, David, here’s my dilemma right now. I have a great CIO; world-class, renowned; does a great job of keeping the lights on. But, I feel I’m missing something. I don’t feel that I’m transforming towards the digital era. Do I need to find a Chief Digital Officer? Is that the right play?”

So, I keep hearing a lot of those discussions coming from CEOs across the US from a healthcare perspective where they know they’re missing something. They haven’t figured out what is that gap. And, I think that’s where the role of the CDO comes in. But hopefully, the CIO can also play that role, because that’s my expectation; someone who can lead the technology and conditions…

Dion Hinchcliffe: It's the old "one throw to choke" argument, right? It's ultimately if you don't want technology rolling up to two high-level parts of the organization doing what will end up being two different things, then you've got to figure out how do you create one, unified technology environment? And that's a big challenge. But, it was very interesting what you said that now that you have to manage down, but you need to manage up and help the leaders in the organization navigate to a digital future; how much of your time is spent doing that, and trying to coach the other C-levels in your organization, coach the board, and any other steering members you might have?

David Chou: I would say part of my role … I’m a sales guy. My wife is always asking, "What do you do every day?" because she knows I can't fix computers, and part of my job is education. I sell upwards, downwards, laterally, so yes; definitely educating the board, educating the other peers in the C-Suite is critical. Because, if you think about the world that we live in, everyone has a great retail experience. Everyone would like that same experience in a hospital setting. The convenience factor, the Amazon factor; that's what we're looking towards. You want to personalize the experience. But healthcare's still catching up in terms of creating that experience.

So, at the same time, we, as consumers in other industries have this great experience, but somehow, were not able to transform that when we're buying care. So, that's what we're going to have to move down to, and that's part of the education process where I want to get people thinking about their experience as consumers and try to transform that into how we operationalize.

Dion Hinchcliffe: Yeah, that’s a great insight. And, I think a lot of people wonder about that; they don’t feel like that consumer-quality experience. Yet, as you pointed out, healthcare organizations are very tech-centric. They’re bristling with technology and how they deliver almost everything they do. Why is that? Is it regulatory constraints? So, what are the issues? Why can’t healthcare consumerize as much as other services?

David Chou: It’s a very risk-averse industry. It’s also very traditional, where if you look a – I’ll be very frank – look a some of the leaders and decision makers in healthcare; this may be their last gig. Or, they may want to set up one more big gig after this. So, when you have that environment, where people are risk-averse, and they may not take the big gamble on making the big decision of change, the whole industry doesn’t change. That’s number one.

Secondly, a lot of these major enterprise healthcare software platforms; they’re built on legacy code. So, mounts, Cobalt, these are what's driving these enterprise multimillion-dollar, billion-dollar systems. So, when you have that core infrastructure on legacy technology, how do you transform? How are you going to create APIs? How do you get to that point of scale and where the other industries are? I would say, those are the biggest two parts, in my opinion, which is forcing healthcare to not adapt as quickly. But the reality is, we have come a long way in the past eight years. We have to come a long way towards being electronic …. There's a key mandate of getting to electronic records. But, that's just a starting point. We have not got to the other …

Dion Hinchcliffe: And then where does that investment come from? When I talk to most CIOs, one of the things they’re asking is, how much are your budgets put into keeping all of your legacy infrastructure running? All the apps and all the services you have now? And, it's usually between 80 and 90%, leaving about 10-20% to innovate and to build out that digital future. Is adding that CDO role giving you more of a budget? You know, how are you able to start growing that, because, legacy mount, as we call it, is really pulling back a lot of CIOs that I’ve talked to.

David Chou: Yeah. And I think that’s a great point. You think about most CDOs coming in, they don’t have the operating budget, and they may be one man shop, or maybe they have another member of the team, but they don’t have the spending authority. Part of the reason why I made the first to get the CDO title, I still had the spending authority as the CIO. The same budget, I didn’t ask for anything more, but now, I could transform and shift the technology spending to maybe more digitally central and centric, versus focusing on core technology.

Getting all things to the cloud, getting out of the datacenter business; moving everything into a mobile-first platform. That’s what we’re moving towards. So, because I have the budget of the CIO, I’m able to transform and get to the digital experience a lot faster with the spending authority.

Dion Hinchcliffe: Yeah. That's very interesting. Now, the CIO tends to have, you know, the experience […] their budget. We just got a question that came in on Twitter. For those of you that are watching the show, you can always post your questions with the #cxotalk hashtag, and we’ll try and get to them. We have a question from Scott Weitzman: “How are patients responding to the digital transformation of their care? Do they actually like this? Do they want to use mobile apps on their phone?" I can imagine some, maybe not. What has your experience been so far?

David Chou: You know what’s surprising? The biggest adopters of digital technology are the elders; folks over 60 years old. Well, obviously now I’m in the pediatric space, but when I had orientations that had both adults and kids, our biggest adopters of technology were the ones that were over 60. And, they should have an excuse that they're people that don't know how to use technology. Those are the days of the past. Everyone knows how to skype; everyone knows how to use technology to communicate with their grandkids or their kids. So, it's everywhere around us. So, the adoption rate is high, especially in the pediatric world, where most of our consumers, the kids' parents, they're millennials. They grew up with this technology. They want it more than ever. They want to be able to text. They want to be able to have a web chat versus calling in, versus having a face-to-face conversation.

So, for us, especially in the pediatric world, it’s in high demand, and we’re seeing some good adoption. Not there yet, because the technology portfolio is not where it needs to be in terms of maturity models yet, bet we will get a lot of adoption from folks who are craving that digital experience.

Me personally, I would prefer to use that virtual care model where I could create an appointment, I could get a doctor [setting aside] thirty minutes for my kid. Then I could get a prescription at the local pharmacy; retail pharmacy; all within that one hour. I will pay the additional dollars, which is not covered under my insurance, to get that convenience, that I would bet most folks would pay that premium for.

Dion Hinchcliffe: Well, I have health insurance, but since I travel all the time, I have a mobile app where I can see a doctor for fifty dollars. I have to say I use that more than I use my regular healthcare provider. So, what’s interesting is that convenience, and kind of the urgency, because it’s a health issue, trumps concerns about privacy or the fact that it’s unfamiliar to most people, still?

David Chou: I would say so. If you think about the world that we live in, right? All of us are social individuals and social creatures, Dion. They know where we are as we speak. So, in terms of privacy, we give up a lot of privacy for the convenience factor. I would say, that trend is going to continue as we evolve, where privacy might not be as big of a concern. But yes, this is still a lot [to deal with] for folks concerned with privacy, but the reality is the convenience really outweighs that privacy concern. That’s the trend that we’re moving towards.

Dion Hinchcliffe: Yeah. Excellent. So, before we take the next question, can you give us an update on your digital transformation journey, since we last spoke with you? What are the major parts of it? So, you know, what did you encounter? What’s up next?

David Chou: Yeah. So the start of any digital transformation is a journey. It’s a cultural transformation that needs to start and we’re very early in that maturity model. So, I would want an organization that did not have a digital strategy, let along a technology strategy. It may seem a lot that the core components have been so … The first year coming in was putting together a roadmap. Getting together the right leaders and shifting our mindset to be more digitally-focused, and that’s why we’re headed on that journey. It’s almost similar to any organization that has gone through a lean transformation where the start to the culture […] is not an easy button I could buy to solve this problem. But, the easy part is actually the technology. I could buy the greatest technology out there that could solve the problem, but without the adoption rate internally and externally, I couldn’t get it done.

So, for us at Children’s Mercy, we’re Year Two of the transformation; of this digital experience. And the end-goal is to try and create…We want to create that retail-like experience for healthcare.

Dion Hinchcliffe: Yeah, so talked to a lot of programs. Now I hear about them being in Year Two, and the stakeholders, they don’t see all of the plumbing in the infrastructure; the opening up of the data. You know, all of that takes an enormous amount of effort to do that before you can even show any results. So, are you running into challenges like that? Saying, you know, “Year Two, you should have all this stuff out?” And you’re like, “No. I’m just trying to build my infrastructure to make the art of the possible happen?”

David Chou: Fortunately for me, I’ve had great support here. People have understood what has transpired. I walked into an organization where my predecessor was here for 25-plus years, so they knew there’s a lot of legacy, there’s a lot of things that were done they way it has been mentality. The change doesn't happen overnight. Fortunately for me, I had a lot of support in terms of getting this transformation started, but like you said, the plumbing and the pipes, the foundational element; there's a lot to it that needs to be fixed. And, my biggest challenge is going to be right now – it is right now. How do I transform while keeping things going? But most importantly, I'm trying to think about how do we leapfrog? We think about developing countries versus developed countries. The developing countries may have an advantage where they can leapfrog and bypass some of that traditional infrastructure. So, that's where I think we may have an advantage. […] That’s the mentality in mind for the organization.

Let’s just leapfrog. Let’s just forget about going through that traditional path building the same infrastructure. Let’s just get the latest and greatest and make that leap. So, fortunately …

Dion Hinchcliffe: You know, I hear that more and more now. Instead of taking three steps to transform, saying, “Why can’t we just hit the target,” right? In the next change, instead of just saying …

So, for those of you just joining us, we have a very special guest, David Chou, one of the top CIOs in the world. He makes the top of the social CIO list on the internet, and somebody I recently met myself in person.

We have a question from Twitter, from Arsalan Khan: “Do you think doctors should be taught in school to get involved in technology even before a start as doctors?”

David Chou: Definitely. If you look at what’s been happening right now, the choice for … Let’s use residents. Residents get trained at […] organization who has System A. I’ve heard that they’re signed where they may make a choice as far as where they want to work based upon the technology portfolio. Now, how is your portfolio? If someone has System B versus System A, and they were trained in System A, they may decide to work in the organization that has System A. So, this technology portfolio has to be a competitive advantage in terms of recruiting.

So, I definitely do agree that we do need to train. The other reality is this. I see lots of positions and medical staff who, because they were not trained appropriately, they’re spending a lot of time documenting and using the system. Going electronic was not … The sales pitch was not the expense to make it faster. There are a lot of benefits to being electronic. Being able to do things quicker is definitely not the reason why we want to go electronic.

So, what’s been happening is we have positions who are going home, spending late night hours finishing their documentation, and part of the reason they may not have […] appropriately, maybe have not set their system appropriately, just based on their expertise. Therefore, they spend a lot of time. So, I definitely do encourage them to think they need to put technology training or some sort of core training as part of that core curriculum because we are a technology world, you know? People grew up learning how to type, and that worked for us and that's a core component.

Dion Hinchcliffe: Computer illiteracy has been such a big deal.

David Chou: Yeah.

Dion Hinchcliffe: But should your doctors and nurses need to learn to code?

David Chou: Code? No. I don't believe so, but there are a lot who love to code, and they do it as a hobby. I would say if you think about where the …

Dion Hinchcliffe: But that’s not teaching the digital mindset about what’s possible, but how the stuff actually works. I’m not saying that they shouldn’t continue coding, but should they learn to code?

David Chou: They may; just the logic. I think learning the logic of coding for healthcare; even just one curriculum, just to understand the logic of how the system works, because you get a lot of questions. How come you just can’t do this, Well, they could understand the logic of why the computer software works. That may help them defer some of the questions that they may have.

Dion Hinchcliffe: Yeah. Well, I think that’s the issue, see, is people get into healthcare feeling very passionate about caregiving, and they’re not technology first. They’re caregiving first. And so, it’s just a matter of priority.

So, you said something important about adoption. And so, how do we do this? How do we change the technology that if we do that, we also have to figure out how to get the people to change, too. So, how do you go about doing that? What ways do you think about making that dual-hand-in-hand technology people-change?

David Chou: Yeah, the simple answer is we have to communicate. You have to go back to the traditional path of communicating. You have to go down to the traditional path of, "May I have a handhold?" and walk people through this change. Unfortunately, there's no easy button. People like to use computer-based training for some of this. Sometimes it doesn't work. So, it is very grueling and brutal, but you just have to do a lot of the traditional communication. Over-communicate! One of the things that I have learned upon any enterprise that uses EMR orientation … I could have the greatest build that only trained them about 20% of the effort, and the output would be a very lackadaisical, even bad install where I can have a terrible build by training users 40% more, 50% more how to use a system. That’s a bad install. That isn’t working right now. But the end-result, “Hey! That’s the greatest install ever!” Why? We were training appropriately. We spent the time training them, educating, the teaching them about the workaround. So, it’s a growing effort, and it all comes down to communication. Unfortunately, in my opinion, I don’t see any technology solution that’s going to be able to solve this adoption and logistical…

Dion Hinchcliffe: Well, it’s just not a technology problem, right? Digital transformation requires an investment in people, then. That is what I’m hearing. Yeah, I don’t think that’s always clear to everyone that’s involved in that process. There’s a lot of focus on technology and the digital possibilities, and not “How are we going to get our people to do that?” And that takes us to, before we get to our next question, takes us to the … What models do we see emerging? Now I know you’ve been kind of involved in those discussions online: the whole concept around empowering change agents, saying, “Let’s not spend too much time with the people that aren’t ready to change. Let’s go and find the people that are super excited to change, that are energized to do it, and empower them to give them tools and resources, support, and education. What are your thoughts on … Is that the kind of thing that we’re going to see in the future?

David Chou: That has been my model. So, I've had the luxury of building a new team. The people that I have hired and brought on are change agents themselves. So, imagine if I bring two leaders who are change agents out of a total of six, direct reports, there's going to be a lot of just peer pressure to transform themselves to be able to keep up. So, I think that's the greatest model, if you're able to create that peer influence, create a group of change agents into an organization, influence their peers to adopt change; that’s the best effort, because you don’t want to be that peer that’s left behind while everyone is moving ahead.

In lots of organizations, the other challenge that I have seen is when they bring too many change agents, but they’re not changing the process themselves. The organizational process … So, it’s a tough balance. How do you bring the right change agents with the right fit? Someone that knows how to culturally navigate an organization; that’s one of these unspoken things that is not brought to as big of attention as I would like to see is this “cultural fit.” You see them during our […] chat that political savviness of driving change is …

Dion Hinchcliffe: Yeah. Great change agents have to be mentioned or look bad, right? You know, folks like yourself, you know… I think it's great to hear that you said all that because I've seen what makes you that visionary CIO is that ability to see that we can't do it all ourselves anymore. We helped with the entire organization, so how do we tap into them, and how do we teach them the things they didn't know yet, so they can be effective in helping us with digital transformation?

David Chou: Yeah. One of the keys that I personally believe is I’m trying to hire all my […] have the ability to take over my job. That makes my job a lot easier. That’s a very …

Dion Hinchcliffe: That’s a great point.

David Chou: Right, I want them to be able to do my job so that I could transform myself individually because it also goes back to me. I need to transform myself. I need to change myself on a daily basis. When I get too comfortable, there’s something wrong. So, there’s a lot of that self-reflection that needs to happen. It’s not about having other change agents in the organization, I personally have to change on a regular basis.

Dion Hinchcliffe: Yes, is there company indicator that you use to know you’re heading in the right direction as you feel pulled a little bit out of your comfort zone?

David Chou: Exactly. I mean, there are days when I say, “You know, I’ve been pretty comfortable the last three months.” I sort of went on, I won’t say “cruise control,” but autopilot. I know exactly what’s coming down. I like to predict things. We probably are not transforming ourselves individually, let’s go back to the drawing board. Let’s talk about what needs to happen. That way, you always keep ahead.

Dion Hinchcliffe: Yup. Great. So let’s go ahead and take our next question. This is from Paul Turner. The question is, and I think this is a good one, “What digital transformation in healthcare technology is most closely linked to moving to a value-based care model?” Maybe you can give us a little sound byte on what value-based care is, and then maybe your thoughts on that.

David Chou: Yes, so let me tell about value-based care. Let’s look at health care in general. This is the only industry where you do not get paid what you charge. I go to a grocery store, I buy a chocolate bar, I buy a milk, and there’s a price for that. What they charge is what I pay. Healthcare’s different. We have this whole reimbursement model where I could charge you a hundred dollars, but I may only get paid at the facility of a hospital thirty dollars, even though I charge a hundred bucks. So, this whole shift of value is now, instead of having the fee-for-service model where if I provide a hundred tests, I get paid for a percent of those hundred tests; now, it’s going to be focused on how healthy is this patient? What is the outcome of that procedure? Are they doing better than before, or do they have to come back and get retreated again, or possibly second procedures? So, that’s where it’s going with the reimbursement model, where they’ve got to measure the value outcome.

The technology that I really see that needs to happen in healthcare: we need to have some sort of CRM model. If you think about every industry, they know everything about me as a buyer, but healthcare, what does that leave the hospital? They have no idea what goes on with David Chou. But, there within the hospital with the care, you know, the electronic medical record knows exactly what they’re going to do, exactly the kind of procedures I’m going to have, the type of food I’ve been given, the type of medication in clinical notes, great stuff within those four walls. But what does that leave? It just disappeared. You had no idea. So, our CRM-like model conceptually, where we know a lot about David Chou outside the hospital is going to be that success factor for our population health, because of the prime example is, let’s just say I’m on a low-sodium diet, but I’m going to drive through Burger King. What needs to happen during that time when I’m going to order my Whoppers, “Hey, David! You’re not supposed to be eating this! Order a chicken salad sandwich! Or better yet, order a salad!” Find I way to change my behavior, and that’s where we need to move down towards.

Dion Hinchcliffe: It’s kind of like the difference between productivity and effectiveness. Just because you can do a hundred tests doesn’t mean you should, and it’s not really focused on the outcome, right? I would make David Chou healthy, that’s the value that we’re going after. Is that a good way of summarizing it?

David Chou: It is. And that’s where the industry is moving towards when we talk about value-based development. Value based is really about keeping the patient out of the hospital, and we’re getting paid to keep the patient out; because when you come in, that’s when we know you really need to get treated, and you need attention, and [we] make sure you get treated in the most cost-effective way.

Dion Hinchcliffe: Yeah, and now, are you being pushed to help the organization shift towards value-based care? Is there a mandate coming down the pipe? Help us understand the imperative there and how that’s going to affect digital transformation in healthcare.

David Chou: The reimbursement rate is declining from CMS, which are the providers of Medicare and Medicaid. As that reimbursement starts declining, you're going to be judged based on the quality of care, that's where digital transformation has to occur. In the pediatric world. It still has not […] to the adult world. The adult world is doing this on a regular basis where pediatric will start to catch up. And a lot of these standalone pediatric hospitals have a big market share, so … It's a very tough balance when you think about an organization that [has] 85% market share. How do you transform? How do you say, "Hey, I need to move towards this value-based care model; value-based health model. But, I may lose five million dollars next year because if I'm doing the right thing for the patient…" So the very top asks for senior leaders to figure out when to play that card, and stretch into a pediatric world where the margins are very well right now and there's not a lot of pressure yet. But, it's coming.

Dion Hinchcliffe: So what is the enabling technology? Is it analytics against electronic medical records? Or, what’s making that happen and then feeding that to the doctors or to managers; or, how does that work?

David Chou: Analytics is definitely key, and ERP is huge as well because you need to figure out the cost of providing care. I would bet that most healthcare organizations can’t even […] the actual cost of providing care for one patient. We may give you a good formula, and on average, round it out for a formula calculation, but we don't have the true cost of providing that care. So, that's where data's going to come in; your ERP data; your supply chain. That's all going to come in, in addition to the EMR data. So, that’s the puzzle that everyone’s trying to figure out: how do we become a data-driven company? How do we get this mapping for all 40,000 tables in the EMR to be emigrated? So, the big ask is touching on [something] that we’re all facing now.

Dion Hinchcliffe: Yeah. It's a fascinating subject and it's going to change our lives. So, we're going to talk about population health in just a moment, since you brought that up. But first, we're going to take a question from Sal Raza, who asks, "Can you see VPN or Facebook-type communications to include the voices of patients' families and their caregivers so they can connect the clinical data to provide better care?"

David Chou: Uh, so we’re doing that right now internally. So I’m trying to make; just within my department; a social department. So, we’re using a Facebook-like technology where we’re using the Microsoft Yammer platform just so the last thing to have; a [single] message out to all. But yes, I definitely do see that interaction. The first organization that really gets that right, who has that Facebook social interaction, who… They're going to create the greatest experience. Most organizations are still very risk-averse as they […], where they view privacy concerns, HIPAA concerns, so, therefore, it has not taken off. But, we're seeing these one-off startups that have provided this technology, but it's upgraded very slow.

Dion Hinchcliffe: Yeah. And this is where we see health tech startups kind of giving the old guard guys a run for their money. So, tell us a little bit about… You mentioned population health. I know that it’s an exciting topic in healthcare but I’m not an expert, and I think you know a lot more. Give us a little overview of what that is and why it’s important to your digital transformation efforts.

David Chou: Right. Good, let’s go back to my previous message. The future of healthcare is not about how many patients you're going to see. Traditional healthcare is, "Let's fill my beds up," similar to airlines. How many seats can I fill? The new model of healthcare is, "How can I keep my hospital empty, but how can I make sure that the people who come in are the ones that are required to be treated?" So now, you need to figure out how you provide wellness, and how you provide care outside the hospital. And that's the game of population health. We're trying to gather a pool of patients, and then this pool, keeping them healthier, making sure they do not come into the hospital; making sure that they exercise regularly; making sure they're eating the right meals that are appropriate in terms of sodium, calorie level.

So now, we’re moving to this journey of, “We need to get engaged with them outside the hospital.” You need to figure out proactively, twenty-four by seven, how do you change someone’s behavior. And it’s a tough thing to do, there’s not a simple technology that’s going to do that, but we’re seeing lots of other …

Dion Hinchcliffe: If you provide tools to bring in their caregivers, their family, their friends, you know, there is different things you can do. But it sounds like there's some big data involved to measure all this, there are operations in the healthcare provider to actually engage with these patients, and then there are digital experiences out in the field that patients are connecting to that gives them the support they need that […] help to their lives. Is that […] what it looks like?

David Chou: That's what it's going to look like. The same way that retail [does]. When you walk into Nordstrom, they know exactly their purchases. And they can probably predict whether you're going to walk to the right or left based on your pattern. That's what we're moving towards in healthcare. Amazon knows exactly when I'm going to reorder my laundry detergent, just because they know my tracking; my history behind it. Healthcare has not got to that stage yet when we’re predicting what make of test may happen. And that’s when the data piece is going to come in; all the social data; that’s what we’re going to have to move towards…

Dion Hinchcliffe: Now, is the big business of healthcare really going to find that acceptable? It sounds like you’re actually going to create a solution that's going to create healthy, less sick people, and is going to create fewer patients and less business. How does that work?

David Chou: That’s a challenge […]. How do you go to the board and the CEO says, “You know, I’m going to lose X million dollars this year, but I’m doing the greatest thing to keep this population healthier.” But that’s really the divide right now. Even though we talk of population, it’s very tough to act upon, because.

Dion Hinchcliffe: Is it as a service? This is where we're having a Chief Digital Officer that deeply understands digital and the business of things like healthcare. You could say, "Well, but population health might be a service we can charge for because we're keeping people healthier." I don't know. This is the challenge you're facing, David. How do you make all that work?

David Chou: Yeah. The other challenge, how do you tie in the insurance carriers to the BL line?

Dion Hinchcliffe: Right.

David Chou: You know, there’s two different lines of business, two different lines of incentives where now you have to bring those folks into play. So, data is like … There’s something that came out yesterday where data’s the new oil. Data’s the new currency, and that’s really what we have to move towards in terms of getting data, but most importantly, it's not about the number of data points you have, you've got to filter out the right data. That's the tough part; filter out the right data to make the right decisions. But, you can't get to that […] until you figure out where's your strategy? How are you going to transform […] care? Are you going to make the big focus on virtual care outside of your hospital? Or, do you want to keep it in your full four walls? So, these are tough decisions that healthcare leaders are going through, which is why you're seeing the trend of megamergers. Healthcare is consolidating. The number of hospitals is shrinking. They’re getting bigger and bigger.

Dion Hinchcliffe: You know, that’s interesting. And so, data is clearly destiny in the digital age, and it’s the new Golden Rule, “He who has the data makes the rules,” right? So there’s a big land grab that’s taking place and trying to get all that.

So, we have another question, and please, we really appreciate the contributions. We have time for a couple more questions, please post them on Twitter with the hashtag #cxotalk. We have a question from Kalim Sharik: “What IT frameworks and methodologies are inevitable for digital transformation in the healthcare enterprise?” I’ll throw a couple – while you think about that, David – on the pile. One is DevOps. I think that’s absolutely essential for the fast feedback loops that we need to have to create rapidly evolving digital solutions in today’s fast-moving markets. Another one I’d put in there is growth hacking, which is how do we create something initially, and then move that over into what it needs to be to be successful as quickly and effectively as possible? I think those are two important frameworks. What do you think?

David Chou: We’ve got Agile. That’s something we’re practicing right now in terms of using the Agile methodology just to … for something as simple as driving down the number of outstanding reports that we need to provide out to our customers. We’re starting out with over 900 reports out; 900 requested reports now. I wouldn’t say we finished all 900, but we cut down half based on the Agile methodology.

Now, I will also add the traditional IT/ITO that’s really customer service-centric. You still have to have that service-oriented mentality when you run an IT shop. So, the combination of Agile, DevOps that you had mentioned, ITO, these are all necessary frameworks. The key is not to be so hung up on perfecting these concepts, you know? Go with the Agile-lite, DevOps-lite just to get things moving. If you’re hung up on that 100% perfection, it may take you forever to get there, and you may never get there; therefore, defeating the whole purpose of trying to go down this path of […] platform.

Dion Hinchcliffe: Although, I’m still thinking of a challenge of we have a risk-averse and a regulated audience as you said, and they’re going to want everything to be perfect, right? That’s your cross to bear.

We have a question from Katie Goss and Scott Weitzman again. They want to know about the role of artificial intelligence, that’s a buzzword we’re seeing in the press all the time now, in healthcare. And also, what are the security concerns about that? Where do you see the role of AI in healthcare?

David Chou: That's going to be weird. I mean, the ability to predict what may potentially happen: huge. Even bots. I mean, a personal example is I've been using a bot as my personal assistant in addition to my regular EA. I've been using them simultaneously. And it has to […] some people. Some people respond to the bot as if it's a normal person. We can also use bots for interacting with patients to see if they have some questions regarding a common cold. That may be able to come in and just create a different level of engagement while satisfying the consumer. So, AI is huge, but unfortunately, healthcare is still…They're thinking about it, but I don't see it really there yet. People talk about it, but I would say we’re still five to seven years away from having anything real in terms of AI. But I definitely see the benefits, especially the ability to predict what may potentially happen in terms of someone’s health. That’s huge, right? If I knew I was going to…

Dion Hinchcliffe: It might play a big role in population health, right? The best demo I ever saw, which I don’t think was real, but it made a convincing scenario, which is: The doctor is in that Facebook-like chat with the patient trying to work through some symptoms that they’re having. And in the background, the AI bot is watching the conversation between the doctor and the patient, and pulling up all the relevant medical records, all the relevant symptom analysis in real-time and presenting all of this additional information to support the doctor’s decision in their healthcare process, during that collaborative session with the patient. And they got something that bots, with AI, can really support caregivers and take that load off them, and do a lot of legwork. So, I think there are interesting things happening there.

David Chou: Yeah. Here's one extra thought just to think about. You think about why it may not happen that quickly. Think about your medical profession. You're a doctor. You went to school for eight years of your life because someone's training you on how you're going to come out with your medical judgment. Now, here, you have a bot coming in saying, "Hey! My judgment's better than yours. I didn’t go to school for eight years. I’m just a robot!” How do you feel as a professional? Going to school spending an entire career getting to that point, and now getting a bot coming in and predicting what you think the test […] to predict.

So, I think that cultural transformation needs to happen, but that’s also a big reason why it has not taken off as much because traditional healthcare, you go to school, you go to medical school, and by the time you graduate, you’re supposed to be that expert versus having someone automate that or most important, having a machine or robot …

Dion Hinchcliffe: This is going to be our big challenge; it’s that. And we’re continuing to get great questions from Twitter, so thank you for those. We have one from Megan Jamis from Textra Health, and the question is: Do you see value in patient-generated data? Do you agree patients might trade that data for better patient experiences?

David Chou: Definitely. We trade our data right now to retailers. I give up that privacy for convenience. But, the challenge is going to be let’s just say we have patient-generated data. Let’s use wearables. I have all this data about myself on my wearable and I transmit it to a doctor. What if the doctor doesn’t trust the data? And that’s going to happen. That’s probably the key barrier right now. A physician or a medical professional may not trust that data even though it’s generated by you as a patient. They may discard it.

Then, you’ve got coming in, “Do the test again,” or do whatever diagnostic it is to get that data again. How does that work? Are we duplicating that effort even though we’re trying to streamline? I think that’s the cultural barrier, and I think that’s the biggest challenge. But I do see patient-generated data as valuable. But, it’s only valuable to us. What about the other side that’s going to …

Dion Hinchcliffe: Now, this is the whole "Not invented here!" discussion. You know, the same problem in healthcare is that "If it's not ours, then how can it be any good?" I think that in digital, we talked about ecosystems, right? We understand that collectively, we're much smarter and better and more valuable when we work together, and so we can partner with our patients, we can partner with other clinics and healthcare organizations, and we can pull our data, and we can do much better that way. And yeah, it's that cultural shift. How do you make that happen? And so, how … We get this all the time: "Culture eats strategy for lunch." How do you overcome that and is that a major barrier for you?

David Chou: It is, and I would say for the entire healthcare industry it is. So, you’re seeing lots of little niches and pockets of innovation happening, but in the grand scheme of things, it’s not there yet. So, how does that happen? There’s no magical answer. You’ve got to have these change agents like you said who are going to be the decision makers. Until we get that, it’s a very slow-moving progress. But, there’s progress where you look at some of these niche players who are taking market share. It is happening, but it’s not at the scale that we have seen like the taxi companies, like some of these others; AirBnB. It has not happened at that scale yet. There’s still small pockets here and there. Unfortunately, it’s not big enough to really disrupt the industry.

Dion Hinchcliffe: Yeah. So, I think probably the most significant challenge that the average organization faces is not so much the people behavior, it's the mindset, it's the natural inclinations. And, I think we will see as Millennials come in, as you mentioned, they have very different expectations. You have probably new nurses and doctors who say, "I want to work with healthcare providers that are making the investments in technology because that's part of our mission. The better our technology is, the better I serve my patients."

So, let’s wrap up with one question, then we’re going to take one more question on Twitter as well before we round out this episode. How do you view customer experience in the content of the latest technology advances? And this also goes in with a question that Paul Turner asked a little while back on Twitter. A lot of healthcare imaging is still on CDs. Yet now we’re looking at, in terms of customer experience … We know that augmented reality, virtual reality, holographic displays; all of this stuff is right around the corner and is about to be in everybody’s homes. The PlayStation VR. It is going in everybody’s homes right now. You know, how do you close that gap?

David Chou: Yeah. You’re starting to see the adoption pick up. Let’s just use AR and VR: instead of just thinking about how it could be used for patients, think about how powerful that technology can be for you to train your employees? Workers’ comp injuries are huge in hospitals. Teaching the clinical staff how to bend at a right angle; how to pick up a patient at the right angle using AR and VR; these are some of the simple things that we can use the technology for. So, I see that adoption rate growing as we speak more and more. I just do not … I don’t see enough focus in […] firm, healthcare organizations thinking about how they can use it rather than hearing the buzzwords. Now, these are some very simple use-cases of AR and VR. But, rather than think about how it can be applied to a patient, think about how it can be used in operations, training. So, there is still a lot of education that needs to happen; still a lot of traditional CIOs who are not thinking that around. They're focusing on keeping the lights on. Let's make sure my poor infrastructure's in place.

Dion Hinchcliffe: Yup.

David Chou: Make sure my EMR’s in place. Because you spent $100 million on EMR, you’d better make sure that’s in place. And then you forget about everything else. So, unfortunately, that’s where the state of healthcare is.

There was a survey that was put out there that was put out there where 30% of the hospitals are still going through EMR organization. That’s huge! I could have thought that was probably an overwork, but I didn’t think that was still 30% of healthcare organizations.

Dion Hinchcliffe: Yeah.

David Chou: That’s still the trend that we’re at.

Dion Hinchcliffe: Well still, it’s a very exciting industry that you find yourself in at a momentous time in history. And so, we appreciate you coming in and taking the time to visit us on CxOTalk. And for everyone out there watching the show, I would really appreciate it. CxOTalk.com has a list of all the upcoming guests, and we’re looking forward to having David come back on the show sometime again, soon!

Data Science at Zillow, with Stan Humphries, Chief Analytics Officer

Stan Humphries, Chief Analytics Officer, Zillow Group
Stan Humphries
Chief Analytics Officer
Zillow Group
Michael Krigsman, Founder, CXOTalk
Michael Krigsman
Industry Analyst

Zillow is one of the largest real estate and rental marketplaces in the world, with a database of 100 million homes in the US. The company pioneered data-driven, automated home value estimates with the Zestimate score. On this episode, we speak with Zillow's Chief Analytics Officer and Chief Economist, Dr. Stan Humphries, to learn how Zillow uses data science and big data to make a variety of real estate predictions.

Dr. Stan Humphries is the chief analytics officer of Zillow Group, a portfolio of the largest and most vibrant real estate and home-related brands on Web and mobile. Stan is the co-author of the New York Times Best Seller “Zillow Talk: The New Rules of Real Estate.”

As chief analytics officer, Stan oversees Zillow Group financial planning and analysis, corporate strategy, economic research, data science and engineering, marketing and business analytics, and pricing analytics. Stan was one of Zillow’s earliest pre-launch employees and is the creator of the Zestimate and its first algorithm.

Stan also serves as chief economist for Zillow Group. He has built out the industry-leading economics and analytics team at Zillow, a recognized voice of impartial, data-driven economic analysis on the U.S. housing market. Stan is a member of Fannie Mae’s Affordable Housing Advisory Council and the Commerce Department’s Data Advisory Council. Stan also serves on the Visiting Committee of the Department of Economics at the University of Washington.

Download Podcast

Data Science at Zillow, with Stan Humphries, Chief Analytics Officer

Michael Krigsman: Welcome to Episode #234 of CxOTalk. I'm Michael Krigsman, and CxOTalk brings to you truly the most innovative people in the world, talking about topics relating to digital disruption, and machine learning, and all kinds of good stuff. Before we begin, I want to say a hearty "Thank you" to our live streaming video platform, which is Livestream. Those guys are great! And if you go to Livestream.com/CxOTalk, they will even give you a discount.

So, today, we are speaking with somebody who is a pioneer in the use of data and analytics in consumer real estate. And we're speaking with Stan Humphries, who is the Chief Analytics Officer, and also the Chief Economist of the Zillow Group. And I think that everybody knows the Zillow Group as Zillow and the Z-Estimate. Stan Humphries, how are you?

Stan Humphries: Hey, Michael! How are you doing? It’s good to be with you today!

Michael Krigsman: I am great! So, Stan, thanks for taking some time with us, and please, tell us about the Zillow Group and what does a Chief Analytics Officer and Chief Economist do?

Stan Humphries: Yeah! You bet! So, I’ve been with Zillow since the very beginning back in 2005, when what became Zillow was just a glimmer in our eye. Back then, I worked a lot on just algorithms, and some part development pieces; kind of a lot of the data pieces within the organization. We launched Zillow in February of 2006, and back then, I think people familiar with Zillow now may not remember that between our first couple of years between 2006 and 2008, all you could find on Zillow was really all the public record information about homes and displayed on a map. And then, a Zestimate, which is an estimated the home value of every single home, and then a bunch of housing indices to help people understand what was happening to prices in their local markets. But, we really grew the portfolio of offerings to help consumers from there and added in ultimately For Sale listings, mortgage listings, a mortgage marketplace, a home improvement marketplace, and then, along the way, also brought in other brands. So now, Zillow Group includes not only Zillow brand itself, Zillow.com but also Trulia, as well as StreetEasy in New York, Naked Apartments, which is a rental website in New York, HotPads, and a few other brands as well. So it’s really kind of grown over the years and last month, all those brands combined got about 171 million unique users to them online. So, it’s been a lot of fun kind of seeing it evolve over the years.

Michael Krigsman: So, Stan, […] you started with the Zestimate. You started aggregating data together, and then you came up with the Zestimate. What was the genesis of that Zestimate and maybe you can explain what that is?

Stan Humphries: Yeah. Sure! So, we were, in the early days, we were looking at different concepts that seemed like there was a lot of interest in from consumers about real estate and, I think there was a lot of angst about really what we think as an economist, we think of as information asymmetry. So, the fact that certain participants in the marketplace of real estate have a lot of information, and other people don't have any information. And, we felt, I think, in a lot of the leadership team that found that Zillow … You know, our reference point was, you know, we were very passionate about more of this social progress of transparency in various marketplaces, which you had seen in the 80s and 90s in stock markets. But we had been part of, actually prior to Zillow at Expedia, about eliminating information asymmetries in the travel agency space. You had seen it in insurance and a lot of different sectors. We were very interested in kind of creating information transparency in the real estate sector, so that got us very interested in where was the information people wanted, and how could we get it; and how could we make it available for free to consumers?

And once we had done that, a lot of that information is squirreled away and county tax assessors and county recorder offices around the country … And how our country's organized is those tend to be more than 3,000 different counties around the country, and each office has a different format of the file, and it became our job to go to all those different places and get all that data and put it online in a standardized way.

And, you asked about the Zestimate. The way that came about was once we had done that and we bring people in in the early days, and we’d show them a UI of what we were trying to do. We showed them these maps of recently sold homes, and then you could click on any house and see the public facts, and when it was last sold. We noticed that people had what we thought was really a really focused interest on recently-sold homes, and they would jot them down on napkins when we brought them into the offices to look at the user interface for focus groups. And we were like, “What are you doing there?” It became clear that they were very interested in looking at recently sold homes in order to understand the value of a home they might be looking to either buy or sell in the future. And that was kind of an a-ha moment where we were like, "Wow! Okay, if you're trying to figure out an estimated price for a home, then maybe we can help you do that better than just napkin math." So that was the genesis of the Zestimate and today, we do a whole lot more than napkin math. It is a very substantially computationally and processed reassessment.

Michael Krigsman: How has the Zestimate changed since you began it?

Stan Humphries: Yeah. So, back in, if you look at when we first rolled out in 2006, the Zestimate was a valuation that we placed on every single home that we had in our database at that time, which was 43 million homes. And, in order to create that valuation in 43 million homes, it ran about once a month and we pushed a couple terabytes of data through about 34 thousand statistical models, which we thought was, and was, compared to what had been done previously, was an enormously more computationally sophisticated process. But if you flash forward to today; well actually I should just give you a context of what our accuracy was back then. Back in 2006 when we launched, we were at about 14% median absolute percent error on 43 million homes. So what we've done since, is we've gone from 43 million homes to 110 million homes today where we put valuations on all 110 million homes. And, we've driven our accuracy down to about 5% today which, we think, from a machine learning perspective, is actually quite impressive because those 43 million homes that we started with in 2006 tended to be in the largest metropolitan areas where there was a lot of transactional velocity. There were a lot of sales and price signals with which to train the models.

What's in the rest of, as we went from 43 million to 110, you're now getting out into places like Idaho and Arkansas where there are just fewer sales to look at. And, it would have been impressive if we had kept our error rate at 14% while getting out to places that are harder to estimate. But, not only did we more than double our coverage from 43 to 110 million homes but we also almost tripled our accuracy rate from 14% down to 5%.

Now, the hidden story of how we’re able to achieve that was basically by throwing enormously more data, collecting more data, and getting a lot more sophisticated algorithmically in what we are doing, which requires us to use more computers. Just to give a context, I said that back when we launched, we built 34 thousand statistical models every single month. Today, we update the Zestimate every single night and in order to do that, we generate somewhere between 7 and 11 million statistical models every single night, and then when we’re done with that process, we throw them away, and we repeat again the next night. So, it’s a big data problem.

Michael Krigsman: How did your, shall we say, algorithmic thinking, change and become more sophisticated from the time you began … What was the evolution of that? That must be very interesting.

Stan Humphries: Yeah. It certainly has been. There have been, you know, there have been a few major changes to the algorithm. We launched in 2006. We did a major change to the algorithm in 2008. Another major change in 2011, and we are now rolling out another major change right now. It started in December and we'll be fully deployed with that new algorithm in June. Now that's not to say every single day in between those major releases; we're doing work and changing bits and pieces of the framework. Those times I described there is kind of major changes to the overall modeling approach. And what has changed is, as probably suggested by the fact of how many statistical and machine learning models are being generated right now in the process, what has changed a lot is the granularity with which these models are being run; meaning, a lot finer geographic granularity and, also, the number of models that are being generated. So right now, when we launched, we were generally looking at a county and in some cases for very sparse data, maybe a state, in order to generate a model. And, there were, like I said, 34 thousand of those different models.

Today, we are generally looking at … We never go above a county level for the modeling system, and large counties, with a lot of transactions, we break that down into smaller regions within the county where the algorithms try to find homogenous sets of homes in the sub-county level in order to train a modeling framework. And that modeling framework itself contains an enormous amount of models, where there are models … Basically, the framework incorporates a bunch of different ways to thin, about values of homes combined with statistical classifiers. So maybe it’s a decision tree, thinking about it from what you may call a “hedonic” or housing characteristics approach, or maybe it’s a support vector machine looking at prior sale prices.

The combination of the valuation approach and the classifier together create a model, and there are a bunch of these models generated at that sub-county geography. And then there are a bunch of models which become meta-models, which their job is to put together these sub-models into a final consensus opinion, which is the Zestimate.

Michael Krigsman: This is very interesting and I want to remind people that we’re talking with Stan Humphries, who is the Chief Analytics Officer and also the Chief Economist at the Zillow Group. And I think most people probably know the Zestimate that automatically estimates a value for any piece of real estate.

Stan, so you’ve been talking about your use of data and the development of these models. But, real estate has been a data-intensive business, right? The analyst shares real estate data, but it’s static data. And so, again, what were you doing, and how did this change the nature of the real estate market? So if you can go from the technology into the disruptive business dimension?

Stan Humphries: Sure. Yeah. But you know, I think you’re right Michael, in the sense that there’s always been a lot of data floating around real estate. I would say, though, that a lot of that data had been largely impacted, and so it kind of had a lot of unrealized potential. And that’s a space that, as a data person, you love to find. And, honestly, travel, which a lot of us were in before was a similar space, which is dripping with data, and a lot of people had not done very much with that data, and it just meant that really a day wouldn’t go by where you wouldn’t come up with “Holy crap! Let’s do this with the data!” And, you know, real estate was one where we certainly had multiple listing services had arisen … But the very purpose of facilitating the exchange of real estate between unrelated brokers and, which was a very important purpose, but it was a system… There were multiple listing services which were between different agents and brokers on the real estate side. There were homes that were for sale. You had, though, a public record system which was completely independent of that, and actually two public records systems: one about deeds and liens on real property, and then another which was tax roll.

And, all of that was kind of disparate information and … What we were trying to solve was the fact that all of this was offline, and we really just had the sense that it was like, from a consumer’s perspective, like the Wizard of Oz, where it was all behind this curtain, and you couldn’t really look…You weren’t allowed behind the curtain and you really just wanted to know, “Well, I’d really like to see all the sales myself and figure out what’s going on.” And, you’d like the website to show you both the core sale listings and the core rent listings. But of course, the people who were selling you the homes didn’t want you see the rentals alongside them because maybe they would like you to buy a home not rent a home. And we’re like, “We should put everything together, everything in line,” and we had a faith that type of transparency was going to benefit the consumer and I think it has where …

You know, what's been interesting in the solution is that you still find the agency representation as very important, and I think the reason that's been true is that it's a very expensive transaction. It will be generally for most Americans, the most expensive transaction and the most expensive financial asset they will ever own. And so, there has been and continues to be, a reliance, I think, a reasonable reliance on an agent to help hold their hand for a consumer as they either buy or sell real estate. But what has changed is that now consumers have access to the same information that the representation has either on the buy or sell side. And I think that has really enriched the dialogue and facilitated the agents and brokers who are helping the people, where now a consumer comes to the agent with a lot more awareness and knowledge, and is a smarter consumer, and is really working with the agent as a partner where they've got a lot of data and the agent has a lot of insight and experience; and together, we think they make better decisions than they did before.

Michael Krigsman: I want to tell everybody that there’s a problem with Twitter at the moment, and so if you’re trying to tweet about the show and your tweet is not going through, try doing it a second time and sometimes that seems to be making it work.

Stan Humphries: I am so glad to hear that you said it, Michael, because I just tried to retweet right before I got on and I couldn’t do it and I thought it was my Twitter app. Sounds like it’s Twitter overall.

Michael Krigsman: Yes, it seems like we’re back to the days of Twitter having some technical issues. Anyway, Stan, in a way, by the act of trying to increase this transparency across the broad real estate market, you need to be a, shall we say, a neutral observer. And so, how do you ensure that in your models, you’re as free from bias as you can be? And maybe would you also mind explaining the issue of bias a little bit just briefly? I mean, we could spend an hour on this, but briefly. So, what is the bias issue in machine learning that you have to face, and how do you address it in your situation?

Stan Humphries: Okay. Yeah. May I ask you for a few more sentences on the bias issue and machine learning? Because as a data person, I’m thinking about it from a statistical sense, but I guess that’s probably not how you mean it. In terms of the business model itself, and how we think and how that interaction with machine learning and what we’re trying to do, we are … Our North Start for all of our brands is the consumer, you know, full-stop. So, we want to surprise and delight and best service our consumers, because we think that by doing that, that, then…

You know, advertising dollars follow consumers, is our belief. And we want to help consumers the best we can. And, what we're trying to construct and have constructed is, in an economic language, is a two-sided marketplace where we've got consumers coming in who want to access inventory and get in touch with professionals. And then on the other side of that marketplace, we've got professionals, be it real estate brokers or agents, mortgage lenders, or home improvers, who want to help those consumers do things. And what we're trying to do is provide a marketplace where consumers can find inventory and can find professionals to help them get things done. So, from the perspective of a market-maker versus a market-participant, you want to be completely neutral and unbiased in that, where you're not trying to … All you're trying to do is get a consumer the right professional and vice-versa, and that's very important to us.

And that means that when it comes to machine learning applications, for example, the valuations that we do, our intent is to try to come up with the best estimate for what a home is going to sell for; which is, again, thinking back from an economic perspective, it's different than the asking price of the offer price. In a commodities context, you call that a bid-ask spread between what someone’s going to ask in a bid; and the real-estate context, we call that the offer price and the asking price. And so, what someone’s going to offer to sell you their house for is different than when a buyer’s going to come in and say, “Hey, would you take this for it?” There’s always a gap between that.

What we’re trying to do with Zestimate is to better inform some pricing decisions such that bid-ask spread is smaller, such that we don’t have buyers who end up buying a home and getting taken advantage of when the home was worth a lot less. And, we don’t have sellers who end up selling a house for a lot less than they could have got because they just don’t know. So, we think that having great, competent representation of both sides is one way to mitigate that, and one way that we think is fantastic. Having more information about pricing decision to help you understand what that offer to … offer-ask ratio, what the offer ask-spread looks like is very important as well.

Michael Krigsman: So, from a data collection standpoint, and then a data analysis standpoint, how do you make sure that you are collecting the right data and then analyzing it in the right way so that you’re not influenced […] wrongly or over-influenced in one direction, or under-influenced in another direction, which would, of course, lead to distortions in the price estimates.

Stan Humphries: Yeah. Let's see, I'm trying to think of vices that we watch for in the evaluation process. I mean, one obvious one is that the valuation that we're trying to produce is a valuation of an arms-linked bear market exchange of a home which, those words are important because it means that there are a lot of transactions which are not full value at arms-length. So, if you look in the public record and you start to build models off the public record, you've got a lot of homes that are a lot of deeds that are […] claimed due to the works. And you know, they are ten dollar exchanges of real property, which is not a fair value. And, you have some that are arms-length, where parents are selling homes to their children for pennies on the dollar, and those aren't fair value either. And then, of course, the most common example from the past housing cycle is a foreclosure or short-sale, where, you know, we're not trying to… We do provide a foreclosure estimate, for foreclosures, but the Zestimate itself is designed to tell you what that home would transact for as a non-distressed piece of inventory in the open market; which means that we've got to be really diligent about identifying foreclosure transactions and filtering those out so that the model is not downwardly biased and becomes really a [...] between a non-distressed and distressed property. So that's one area that we kind of have to watch for quite a bit.

Michael Krigsman: And we have a question from Twitter. I’m glad this one went through. I’m having trouble getting my tweets out there. And, this is an interesting one from Fred McKlymans, who asks; he’s wondering how much the Zestimate use-case – how much as the Zestimate helped define, rather than just reflect real estate value? So, what impact has Zillow itself had on the market that you’re looking at?

Stan Humphries: Yeah. That's a question we get a lot, and particularly as, you know, as our traffic has grown is people want to know, "Do you reflect the marketplace? Do you drive in the marketplace?" And my answer to that is that on any given… Our models are trained such that half of the Earth will be positive and half will be negative; meaning that on any given day, half of [all] homes are going to transact above the Zestimate value and half are going to transact below. [...] I think [this] reflects the fact of what we said since launching this Zestimate, which is we want this to be a starting point for a conversation about home values. It's not an ending point.

You know, there was a reason why the name “Zestimate” came from the internal working name of a Zillow Estimate. We got tired of calling it a Zillow Estimate so we started to call it a Zestimate. And then when it came time to ship the product, we're like, "Why don't we just call it that?" You know, but it was called the Zillow Estimate, Zestimate, not the price because it is an estimate. And, it's meant to be a starting point for a conversation about value. And that conversation, ultimately, needs to involve other pay means of value, include real estate professionals like an agent or broker, or an appraiser; people who have expert insight into local areas and have actually seen the inside of a home and compare that inside and the home itself to other comparable homes.

So, you know, that’s kind of designed to be a starting point, and I think the fact that half of homes sell above the Zestimate and half below, I think reflects the fact that people are … I think that’s an influential data point and hopefully, it’s useful to people. But it’s not the only data point people are using, because another way to think about that stat I just gave you is that on any given day, half of sellers sell their homes for less than the Zestimate, and half of buyers buy a home for more than the Zestimate. So, clearly, they’re looking at something other than the Zestimate, although hopefully, it’s been helpful to them at some point in that process.

Michael Krigsman: Mhmm. And, we have another question from Twitter. And again, I’m glad that this one went through; it’s an interesting question: “Have you thought about taking data such as AirBnB data, for example, to reflect or to talk about the earning potential of a house?”

Stan Humphries: That is an interesting … I'm noodling on that. We've done some partnerships with Airbnb on economic research, kind of understanding the impact of Airbnb by housing data that we have. We do a lot of work on that. I think probably the direct answer to that using AirBnB data is "no," but when you say the earning potential, I guess what I'm hearing is the potential to buy that home and convert into a cashflow-positive rental property, and thinks like what's the cap rate, or the capitalization rate of the price to rent ratio. And, that we do a lot of, because we also have the largest rental marketplace in the US as well. So, we have a ton of rental listings, and then we use those rental listings for a variety of purposes, among them being to help understand price to rent ratios and what we compute as … Call it a "break-even horizon," which is how long you have to live in a house to make buying it more worthwhile than renting it.

So, we … And I guess the other thing would directly help that question would be the fact that on any home page, on a page that lists a home, we call them internally a home details page. On any home page on Zillow, we show both the Zestimates, so what we think that home would sell for, and we also show a rent Zestimate, what we think it would rent for. And, that hopefully allows the homeowner to have some notion for if they decided to rent it out, what they could get for it.

Now, the question that I think from Twitter is an interesting new one, which is; our rental estimate is on the rent of that entire home. What if you just want to rent out a room or part of that home? What's your potential on that? And that is a very interesting question, which we thought some about. We don't have a product to directly … Cool product there that seems directly related to the question would be, an estimate on Zillow that would tell you if you did want to rent out a room or two on that house, what would you fetch? And, that's a very interesting […]. Duly noted!

Michael Krigsman: Let’s go back to the discussion of machine learning. Machine learning has become one of the great buzzwords of our time. But, you’ve been working with enormous, enormous datasets for many years now. And, when did you start? Did you start using machine learning right from the start? Have your … We spoke a little bit about this earlier, but how have your techniques become more sophisticated over time?

Stan Humphries: Yeah. I would say I’ve been involved in machine learning for a while, from I guess I started in academia when I was a researcher at a university setting, and then at Expedia, I was very heavily involved in machine learning, and then here. So, you know, there has been … Biggest change … Well, it’s hard to parse it. I was going to say the biggest change has really been in the tech stack over that period of time, but, I should minimize the change in the actual algorithms themselves over those years, where algorithmically, you see the evolution from at Expedia, personalization, we worked more on things relatively sophisticated, but more statistical and parametric models for doing recommendations; things like unconditional probability, item-to-item correlations. And, now, most of your recommender systems, they’re using things like collaborative filtering for algorithms that are optimized more for high-volume data and streaming data.

And in a predictive context, we’ve moved from things like decision trees and support vector machines to now a forest of trees; all those simpler trees with much larger numbers of them… And then, more exotic […] decision trees that have in their leaf nodes more direction components which are very helpful in some contexts.

In terms of the tech stack, you know, it’s been transformed. You know, back in the day, you were doing stuff with seed code; we were using … Maybe we were doing prototyping in ADS-plus. You were usually coding in FORTRAN or C, but you were doing it all from scratch […] on a single machine and trying to get as much as you can into memory. And, you know, today, from that, it has gone through to more proprietary systems; maybe you were using SaaS scale, to then you were maybe using a database, maybe in my sequel, you were using Hadoop. And then today, generally, our firm and other firms that are on the cutting edge here are using something like Spark, probably. Maybe, in coding directly in Scala, or maybe using Spark to plug into Python or R.

And then, generally, those frameworks are now running into Cloud, and are using streaming systems like Nexus or Kafka; real-time triggering of events. And so, all the infrastructure has changed, and I would say for the better. As a data scientist now, you can get up and start working on a problem on, you know, AWS, in the Cloud, and have an assortment of models to quickly deploy much easier than you could back twenty years ago when you were having to code a bunch of stuff; start out in MATLAB and import it to C, and you were doing it all by hand.

Michael Krigsman: Are you looking at the … Are you making predictions about the future value of the home, or only from the past to the present moment?

Stan Humphries: The Zestimate itself is, you know, as a, I guess some people would call it a “now-path,” so it’s a prediction for what the home will sell for today if it were on the market. We do also forecast the Zestimate forward in time. Right now, we project forward about a year. And, that model is a combination of the machine learning models I described before. The point-estimate of what we’re estimating today; and then moves it forward. It’s combining it with the modeling framework - a forecasting framework [with] which developed the purposes of forecasting our housing index – of the Zillow Home Value Index, which tells you basically the home values have done over the past twenty years, and what they will do with the next year.

That forecasting framework is itself a combination of some straightforward, univariate areas, and some more complex structural models that are taking as inputs economic variables, and trying to predict what those economic variables are going to do to home prices in your local market over the next year. We take those forecasts from the index and then apply them to the individual level, with some nuances where it's not just the forecast for your area. We then break that forecast down by housing segments so that maybe, high-end homes are going to appreciate more quickly than low-end homes, or vice-versa. That nuance effects the forecast that is then applied to the property level to create the forecast for the Zestimate.

Michael Krigsman: I want to remind everybody that we’re talking with Stan Humphries, who is the Chief Analytics Officer, and Chief Economist at Zillow. And, if you’re trying to ask a question by Twitter, just keep trying and some of those tweets are actually getting through.

Stan, what about the data privacy aspects of all of that? […] I know you’re aggregating public data, but still, you’re making public information about the value of people’s homes and there’s a privacy aspect to this. So, how do you think about that?

Stan Humphries: Yeah. That’s a, you know… We’ve been fortunate in most of our business operations… Really almost all of our business operations involve the domains that are all a matter of public record. And, a lot of the value-added that we’ve done is to bring that public record data in collating it together into one spot. And, putting it, standardizing it so there’s kind of a standard way to look at it regardless of how they collect data in Idaho versus Florida. They’ll standardize it so that on Zillow, you’re looking at it. Truly our […] easier […] you’re looking at it all the same way.

But, at its core, that's all public record information, which is beneficial when it comes to privacy because all of that data is, at this point, generally accessible. It's all available if you were to walk into a county tax assessor or county recorder's office. And at this point, most of those offices are now online. So, if you knew where to look on the web, then you could find that information online because it is a matter of public record, because of the fact that real estate is based on property taxation. And, it is a longstanding history for why things involving real property, liens, and actual information about real property is public-domain information. But, in all states, most of that information is public, except there are some states where the actual transaction price itself is not a matter of public record, and those are called "non-disclosure" states; states like Utah and Texas. Everything else is public record.

And what we're doing is then providing estimates and derivative data on top of that. So, we're creating housing indices out of that data, or evaluations. And those evaluations are, theoretically, no different than if you were to go to the county tax assessor's website or into their office, there is already a market value to access they're putting on your home, which is a matter of public record. Ours is, you know, we're applying probably a lot more […] hour and algorithmic specification that a lot of tax assessors are able to do. But, in principle, it's exactly the same concept as that.

Michael Krigsman: So how it feels different when that data is aggregated and then presented in such a succinct form. And it’s also easily accessible. Somehow, it feels different.

Stan Humphries: Yes. That's true. I would say it feels different. I would say it feels different in a lot of different ways. For most consumer applications, it feels different because it feels really good. When, for an individual, there are some individuals who would like that information to be…would like all the information! They would like no facts about their home to be public, so they would probably prefer the county assessor to make it public. They would prefer the transactions were not a matter of public record, and they would prefer if companies weren't able to put a derivative product on top of that. I certainly get that. They problem becomes a collective action problem where individually we would all prefer to take all of our information offline, but collectively we would like the ability to look at other information for us to make a better decision. And, collectively, as a society, we have decided that this information should be public. And, because of that, properties like … companies like Zillow are able to make that information public as well, which we think the consumer benefits far outweighs the individual concern that they would prefer the facts about their homes not be public.

You know, I've … There are also, I think, real social equity issues here of there's a lot of research. When you look at kind of disclosure; non-disclosure states, for example, you will find that taxation policy is … There's been some fantastic academic research on this issue, but property tax, there's more inequality in property tax in non-disclosure states than disclosure states because people are able to look at those transactions and figure out how does that tax relate to what that home's really worth? And therefore, disputes are less likely to refer to the lower half of the price spectrum, but wealthy people will always go dispute and try to get a lower assessment on their home. And that leads to more inequality in the assessment of tax than would exist otherwise, which we think is a harm to the overall public benefit.

Michael Krigsman: And you just raised some public policy issues. And so, in our last five minutes, I’ll ask you to put on your economist hat and share your thoughts on how this data economy; and in a way, you’re right in the middle of the data economy. How is … How do you see that changing the workforce and the public policy issues around that?

Stan Humphries: Yeah. I think, you know, we … You know, I do a lot of writing now on policy […] to real estate and housing. And also, some kind of more broader economic discussion. And that broader economic discussion, I do do a lot of … One of the themes I touch on somewhat often is the need for us to get ahead of the changes that are coming due to machine learning and the data era, where I think there are two parts of our social, of our societal framework that will really establish facts. Generally with the last transformation; well, not with the last transformation, probably the one prior to that; where basically, we moved from an agrarian society to a manufacturing society, it was around that time when we started mandatory, compulsory public education. We also started to set up, with the progressive era in the early 1900's, social security systems and unemployment systems that allowed for people who may be thrown out of work from a manufacturing job to have a little bit of a safety net, where they found their other job.

You know, I am concerned in the current … This is less real-estate related and more the impact machine learning is going to have full-bore on our economy, thinking about the impact of driverless cars, for example, on people who drive trucks and cars. You know, that's five to eight million people, and you know, they're going to come under pressure as self-driving car technology becomes more ubiquitous. And, I am concerned that one, we need to up our educational game, where we need to think about college education as being the equivalent in the late-1800's of high school education, and we need to be doing a better job of training in our college graduates for the jobs that exist. And, then I would say that on the unemployment side, that system I described is set up for a world where you lose a job, and your next job is likely to be in the same town you're in, and in the same field.

We're going to go through […] in the next thirty years, a lot of unemployment where the job you need to get is probably not in the area you live in and it's probably not in the field you're in, so it's going to require some re-tooling. And that's more that, like, six weeks to three months of unemployment. We need to think hard about people who are moving from a manufacturing job, and maybe their next job needs to be a computer-assisted machine operator, which is a non-trivial job that needs to be trained for. And you're not going to learn it in four weeks. So, I'm definitely interested in public policy trying to address those issues in a better way.

Michael Krigsman: And in the last two minutes, what advice do you have for public policy-makers on these topics? You mentioned education as being one thing. Any other thoughts on this?

Stan Humphries: Yeah. I would just encourage us to … We seem to be in particularly ideologically-charged times. You know, I would encourage us to think broadly, you know, like we did when we came up with compulsory public education for children and try to think, there are a lot of these ideas that if you think about [it], there are a lot of these ideas that have been suggested from both the left and the right. And, for example, a viable, possible replacement for, you know, short-term unemployment insurance with something more like a more robust negative income tax, which we have a form of that in this country called an "earned income tax credit," where you, for low-wage workers, we supplement their income to the tax system. You know, Milton Friedman was a champion of a very robust negative income tax on the right. We've got a lot of liberal thinkers who have championed it on the left. That type of system, where people can kind of step out of the day-to-day work, and be assured that they're going to make a base-level income for a longer period of time, and that income's going to allow them to get another job. Those ideas have come from the Left and the Right, and I would hope that we're going to be able to fashion a system that's going to work better for the next thirty next thirty years than we've got now; and that we don't get hung up on rigid ideology on it.

Michael Krigsman: Okay. And, I’m afraid that about wraps up our show. We have been speaking with Stan Humphries, who is the Chief Analytics Officer, and also Chief Economist of the Zillow Group. And Stan, thank you so much for taking your time and sharing your thoughts with us!

Stan Humphries: Michael, thanks for the interview! It’s a broad range of topics we got to cover. So, it’s quite unusual, but it’s been fun!

Michael Krigsman: Yeah. That’s great! Forty-five minutes is enough time to dive in.

Everybody, thank you so much for watching, and go to CxOTalk.com/Episodes, and be sure to subscribe on YouTube. And also, “like” us on Facebook. You should do that! “Like” us on Facebook as well.

Thanks so much, everybody! Have a great day! Bye-bye.

CIO Report: Cloud First in Financial Services

Mary Cecola, CIO, Antares Capital
Mary Cecola
Chief Information Officer
Antares Capital
Michael Krigsman, Founder, CXOTalk
Michael Krigsman
Industry Analyst

Historically, financial services has lagged other industries in moving to the cloud based on concerns about security and data protection. As cloud gains greater acceptance, financial services organizations are adopting cloud computing in greater numbers. On this episode, we talk with one CIO who has adopted a cloud first approach to computing. Mary Cecola, the Chief Information Officer of Antares Capital, explains her approach to cloud and offers advice for making a smooth transition. 

Mary joined Antares in 2016 from Federal Home Loan Bank (FHLB) of Des Moines where she served as chief business technology officer. Prior to FHLB, Mary was global CIO, asset management for Deutsche Bank and held IT-related roles at Zurich Scudder Investments, Scudder Kemper Investments and Zurich Kemper Investments. Mary holds bachelor’s degree in political science and a masters degree in computer science from Northern Illinois University.

Download Podcast

CIO Report: Cloud First in Financial Services

Michael Krigsman: Welcome to Episode #226 of CxOTalk. I’m Michael Krigsman; I'm an industry analyst and host of CxOTalk. Today, we are going to speak with Mary Cecola, who is the Chief Information Officer of Antares Capital. And, I want to thank Avanade for underwriting this episode. Avanade is a professional services firm. It is the largest professional services firm serving the Microsoft platform. And, I have worked with Avanade for a long time, and it’s a great company. Avanade, thank you so much for sponsoring, for underwriting this episode of CxOTalk.

Mary Cecola, how are you? Thanks for being here!

Mary Cecola: I’m good today. Thank you for having me, Michael.

Michael Krigsman: So, Mary, tell us about Antares Capital. I know your company began life as an $18 billion company.

Mary Cecola: Yes, sort of. I mean, Antares Capital is a leading provider of financing solutions for middle-market private equity back transactions. The Antares brand has been around for about twenty years and has approximately $18 billion in loans. Antares maintains one of the U.S. middle market's largest senior loan portfolio. But, you are right about starting as ... We're a 20-year-old startup. For a long time, Antares was GE Antares, and has recently, with GE removing itself from the capital area, has become its own company in the last eighteen months.

Michael Krigsman: So, you spun out of GE, and when the company spun out, you decided to come on board, and then you decided to revamp systems. But, maybe that’s not the right word because, in fact, you’re a new company, so you had no systems.

Mary Cecola: Right. Since Antares spun out of GE Capital as its own organization, you had a twenty-year deal credit area that was an ongoing business. But, you know, in infrastructure, we had no technology. We were under a transition service agreement [(TSA)] with GE that was lasting 18 months. I was hired in about four months into that 18 months, which left us about 14 months to replace the infrastructure across the entire organization: five offices, 350 people. As well as we chose not to use any of the GE systems and migrate completely onto new systems.

Michael Krigsman: Before we go into it, I really want to talk about what you did, and how you did it, but before we do that, you’re the CIO. Tell us briefly, as the Chief Information Officer, what is your role?

Mary Cecola: Yeah, as the CIO here at Antares, I’m in charge of strategy implementation ─ everything around technology. And for the first 14 months of that, it’s been the transition services agreement and really making sure we move off of all the GE services and onto Antares services. So, it's been a roll, not only technically, but also this transition services agreement. And that includes watching the facilities migrate out in the different parts of the business.

Michael Krigsman: We are already getting questions from Twitter, and just hang on there, folks. We’ll get your questions. Let’s just get a little bit more into the conversation. Mary, you’re cloud first. You’ve used the term “cloud first,” and you think of yourself as a cloud first organization. That’s really surprising for financial services. So, how did that come about?

Mary Cecola: We had this great challenge ahead of us, right? We needed to migrate out of GE under this timeframe, and bring up all of our systems and create an entirely new infrastructure. You had a greenfield of technology, which is a gift a lot of new CIOs don't have. A lot of my greenfield of technology, though, I had a greenfield of staff. I had no staff. I was the first IT employee as part of Antares Capital. So, one thing we had to do… I took it as a role commitment to be as technology-forward as possible. I did not want to create a technology landscape that we would then be reinvesting in or getting rid of. I did a lot of research on cloud-based technology, and I really wanted to be as forward there as we could.

I will say I saw that as an infrastructure play, if you will, just kind of replacing my servers. And as we get down the road, I’m going to tell you, going completely into the cloud is a lot more than infrastructure play. And, as we went down the line, and we realized we had to have test systems up before I had a network in place, and before I could possibly have bought enough servers to do this. So, if you know you have to do that and get your SaaS systems up because you have to place every application at the same time you're putting in a network across five offices, the only way to do that is actually to host it in the cloud. And, we leverage the Azure cloud heavily.

Michael Krigsman: You say that cloud is not just infrastructure, and would you please elaborate on that?

Mary Cecola: Yeah. You know, as we started moving more and more things into the cloud, we did choose Azure heavily because we wanted to be extremely mobile. The Antares deal teams and the employees here are mobile. They’re fast. They’re dedicated. They can move through and close a deal in 6-8 weeks. So, we wanted to really give [them] mobile technology. So, we are committed to the Microsoft Suite here, front to back with Skype, and everything else to give them that on the road. We also then decided to do virtual desktops in the cloud, and we have our virtual desktops leveraging Citrix in the Azure cloud.

We chose to use OneDrive instead of a local drive and share drive. Instead of share drives, we use SharePoint so they really have all the tools at their fingertips wherever they are. As we kept moving, it almost became a challenge. We moved our first things into the cloud. We still talked about having a datacenter and having some things, but it almost became a challenge: “How much more could we put there?” And, I will say now that we are fully in the cloud, 100%. We have our desktops, we have all of our servers, domain controllers are out in the Azure cloud. And one thing you find is it changes your business model with your business.

I don’t … At my last job, I remember happening, one year, I could not do major projects because we had replaced all our SAN. And, I sat in front of a board member. I had to say, “I’m going to spend the year doing SAN into the datacenter.” And, the question I got was, “It seems silly you have SAN in your datacenter. Can you explain, and why it takes so long to replace?” You don’t have to have those board discussions anymore. So much of this is gone. We can focus on the applications. We can focus on the change-the-business stuff and not worry about the run-the-business things. So it really changes my relationship with the business and your, if you will, your corporate items that you have to worry about by being so into the cloud. I don’t have a datacenter to worry about.

Even when I talk disaster recovery with people. They realize the way we’ve done this with mobility, we can work anywhere. I think a larger thing we learned about was architecting in the cloud. A lot of people piecemeal the move into the cloud. They move test. Then they move disaster recovery. When you architect it front to back to be fully cloud-based, you don’t do it that way. You build the redundancy into your region. You build redundancy into another region, and it all just sort of works together.

Michael Krigsman: We have a really interesting question from Wayne Anderson on Twitter. And, he says, "With a spinoff in a cloud-capable world, what did you not do that weighs other companies down?" To me, that addresses two points. One is the greenfield aspect, and the second is what you were just talking about. You know, you don't have to talk with the board about disaster recovery because it's sort of just baked in.

Mary Cecola: Yeah. I think we’re not as weighed down by infrastructure. The speed we can bring a  region up with we can change very quickly. Now we're rolling out all our applications at the same time we're doing the network, and suddenly they need another region for something. That could be built in a day. So, that's one thing that didn't weigh us down. I would also say, as we built this, the second person I hired was my information security officer. We layered security all the way through our implementation as we knew we had to do, being in the cloud. We are tech-forward with all the cloud we built, but  are also security-forward with all the new tools that you can do there. That is another thing I think weighs other people down.

On top of it, Antares … Right now, his position is in such a good place. We have extremely modern technology front-to-back. So, when we look to add something, or when a new idea comes up, we’re not beholden to legacy technology. I mean, we could have kept some of these old systems and just migrated them over, but we’re not beholden to that anymore. So, somebody wants to put in something like Cortana so they can ask our data repository something. We’re built to do that.

Michael Krigsman: What is that like? You came from very large organizations with very well-established systems, I assume, both in the private sector and in the public sector; and this is entirely different. So, what's that like to be CIO in this kind of very, very different environment?

Mary Cecola: Yeah, it’s really exciting. I mean, 1) Antares is an exciting place to work for. One of the guys I hired recently said, “I love this place because you can really get things done!” So, 1) We have an environment where we're moving quickly. We have a team that's really committed to being technology innovators. But, also, one reason I'm so committed to being technology-forward and making sure we didn't repeat the sins of the past is when you're in a really large organization, just keeping your desktops up to date, staying in the most current version of Office can be so difficult! And, the way that we’ve done this with 365, we’re on the latest version all the time. We receive the updates. We don’t have those kinds of problems.

So, I think having been in large organizations that struggled with that legacy-technology difficulty interfacing systems, we really work to eliminate all those problems, that we built this. And it’s exciting! I will tell you that. It doesn’t mean we didn’t find new challenges along the way, so…

Michael Krigsman: So, of course, I have to ask: What are some of the challenges that you discovered as you were making this, I was going to say, “cloud-centric, cloud-first migration.” But it wasn’t a migration. It was a construction.

Mary Cecola: It was. And, our timeline was so small coming out of GE, you know? I'm meeting these… Basically, it [the timeline] was fourteen months when I was hired. By the time staff was in, you had twelve months to really build every layer of your technology and go live. And we did meet our date on that. I think that it was a really interesting challenge, and I think being… You could say it’s “cloud first” but we did build the entire thing in the cloud.

Some of the challenges we had with that, though, is tools you were familiar with, or maybe companies you were familiar with dealing with, they weren't ready. A lot of people tell you they're prepared to be in the cloud, but you have to dig into that and get under the covers. Some of the tools that we wanted to use, and that were Microsoft tools, weren’t ready yet. So we had very honest conversations with Microsoft. We knew where they were on certain things, but we had to use other tools in the interim. And now, we're migrating over.

So, I think some of the challenges we’ve had, and you’ll find when you’re in the cloud, is that vendors you might have used before that you were comfortable with, you have to look at them again. Some people you know might not be as open-minded to it. And, you know what? We find a lot of fun and interesting problems, but having a team that likes to take those on and says, "No, let's not step back. Let's keep moving forward." That is one of the key things you're going to need to overcome challenges.

Michael Krigsman: We have some more questions from Twitter, but I want to follow up on something you were just talking about, which is you don't have to spend time and resources on things like updating desktops; updating the software. Does that free you to apply the resources to innovation or customer-facing activities? What do you get when you're freed up in that way?

Mary Cecola: Yeah, and that's all part of this having all of your infrastructures in the cloud or leveraging these things like 365. You're exactly right. These run the business activities that drag down your budget and your opportunities as a CIO. They're truly taking care of. And we see ourselves as an IT department that is business-forward. We want to be there; we want to be at the table asking strategic questions. We want to help solve business problems as they come along. We don't want to have it be, "Oh, you're looking at your product. We're too busy upgrading desktops to be a part of that." We want to be strategic partners with the business. And we've built an infrastructure. We've built a technology base that's going to let us do that.

Michael Krigsman: And how does this change your ... Let me ask the question this way: Has this changed the relationship between IT and the business? Is it different from other organizations with which you have experience because of this?

Mary Cecola: Absolutely. No, I think 1) The business is very excited about how tech-forward we are. They look at a lot of different deals, and some of them have technology software companies and hardware companies. They invite us into those discussions to ask us to look at that technology with them. I've never been a part of business before where IT was brought to the table on business decisions like that. We're looking at new products. We know they've gone through that. I think we've created … It's both because of technology that I've also hired and created a team that is business-forward as well. We have great technology, but we're not just focused on that. We want to be part of the Antares business and my team is very good about integrating themselves with Antares. And we are part of Antares.

Michael Krigsman: So, you think about IT. The term you just used is “business-forward.” That’s how you think about the relationship of IT to the business.

Mary Cecola: Absolutely! I think if you’re not, you’re going to be left behind. And, I never want to be in a place where they’re coming with new technology that we don’t know about. We want to be looking at technology, thinking about how to apply it, how to make Antares Capital a better place through technology, through automation, [and] digitize items that they're still trying to do manually. Our opportunity to go and help them; that's where I see us as “business-forward.”

Michael Krigsman: We have a couple more questions from Twitter; some really interesting questions, actually. One is from Arsalan Khan, and Arsalan is a regular listener so thank you, Arsalan. Arsalan asks how much business process re-engineering you had to do to move to the cloud, and what were the cultural impacts? And, I know you were a new company, so you’re setting processes up greenfield, but still, the folks inside Antares Capital are from established financial institutions. And, so how did you have to re-think about it?

Mary Cecola: Yeah, and we might have been a greenfield of technology. Remember our systems were being run on GE platforms until we migrated over. So, we did have a legacy platform we were moving from. We ran about 35 different applications at GE. We were able to consolidate that around five major applications. The technology was not as well-integrated. There was a lot of gaps in manual input on the other side; we've integrated that. So, we've got very clean data the way it flows through. We did major business process engineering through this. As well as building up an IT team, we had to build up the finance team. The operations team was brand new. So, you had new teams coming in, a lot of them with GE experience because we relied heavily on hiring from there.

But, you know, our opportunity to restructure this ─ look at the way the system should interact, and build … We’ll talk about the cloud, but really what we did with the application infrastructure was just as amazing, [and] really being data-centric with it. Where is the data owned? Where is it moving to? And we were reducing manual entry as much as we could. And so, we did a significant amount of business re-engineering.

Michael Krigsman: How did the folks in management, the senior management of Antares Capital, react when you said, "Okay. This is all going to be cloud," right? I mean, isn't this all kind of practically unheard of in financial services?

Mary Cecola: It is. And, you know, the first question everyone always has for you is, “It it safe? Is it safe to move to the cloud?" And, you know, my answer to that, first off, having an information security officer in-house initially helping build those safeguards and the different layers of security. I think at this point, we're far safer. I don't think I could ever secure a datacenter the way that Microsoft has the ability to do with all the things they have there ─ the ability to sort of break things up into different regions within the cloud. Again, it gives me a flexibility that would have taken me a lot to build.

You know, my answer to that is it depends on how you do cloud, and which cloud you pick. You have to choose a safe provider. You've mentioned Avanade. We've also selected a vendor who's very knowledgeable about it [who] could bring to the forefront the Microsoft resources we need to figure out all of these solutions, and get that relationship very tight. I think going through those security standards, how we would do it, and why it can actually be safer.

On-prem used to be safe because you could guard your datacenter like a castle if you think about it. You've got a drawbridge and one entry and exit. And we know over the last 10-15 years that has disappeared. You have mobile devices now coming forward and other things that are coming in through those walls. Your opportunity to build security to protect your key assets which are your data, and protect those over everything else. It gave us a greenfield of cybersecurity which I think a lot of cybersecurity solutions are bolted on because there’s so much change in there. We were able to layer on modern cybersecurity right into it. And, it helped to hire somebody who was in cybersecurity. But, also what I would say, he was as passionate about moving into this, and he could explain it in good terms to auditors and to other people who ask the questions.

Michael Krigsman: So, it’s really interesting. You definitely believe that you are safer with the cloud than with on-premise - than with controlling your own systems.

Mary Cecola: Yeah. I really do. And, we could have a whole other discussion on that and why that's true. I do think it depends on the cloud provider you choose and making sure you find one that has strong security standards front-to-back.

Michael Krigsman: I totally agree. As a matter of fact, I’m always surprised when I talk with CIOs of any but the largest companies in the world who say, “Well, we can protect our datacenter better than Microsoft or any of the large cloud providers.” I mean, it’s impossible.

Mary Cecola: Yeah, I think that’s true.

Michael Krigsman: That’s how I see it.

Mary Cecola: Yeah.

Michael Krigsman: We have another fascinating question from Twitter, from Wayne Anderson again. And Wayne asks fantastic questions. Thank you for that, Wayne. So, Wayne says, "When you hear another CIO say, ‘I'm in financial services, and my key focus is to control costs,' how do you react?"

Mary Cecola: So, I think that has been true historically and I spent 25 years at Deutsche Bank. I really do understand that. I think this cloud migration ─ freeing up your run-the-bank resources. Because the other thing that's also a pressure there is not just reducing a cost, but reducing your run-the-bank and bringing up your "change-the-bank, change-the-business." So, in financial services, I think that doing this type of thing, you could go … If you decided you wanted to go full cloud, pitch to your board, pitch to your senior management, “This will be the last major infrastructure project I'll ever come talk to you about.” Separate from yourself. Move your datacenter into the cloud, and at that point, I think you're going to see cost savings either from continual capital investment, or just the way you're running things in the cloud. But you'll also be able to give a lot more time and attention to the business, which I do know, in financial services, they are desperate for.

Michael Krigsman: How did you have the conversation and convince the board that cloud was the way to go? What were some of their concerns and how did you overcome those concerns?

Mary Cecola: You know, security is always a major concern. So again, it's really kind of going through cybersecurity items. The speed at which we moved, and our ability to change quickly as we rolled this out getting off our TSA was a big goal of the organization and explaining how migrating into the cloud would help with that. Then, also talking about this business case of going forward and not having to continue to do these types of investments. And also, putting us at the forefront that we can take new technology on extremely quickly with the infrastructure we have. Those were all really solid points.

Michael Krigsman: So then fundamentally, it was, correct me if I’m wrong, the ability to scale, the ability to be fast, and the ability to have greater responsiveness and participation with what the business wants and needs.

Mary Cecola: Yeah, you articulated that very well. But, that’s a big part of the business case, and their concern is always the safety. So, your ability to prove that you’ve built those cybersecurity things in, I think waylays the fears and provides the business to the board about why it’s a good idea.

Michael Krigsman: Can you give us some examples of how the resources freed up, or the focus or the mindset of going to the cloud enables you to participate more fully and be more responsive to the business?

Mary Cecola: Yeah, it's a good question. Now, remember I built this team. So, it's not as if suddenly resources were more available. I will say my infrastructure team right now is about four people, which I think is tremendous. And we have some people who help with the desks and going around. So, [a] four-person infrastructure team, with $18 billion dollar business, I mean, those are pretty good numbers. That said, we do leverage Avanade heavily to help us with that; and a lot of the space, especially around Microsoft and SharePoint. But, we've got a managed service provider and that's the relationship we're going to have there. So again, we're not doing a lot of our management around things that don't add the most benefit to your business.

Now, as I say that, adding your networks up, having everything work, is very important. Downtime you can’t have there. But when that’s up and working and running, nobody in the organization tends to see that as you providing that to them, right? So working in a managed service model, reducing the costs with the cloud, and getting the majority of the focus on the applications, and changing those, and making those better for the business, that's really where you're going to see the savings. And even when I talk about four infrastructure people, two of them work almost full-time with the application development teams.

So, [it] really brought down the focus on my team for a lot of the infrastructure work, and then we focused back again on applications, data stores, producing new opportunities.

Michael Krigsman: Where does Agile fit in? And if you can talk about it in terms of not just project methodologies, but the mindset. Because I think it’s so central to that interactive and more responsive relationship that IT can have with the business.

Mary Cecola: Yeah, you’re absolutely right. Doing this entire project in about 12-14 months, we were obviously agile. And I think Agile goes, to your point, much deeper than just a way to run projects. It’s not just having a scrum meeting or a Kanban board. Agile is really a mindset. And, our ability to think agilely to get … Our networks rolled out six months into this. We talk about having 12-14 months. We had to have the networks up and running and all the offices converted on to the new technology by mid-summer / mid- to late-summer. And right after that, we rolled in our general ledger. That was a few months later. We put in the other major applications.

In order to do that, we had to apply an agile methodology. We could not pull back and do requirements for six months. That would never have worked. So we had very interactive meetings with the business, we brought them all together on a regular basis. We reprioritized things quickly with them; got delivery back out; showed it to them ─ had that sort of interaction with the business. I also think we deliver a better product, because, you know, when someone asks you what you want, you don't always have all the answers, right? And this agile methodology you can do ─ of then showing it and changing it. I'm not saying, "Well you gave me these requirements nine months ago. You have to stick with them."  [This method] also really helped us provide good solutions for the Antares Capital organization.

But Agile, I think, is more than a project methodology. It's a way to think. It's a way to approach problems so you're not sitting back and dithering for a long time. You're looking for solutions quickly. And then also, the ability to get changes out on the desktop. Or at one of our applications, we’re going to go on four-month sprints, delivering something every two weeks. For financial services, that’s fantastic.

Michael Krigsman: I want to remind everybody you’re watching Episode #226 of CxOTalk. We are speaking with Mary Cecola, who is the Chief Information Officer of Antares Capital. And, Avanade has underwritten this episode, and we are very grateful to Avanade. They are the largest service provider for the Microsoft platform, and I’ve worked with Avanade for a number of years now and it’s a really great company. So Avanade, thank you so much for underwriting this episode of CxOTalk.

Mary, you mentioned this agile mindset, and it’s funny. As you were talking, I was thinking to myself: Agile, in a way, is a kind of ongoing show-and-tell. Build and show, build and show, right? And get feedback on that. And you mentioned the term “mindset,” as distinct from project management methodology. So, what is that agile mindset?

Mary Cecola: It’s a good question. I think it’s … It is Show and Tell and going back and forth. But, it's also, again, thinking quickly on your feet; looking for solutions very fast. [It’s] keeping that delivery to the business not just about what they asked you for initially, but getting it on the desktop and making it what they want instead of what they might have originally thought.

Agile’s a mindset to me because what you're really trying to do is deliver something that makes the end-user’s life better. And, doing that means you have to make changes often. Sometimes, what you developed wasn’t all right, and you might have to throw half of it away. That’s okay if what you’re trying to deliver is what they want at the end of the day. So, I think Agile’s a mindset because if you keep that in the forefront, then you're going to stay agile, and you're going to stay delivering. Show. Tell. It isn't going to be right. Fix it, and that's a good thing.

Michael Krigsman: Okay. So now, I’m slightly confused.

Mary Cecola: Mhmm.

Michael Krigsman: Because, when you say you always want to be doing the right thing for the business, and giving them what they want, how is that different from IT before the cloud, before Agile? I mean, right? CIO and IT, haven't they always wanted to do the right thing for the business?

Mary Cecola: Maybe. And, I’m not going to say anyone didn’t, but you often get hung up on internal IT department processes and things that are important to you. And that often, in organizations, can create a gulf between the IT department who’s trying to do things a certain way especially when you have to keep a datacenter up. When you’re spending a whole year replacing your SAN. When you’re doing things that you want to say, “But this is important,” but at the end of the day, it doesn’t feel important to people who are trying to move the business forward.

So I think 1) That's where the cloud can add to this sort of, "Now we're focused both on the same important thing." Also, with traditional waterfall and collecting requirements and you sign off on it, I think often of developing systems [is] like building a house. When I say, “I want this sink,” or that it doesn’t turn out to be what you want at the end. You change your mind. You want different colors; you want different things. And, when you sign a requirements document and people disappear for six months, you deliver that [and] you say, "Wait, that's not really what I wanted." You often got again this gulf between, "Well that's what you asked us for." Or, you know, "This is what we've been working on. You signed off. You're done with it." Agile lets you throw that out.

And if you have the agile mindset that, look, everyone doesn’t get it right the first time. We’re going to show, we’re going to change, we’re going to do it, and deliver something that is what people want, but maybe not what they asked for. Does that make sense?

Michael Krigsman: Yeah, it makes sense. So, the agile mindset, then, forces you into a different kind of project management methodology.

Mary Cecola: Exactly. It really does. You still need the proper documentation ─ we’re all in a highly-regulated business. And following that, making it highly light and flexible ─ not making too much process ─ and focusing on the end-deliverable.

Michael Krigsman: What about change management in an Agile environment. How is that different from a traditional software rollout?

Mary Cecola: It’s a really good question, and what I’m going to tell you that you need to keep certain controls and processes in place. Change management is one of those that becomes more important. Things are changing quicker. You can build environments quicker in the cloud world, and make those changes. You can make application changes much quicker following structure, and making sure that you got good source control. Following a very structured approach to migrating that into production, having good back-out abilities, is more important when you’re putting change in quickly.

Michael Krigsman: It sounds like you spend a lot of your time – we’re going to say managing vendors – I’m not sure if that’s quite the right term, but selecting partners and working with partners and negotiating with partners. Is that a fair statement?

Mary Cecola: You know, that is fair. And, the fact that, again, around this fourteen months to create a team to implement all these items … We made some key hires along the way, but I also really leverage some key business partners ─ firms I’ve worked with before or I could really trust for the different things they do. And, I was using three different vendors. Avanade was a very key one. Again, right in that Microsoft space. And, working closely, we made an early decision that we were going to use managed service providers for parts of the IT department that we did not consider, I’m not going to say, “mission critical” ─ everything is mission critical ─ but it’s something that you can outsource and monitor instead of spending your precious hours on it. And so, we have some very good strategic vendors.

Michael Krigsman: Why did you make that decision to work with managed service providers? Why did you do that?

Mary Cecola: There are a couple of reasons. 1) We did have a speed issue. So, having providers come in and help us go very quickly as sort of surge consultants. But longer term, migrating certain things to managed service providers and being able to check service levels. It again takes the focus, the day-to-day focus, of a lot of your team in management away from these items that may not be as crucial at the end of the day having to build that entire group.

I also find sometimes, keeping an innovative mindset, keeping an Agile department, keeping it lean, keeping those types of people focused, and not creating a lot of groups or being able to make changes with those managed service providers when they’re not delivering the way you need it also makes managing your department a lot easier.

 Michael Krigsman: And what about the skills issue? To what extent were you thinking, “Well, if I go with these managed service partners, I don’t have to develop these skillsets in-house.” To what extent was that part of the thinking process?

Mary Cecola: Absolutely, and I think it’s a really good point. Either A) I was able to bring in managed service providers that knew a lot of things and could help the team I had on the ground with some of their problem-solving, but there are a lot of skillsets that we just went with ─ a new help desk. I didn't want to have to create one of those and figure how to train that up. We will leverage that and then the tool that they were using to do the problem management along the way. Those were items that we could really add that talent and that competency without that being something we had to grow in-house.

Michael Krigsman: We have another question from Twitter. Again, from Wayne Anderson. Wayne Anderson asks the questions that I should have thought about asking, but didn’t. So thank you, Wayne. And he says, “What was the single most important trait or factor to contribute to vendor trust for you?”

Mary Cecola: That is a really good question.

Michael Krigsman: Isn’t that? That’s a great question!

Mary Cecola: And my answer is “delivery.” I mean, well, a couple things. When I have a vendor who can come in and deliver on their promises, that’s crucial to me. A vendor who’s transparent. So, consequently, there’s a problem with what they’re working on, they let me know. They let others come in and help ─ also really important. The worst thing is for a vendor to go off, have an issue, and let you know later. Ones that are very strong in the competencies they have. And then their ability to move resources out when they don’t work… Sometimes, resources aren’t working for talent, and sometimes its cultural fit. The ability to say, “This isn’t happening and get those changes very quickly,” is important to me. They have to be trusted advisors with me. And in some ways, the lead of these managed service providers that I've used; those people are on my team. They meet with my team, and they feel like a part of our organization. So, we don't treat them like they are something different.  But, those traits: really being good at what they say they’re good at; having transparency into what they’re doing ─ when they’re having issues, and what’s going well; and then also the ability to work with us on resources is key. You earn that trust, don’t you?

Michael Krigsman: I think that the issue of … I’m so glad that Wayne asked that question, because the issue of trust, especially in the cloud, where you are putting your crown jewels in somebody else’s hands.

Mary Cecola: Yeah.

Michael Krigsman: It’s so important. And we have another interesting question; this time from Scott Weitzman. And Scott asks, “With the cloud environment, is your COE, your center of excellence, also held and built within the cloud?”

Mary Cecola: Umm, so, it depends on what our center of excellence is. I have heard that term used around different sort of functions within an organization. But we have no on-prem servers. We have none. So everything that we have built is in the Azure cloud environment.

Michael Krigsman: Wow. Now, I want to continue with this theme of the relationships with vendors. And, I know that you’ve been working very closely with Avanade, and so where does Avanade fit into this picture? And, you’ve kind of alluded to it, but where does Avanade fit?

Mary Cecola: Yeah, it’s a great question. One of the first people we had in was from Avanade. He helped us to build our internet site. He’s helped us to move all of our files. And, these are not data files. These would be files out of share drive at GE into SharePoint. And as we architected that working directly with different departments, I’m going to bet there are a few people at Antares Capital who don’t realize he’s not one of our employees.

They were key with the network rollout and we had a couple good Avanade partners who were on the floor. When we talk about this network rollout ─ five cities in seven weeks, and some of these offices ─ two hundred desks. I mean, just unloading the boxes sometimes. We changed every technology component on the desktops in that transition. Just getting the equipment on the boxes, setting it up on the desktop, hooking it to the network. I loved when I was in Chicago and half the people we had here to do other things … So, I’ll come in the week and I’ll help you with that.” So, we had people of all different backgrounds helping us basically set up desktops, and those were our Avanade partners with us.

So, we’ve leveraged them around the Microsoft Suite of SharePoint desktop. They help my ISO with the cybersecurity decisions and looking at that. They helped us with a lot of the early architecture around the cloud. I actually remember going to Avanade and talking about what we wanted to do. And, one of their key partners said, “You know, everybody else Mary is halfway down this marathon. You guys are standing at the end line saying, ‘Where do we go next?’” And it was a really good insight to why we were having trouble getting answers to certain questions because we were in a place other people weren't looking at. But, I appreciated [that] they've helped us look down that road and look forward, and try to find new solutions.

On top of that, they really helped build up our relationship with Microsoft, making sure we were talking to the right people.

Michael Krigsman: Any time you’re doing this kind of project that you’ve described, which is so encompassing and especially on a very tight timeframe, it’s always fraught with complexity. And, it’s hard enough to do that if everybody works for you inside your organization and reports to you. So, in that relationship with Avanade, how did you manage all of these pieces? And how did you ensure that everybody was working on the same page and moving forward in just the right way since you're dealing with multiple organizations?

Mary Cecola: Yeah, that’s a great question. I mean, I think project management and time-boxed projects, and hitting deliverables like that … I mean, we had teams: we had infrastructure team; we had desktop teams; we had every single application being developed at the same time; and in the background a whole separate team who is bringing the data over through GE, cleansing it, and deciding what areas it went into. We used very strong project management. I had brought in a key delivery manager that I had known very well in the past to help with that, and really those streams all reported up. We had regular meetings together.

To your point, people don’t always want to tell you about the problems they’re having. We had a very open door about that. And it was more. Again, I bring up this transparency because what you’re talking about and the ability to do this type of work so quickly … If people don’t feel like a group can walk into your office and go, “Mary, we’ve got a problem!” You’re not going to get through it because these problems are going to linger in the background.

We had a very open door on that, we talked about problems very openly. Problems were brought to the forefront, and this is where I’m going to get to the agile mindset. It wasn’t like, “Oh, I have a problem I had better hide it!” It’s “I got a problem, guys! Help me solve this so we can move on to the next thing.”

You know, Avanade brought their management around that. I think I mentioned their senior person is one of my team and a partner to us. I think having … It was both the managed services working with them, but also having this very open mindset of problems are things to discover and solve, and not to worry about and hide.

Michael Krigsman: We have just a few minutes left, and I think it would be really interesting if you can share advice or share your learnings about working in this kind of environment where you’re managing a mission … vendors working on mission-critical parts of your business. So, how do you develop that trust? How do you manage that relationship? How do you keep the complexity inherent in check so it works?

Mary Cecola: It’s a great question! I think having a shared vision is very important. We know our shared vision of moving to the cloud. We know our shared vision of being technology-forward. But, also seeing that whatever impact we have, we have to have the lowest impact on the Antares Capital business side that we can have. And everyone has those in the forefront of their minds as they move forward.

Having oversight and clear delivery, and understanding when things are missing, and what you’re going to do about it ... So, you need, I mean not a formal contract, but a clear contract with these vendors of what the mission is, how we're going to approach it, getting them to understand that vision but also your corporate culture and then having good controls and a good ability to monitor what’s being delivered and how it’s coming through. And, prioritization. Never forget prioritization.

Michael Krigsman: But you can’t write all, as you said, you can’t write all this down into a contract. And so, when you’re working with Avanade, just as an example, how did you make sure that that trust was there? I mean, what did you do? What advice can you offer other people who are staring into a huge project with external vendors, and they’re terrified?

Mary Cecola: That’s a complex question. I have done it a number of times. I think 1) Bringing them into your team and bringing them into your fold, and treating them as part of the project is very important. Don’t delegate to them that they’re going to do this, and [that] you’re not going to have any oversight. Staying very clear. Having those meetings with them. We would also have meetings between the technology departments, [and] the business areas we’re working with. We talked about issues openly. You have to be very actively involved. You can’t be hands-off.

But on top of that, I guess that’s where I talk about moving people out if they’re not working out. If you see someone who isn’t fitting the culture and the vision, having honest conversations and having a firm that will have those with you. And migrating people out when you see them not either being transparent or making decisions around technology that is taking us backwards. We ran into that with certain people.

I don’t ... I think the big answer there is, you can’t just delegate it. You still have to own that project. You still have to own that firm that’s doing it, and you know, build trust but verify.

Michael Krigsman: And obviously, it worked. And then, my final question to you in our last one minute is what advice have you got for other CIOs who are trying to convince their management that cloud really is okay; we can do it?

Mary Cecola: Yeah. I think my first advice is “Don't appear to own it.” If you're going to go into it, move an entire application, and architect for it. If you're looking to explain to people and they have questions and they say, well, "Is the cloud safe?" or "Is the cloud this?" Changing that argument and saying, "What controls do you need to see? What items do you consider safe? What are you holding this up to?" Because if you just say, “Is the cloud there?” That’s not the answer. The answer is “What controls are you looking for? What way would you consider something safe?” And being able to answer those questions.

So, sort of changing it from, “Oh, it’s the cloud. It’s big and scary.” Picking good vendors is very important, and showing that you did that well ─ that you’ve checked that they are secure. And then saying, “What kinds of things do you need to see to say any environment’s secure?” I mean, you and I challenged earlier, I don’t think on-premise datacenters are that secure. So what makes comfortable that it is secure, and how can I show you that that’s true about where I built mine? I think that’s very important.

Michael Krigsman: So always have as your reference point the business value, the needs of the business and not get hung up, as you said earlier, on IT processes.

Mary Cecola: Yeah. Exactly. That’s very important.

Michael Krigsman: Okay. Well, we have been talking for about 45 minutes and unfortunately, our time is up. I wish we had a lot more time.

You have been watching Episode #226 of CxOTalk, and a big thank you to our underwriter Avanade, and a really big thank you to Mary Cecola, who is the Chief Information Officer of Antares Capital for being here and sharing your experience, and your wisdom with us. Mary, thank you so much!

Mary Cecola: Thank you, Michael!

Michael Krigsman: Everybody, come back next week. You can go see our upcoming episodes on CxOTalk.com/episodes. And, for sure, you should subscribe to our YouTube channel by clicking the YouTube button that’s on your screen. Thanks so much, everybody. Have a great day. Bye-bye!

Building AI: A Software Maker's Perspective

Sean	Chou, CEO and Co-Founder, Catalytic
Sean Chou
CEO and Co-Founder
Ed Sim, Founding Partner, Boldstart
Ed Sim
Founding Partner
Boldstart Ventures
Keith Brisson, CEO and Co-Founder, Init.ai
Keith Brisson
CEO and Co-Founder
Michael Krigsman, Founder, CXOTalk
Michael Krigsman
Industry Analyst

Artificial Intelligence is surrounded by marketing hype, making it difficult to assess what's real and useful. In this episode, we talk with a venture capital investor and two software entrepreneurs to learn what's involved with creating products that rely on artificial intelligence and machine learning. Join us as we cut through the hype of AI.

Ed Sim is the Founding Partner of Boldstart Ventures. He was an early believer in SaaS and led first round investments in market leaders like LivePerson (Nasdaq: LPSN) and GoToMeeting (acq. by Citrix). Over 19 yrs, he has led many seed and first rounds and helped a number of entrepreneurs successfully scale from seed to market leader.

Sean Chou is the CEO of Catalytic, which makes the Pushbot platform. Prior to Catalytic, Chou was the Chief Technology Officer and EVP of Services at Fieldglass. Chou was responsible for the overall development and delivery of Fieldglass solution and services, oversaw the product development, hosting, professional services and marketing departments. He provided the strategy and vision for Fieldglass’ award-winning cloud solution since inception.

Keith Brisson is CEO and co-founder of Init.ai, a venture-backed developer platform that enables companies to create conversational apps. Keith is a software engineer and has extensive experience building conversational applications using modern machine learning techniques. He focuses on and writes frequently about the end-user experience of conversational applications, the machine learning and technology that powers them, and the future of human-computer interaction.

Download Podcast

Building AI: A Software Maker's Perspective

Michael Krigsman: Welcome to Episode #222 of CxOTalk. I’m Michael Krigsman, and we have a really interesting show. We’re going to be talking about artificial intelligence, machine learning, and natural language processing. And, we’re going to be taking a development perspective, from the point of view of people who are actually creating products. And we have with us today two company CEOs, as well as a venture capitalist and investor. So, let’s dive in and Ed Sim, you are Guest Number One, or at least, I am introducing you first. How are you, thanks for being here, and tell us about yourself.

Ed Sim: I'm good! Thanks for having me! I really love your show, and I'm glad to be here with two of the founders in the Boldstart fund. So just real quickly, I'm a founder and partner of Boldstart Ventures. We like to be the first check in enterprise founders with respect to AI and ML, etc. We go for the opportunity to look for AI and ML companies. We look for enterprise businesses solving big problems, and if they happen to use AI or ML, like Keith and Sean are doing, then that's even more exciting for us. So, looking forward to chatting about this.

Michael Krigsman: Fantastic! And, you have been an enterprise investor for a long time.

Ed Sim: Yes, yes. Over twenty years done stuff in the back-end with the SaaS side, and companies like Greenplum, GoToMeeting, LivePerson, along with Init.ai and Catalytic.

Michael Krigsman: Fantastic. Our second guest is Keith Brisson, who is a founder of the company Init.ai. Hey Keith, how are you?

Keith Brisson: Excellent, Michael! Thanks for having me!

Michael Krigsman: So Keith, tell us about Init.ai.

Keith Brisson: Sure. Init.ai provides technologies that help enterprises converse with consumers at scale. So, we provide language understanding technology that helps them automate customer conversations, assist sales, and support agents in live chat, and analyze conversations that take place between their users and their agents. These can be external-facing conversations. With consumers, it can be internal as well, allowing employees to access information from systems of records, CRMs, clients, etc.

Michael Krigsman: So, essentially, would it be accurate to say you’re building a natural language processing/AI - hate that term, AI …

Keith Brisson: [Laughter]

Michael Krigsman: … toolkit that enables large companies to build their own products?

Keith Brisson: Exactly! So, we’re providing the technology that lets it unlock the data and the information within those conversations so they can incorporate them into their workflows, whether it’s communicating with their customers, or making their internal processes more efficient, it’s our belief that we should be the ones to take that advanced technology and bring it to them rather than them needing to do it internally.

Michael Krigsman: Fantastic. Well, I’m looking forward to diving in. And, Sean Chou, you are the third member of our behind-the-scenes in AI panel, and you are the founder and CEO of Catalytic.

Sean Chou: Yup. Thank you for having me on. So, Catalytic: We create a product called “Pushbot,” and our customers using our product can quickly and simply create these process bots that leverage work orchestration, automation, and AI; which I agree with you by the way, as a term, it’s definitely a little bit irritating, but now I think it’s popular and it captures a whole broad range of technology which I’m going to talk a little bit more about. But, with our product, our customers really focus on operational efficiency, reducing the number of dropped balls they might have, and tying together a lot of the people and the different systems within their company.

Michael Krigsman: So, you mentioned the term, "process-bot." What is a process-bot?

Sean Chou: Yeah. As entrepreneurs, we feel obligated to make up terms.

Michael Krigsman: [Laughter]

Sean Chou: [...]

Michael Krigsman: Now that's good, and large companies do the same thing. But seriously, when you say "process-bot," tell us what you mean by that?

Sean Chou: Uhh, so, I say mostly to separate us out from, let’s say, chatbots, which are a really popular concept. We do have conversational interface aspect[s] to our platform, so in that regard, it’s kind of got some chatbots. But, that’s not really the main purpose of what we do. The main purpose of what we do is really; and the main purpose of what Pushbot does, as a bot; is to push and promote a process. So, we like to say, “Processbot,” most[ly] in a way to kind of constrain down what we’re doing.

Michael Krigsman: I think a good place to kick off this discussion is all three of you are very actively involved in the consideration of different types of AI and what does it mean. And, Sean, maybe you can begin by helping us understand what do we mean by the term, “AI,” and what does it actually encompass?

Sean Chou: Yeah. For sure. It certainly covers a lot of different things, and I think with all major new technologies, there’s always this retrospective period where people look back a little bit and say, “Hey. This really looks like it should be under this umbrella,” and you get a lot of repackaging of things that once maybe weren’t part of AI, but now, because it’s the hot, buzzy topic, now get rebranded under AI. But, I think generally, when we think about AI, we think about it in three different categories.

There's really a strong AI, which is to try to create basically machines that are able to think in a general sense, in the same way that you and I are able to think. So, "strong" or "general AI," there are only a handful of companies that really should be considering that. You need a ton of resources; it's the Google, Microsoft, Amazon, you know, of the world that are going to sort of win in that type of space.

The second category is really more "weak AI," or "narrow AI." And, that's not as difficult. It's still extremely hard, but now what you've done is you've set instead of a general, thinking machine, we're going to focus on a specific domain or a specific field. And so, you see a lot of that in virtual assistants like Siri or maybe Ingram, or Clara, you know; these are folks who are saying, "We're going to create AI," and its personality oftentimes, but it's only going to solve a very narrow set of problems.

And then the third category, which I believe Init.ai and I both fall into, which is: The users of the technology and the research has come out of all this primary research on AI. So, we are beneficiaries of the research that’s gone into natural language processes, sentiment analysis, machine learning; all the things that kind of power AI, we take them, we repackage it, we make it, at least in catalytic space, we make it available for the average business so that they’re able to use it in their processes. And then we’re using it for our product in a very, very applied setting. So, you know, it’s not machine learning to be able to act as humans, machine learning will figure out how to improve their business processes.

Ed Sim: Yeah, and Sean, I think that’s a great point. Kind of how we look at the world is applied AI. I mean, AI is such a buzzy word these days. Everyone has AI in their business plan, the kind of time that everyone had “.com” back in the day. And the reality of it is, what business problem are you solving? And applied AI is very exciting because you’re not going to out-Google Google in AI learning, or Facebook with that AI, but how do you leverage best what’s out there and then apply to enterprise data? That’s the data that they don’t have and can get, and that’s what I love about what you guys are doing and what other companies are doing as well is working on that private enterprise [...], and learning from that.

Michael Krigsman: What are some of the really interesting use cases for this type of applied AI that any of the three of you are seeing?

Keith Brisson: I can give a few examples. You know, I mentioned we're focused on making customer conversations more efficient. You know, right now, companies need to hire teams of agents to serve their consumers and support in sales areas. And, it's our belief that we can take technology that happens to be using machine learning to make that process more efficient by offloading some of the routing portions of that to the computer so that the agent doesn't have to go through those routine processes themselves, you know.

So that is, for us, is helping displace the actions that people would not take otherwise. And, kind of a more broad scope, machine learning is incredibly powerful in finding patterns in data. So, you see it in all areas of enterprise uses where you're trying to detect signals in large volumes of data that matter to the business.

In finance, it has been used for fraud detection for ages, where you’re trying to find something that really matters to the business in this massive set of data. And, that’s just going to expand where you have more and more data coming in from different sources. You need to figure out how to extract real value from that unstructured data, make your business more efficient.

Sean Chou: Yeah, for sure. You know, it’s interesting, and I don’t think it’s a coincidence that AI is the trend right after kind of data science. Like, remember … This or a year ago, we’d all be saying, “data science.” But, I think there’s a reason for that. It’s because data science ended up producing so much data and kind of cheap storage, cheap processing power; just created so much data, that data science came in and said, “How do we make sense of this data?” But, we’ve actually reached a point where people have thrown up their hands and said, “We can’t make sense of this data,” and so we really need other things to make sense of this data, and those other things ends up being AI and that actually ends up being a great initial use case for AI, which is definitely one of the areas in which … I think also IT.

The notion of AI being used for human augmentation, as opposed to trying to replace humans, is, I think, still the right current curve for AI. I mean, we both … It’s entirely human augmentation: How do we take processes that have people and make people all more effective?

Ed Sim: Yeah. That’s a great, great point. I mean, I invested in GreenPlum back in the day solving the big data problem. Now, it’s part of the EMC and Pivotal. And, we coined the term, “smart data” five or six years ago, meaning you’ve got all this data … I mean, there’s three certainties in life. It’s taxes, death, and the growth of data. [Laughter]

So, when you go back to the applied AI and learning, that's kind of the next phase with that and the GPUs, and great neural network models, and cloud. I think it makes it more accessible to all of us from that perspective. So, when you look at any manual in the process, you know, including security: We invested in a company called Security Scorecard that allows you to look at the security posture of a company from the outside in. Using tons of data, we give classification, clustering, we give use regression analysis, and all that stuff to come up with scores on security. So, it can be anything from internal data entry, business process, but also externally you can use it for security as well. So, it's a pretty broad field.

Michael Krigsman: So, there are so many different areas and applications where patterns can emerge and technology can help discern those patterns. From a businessperson’s standpoint, how can they go about figuring for their own business, and their own processes, where it makes the most sense to apply these types of machine-intelligent technologies?

Sean Chou: Yeah. I think this is an area where, “You look really young, but I’m not going to make a call on your age.” But, certainly like Ed and I, being old guys in technology, I think we would say if you’re looking at this from a business user perspective, AI is just a shiny object, you know? If you’re looking at how to assess AI companies, I mean, stay focused on the business problem and realize that AI is a solution. I mean, when we set out to make Catalytic, we didn’t say, “Hey, we’re going to create an AI company.” We said, “Hey, we think the problem is that these processes are horrible, awful, and in spite of the technological advances that we have, they’re still horrible and awful, and we need to do something about that.” And sometimes, AI is the right answer. And sometimes, it’s not. So, we have AI for some things, but it’s not; certainly our solution doesn’t revolve around or hang its hat entirely around AI.

And I would say for the businessperson, they should have those things - that same mindset: What’s the problem; what value is going to be created; AI is just, you know, a new big, shiny tool that is hard to ignore.

The one caveat I’d put to that is, AI is … It opens the door to new category or problems that previously would have been almost intractable feeling. Just like trains and automobiles in the industrial age, now allow for low-cost nationwide distribution; AI is going to open up a new category of problems that we’re able to tackle that previously were just like, “We just can’t tackle those issues; [they’re] you know, unaddressable.” So, I think you’ll see a lot of those.

Michael Krigsman: But there’s a point at which I am now confused. So, on the one hand, you mention that AI is just a shiny object. And by the way, in doing that, you have just taken the wind out of the sails of half the technology marketing departments in the country, [and] in the world, okay?

Ed Sim & Keith Brisson: [Laughter]

Sean Chou: I’m going to be getting a lot hate on Twitter; a lot of hate tweets.

Michael Krigsman: But at the same time, you said that AI creates a whole new set of problems that can be solved or types of solutions to problems. And so, how do you reconcile the fact that on the one hand, it's just technology, and on the other hand, the implications are so profound?

Sean Chou: So, if you have just an AI solution out there, and it’s not solving anything, it’s just a shiny object that has no meaning. But, if there’s a new problem that you’re now able to solve as a result of AI, then to me, it’s kind of like what it does is it broadens the scope of types of things that technology can solve. So, in that sense, it is remarkable. But, at the end of the day, it still has to be solving the business problem; and I think one of the big dangers, whenever you have [...] things, is that people just start saying, “We’re an AI company,” but they ignore the business problem that they need to tackle. So, I guess I’m trying to speak out of both sides of my mouth here, but I think there’s truth to this; because on one hand, AI does now allow for solving things that we just would not have thought worth solving; when in fact, going back to what I was saying about it being a response to big data - like, big data: designers felt like we had reached a point where we just didn’t even know what to do with all of this data. So AI actually opens the door to being able to really significantly, meaningfully address terabytes of data that are coming in on a real-time basis. But, you know if you’re not doing that for a purpose, it’s pointless.


Michael Krigsman: So Keith, how do you, with your company and your customers, [...] go beyond the AI-as-a-shiny-object into AI that’s something that’s really useful in a practical sense?

Keith Brisson: Yes. So, what we’re 100% in agreement with [regarding] what Sean said [is] that ultimately, our goal as technology providers should not be providing technology. It should be to solve the business use-case. So, there happens to be the letters AI in our name, Init.ai, but we don’t generally talk about the technology. That’s kind of what we advertise when we’re talking to companies. We talk about the use-cases that we can help them be more efficient in, in the problems we can help them solve.

And, to Sean’s point, I kind of cringe when I see other companies really advertising the technology, focusing on the fact that they’re an AI company because it tends to mean that they’re not focused on solving your actual problems. So, what we always strive to message is how can we make your employees more efficient; how can we increase your revenue, decrease your cost, not what kind of neural network we’re deploying into our system; because that’s not really what matters at the end of the day. So, you know, for us, yeah we’re an AI-powered company, but we’re not an AI company. We’re solving business problems. So, does that answer your question?

Michael Krigsman: Yeah, and I think Ed, and certainly from what you were saying earlier, when you invest, you’re also investing in companies that are solving business problems, although they may be using AI to produce those solutions in a better way.

Ed Sim: Absolutely. And, AI’s like not a panacea in the sense that, you know, robots today or AI is not going to replace every human, right? So, the question is … You know, I’ve been very interested in human augmentation using AI so you do things that are better, faster and more efficient. I think that’s a tough sales proposition; the company comes in and says, “I’m going to replace everyone.” The question then is, you automate things, you do things 5x faster, connect faster, what else can those employees do? And that, to me, is more exciting so …

If I were a business, we talk to a lot of CIOs, I ask them, “Find out two or three pressing problems. What’s a low-hanging fruit inside of that business that can be partially automated?” Some of that can be automated data entry, right? Tons of people are doing that. “Look at anything that’s put offshore.” All that’s stuff that’s put offshore: you remember that big trend back in the day? A lot of that’s going to be replaced with AI, you know? And so, we think about that customer service, right? That’s another area that’s ripe for disruption as well. Accounting, AR and AP can gather that data, load it up, and try to automate.

So what we want at the end of the day is … I don’t care about the AI. If I’m a SaaS company or a software company, I’m going to look at the dashboard that the end-user uses, and “Give me answers.” Don’t give me, “Invest in beautiful data and visual screens.” I want to go to that screen and say, “Hey. These are the three companies you need to call today to see if you get paid.” And that’s what I want. I want answers. So, that’s what I want AI to do for me, not give me tons of dashboards, but give me an answer; make sense of that data, and give me something that I can really trust.

Keith Brisson: One thing to keep in mind, Michael, is that we’re part of this. You know, people are using the word “AI” as though the strong AI that Sean explained at the start exists today. The fact is it doesn’t, you know? We don’t have computers thinking like humans today, and we aren’t going to in the near-term, so we’re not going to be able to replace one-for-one a human with a computer. So what we need to do is take this technique, and apply them to specific verticals; to specific use-cases within them where we can see real value generated today.

So, AI as a term, I think that many people think about it aspirationally for where it’s going, and it is going there. And companies like ours and Sean’s are really going to help get us there in conjunction with the big guys where we’re doing some pure research. But today, we can take pieces of that; take pieces of what is going to get us to that point in the future and solve real business cases today. So, there’s a little bit of conflation of terms in this. You know, it’s a real buzzword, but there’s value there in the pieces and how they can be applied.

Michael Krigsman: We have an interesting question from Scott Weitzman on Twitter, who is asking, “What are the key elements that you can use to classify or differentiate AI on the spectrum from technology to business, in order to help businesses understand the nature of the kinds of problems that AI can solve?” In other words, how do you explain it well to businesses that they get?

Keith Brisson: You know, I think that we’ve certainly faced this challenge when we’re describing how we augment or replace humans in customer support, and you know, we’re very clear with companies that we’re not creating something that’s inimical to humans. What we’re doing is we’re making the business process more efficient, and really focusing on those particular segments where we can apply the technology. And I think it’s a matter of protecting the messages. So, if a company comes to you, and they’re telling you about the AI technology and not telling you how they’re going to help your business, that should raise a little red flag; raise an eyebrow for you and you look in a little bit more deeply.

So, I’d say when you’re trying to evaluate these solution, really consider what business process it’s trying to solve, and the ones that are appropriate for that are ones where there is a fair amount of data that is unstructured and it needs understanding around that. It could be language data; it could be financial data; it could be anything else.

Ed Sim: Yeah. I find it comes into two camps in enterprises, and just depends on the use case. It comes down to, "I can help you be smarter," i.e. there are predictive natures of AI, so whether there's a marketing dashboard, or just helping make decisions faster, so that's one aspect of a pitch. And then you can backfill that with whatever problem you're solving. And the other is, "I help you save a lot of time and money that you can free up your resources [with]." There are two different kinds of approaches. Predictive versus the general saving money aspect, and that's kind of the business problems of how AI helps you.

Michael Krigsman: Or to put it into broader terms, you have the efficiency aspects, and say, the innovation aspect?

Ed Sim: Sure.

Keith Brisson: Yeah.

Michael Krigsman: Let’s change gears here, and talk about when you’re thinking of designing your products, developing your products: Is it different when you start incorporating AI-related technologies from traditional software development? What are the differences? How do you think about it differently, design products differently, skills, and so forth?

Sean Chou: I think there is a difference, but it’s largely in the newness of AI technologies. If I were to replace the word “AI” and just say “database,” for example, when I think about a product, there’s no challenge in where a database lies, architecture, or solutions out of our product or the features that it can enable. AI, I can think of in the same way as a component that I can leverage, but it’s new. So, it’s kind of unlocking a set of features and a set of capabilities that we haven’t … like, patterns just aren’t as well established.

I think some are, and that's why among a lot of AI companies, there's a common set of things that are very common. Like, you'll see sentiment analysis, you'll see categorical processing a lot, you'll see machine learning a lot; these are patterns that have emerged as output, and so those are kind of a little bit - I don't want to say a "no brainer," but established patterns that product architects really use to leverage, whether they're engineering or thinking about a product management perspective. But, there are a lot more use cases that we haven't really quite figured out yet because of the newness of the technology.

Michael Krigsman: How …

Keith Brisson: [...]

Michael Krigsman: Oh, oh please. Please, go ahead, Keith.

Keith Brisson: Yeah. I want to just mention that it definitely requires a little bit of thinking, at least in terms of the implementation level, because there’s generally the requirement for data. If we’re using a solution provider, they can offer that data up-front to help you get started. It may not be a consideration, but data plays a role in any kind of application of AI and machine learning today, and if you talk with a solutions provider, that is going to be a concern, like how do you use data to make this thing more efficient?

And the other thing to consider is that these AI systems and machine learning systems, they evolve over time based on back-cycles, or at least they should. So, you can get an immediacy. The value that you get from these kinds of solutions up-front tends to increase over time, as they see more cases in their then-working back-cycles. So, it's a little bit different in that it's not static; it's something that's evolving over time.

Ed Sim: Michael, can I add one other piece? I think Keith makes a good point from the data itself, and if you're selling in enterprises, any type of solution that leverages their data I think a big question no matter how many folks talk about the cloud is, "Can you drop it on-prem?" And, I don't know if either you've been asked that question but, it seems to me that every company that we have leverages some type of data, and to run kind of AI on top of it is, "Can you drop it?" I can move the data, there are all these regulations, and I need the securest you guys can [provide].

Keith Brisson: Well, we get that all the time, especially since a lot of the innovation in AI has come from large companies that have a tendency to suck up data, you know, the Googles of the world, which most enterprises are not willing to trust with their valuable customer data. So, we get that all the time, but data is a requirement, right? So, our [plan] at least, is to provide our own data to help get things started. So, like in our case, it’s a general language understanding data, but then let companies maintain control over the data that’s specific to them. Our strategy on that is to offer on-premise or virtual cloud insulation so they can maintain control over that. The simple fact is most companies don’t want their data training other companies’ datasets.

Ed Sim: Yup.

Keith Brisson: … especially in regulated industries like healthcare and finance.

Sean Chou: Yeah. Data is definitely kind of the muck-side of AI that people like to not talk as much about, but it is really important, I think, for any company that's really looking at AI and what their AI strategy is going to be over time, which, I think, you know, we all are in agreement there should not be an AI strategy, there should be a business strategy powered and enabled by AI tech. But, you know, you have to get your data policy in order, so they definitely have to have a point of view on where data resides, how to own, and so-forth. And I don't think there's a right answer right now. Certainly, at that point, we get the questions to of, "Hey, where exactly is this data? Where does it reside?" It becomes a bigger problem once you start talking about a global context because then it's no longer just US privacy issues, you have entire governments that have a very strong perspective on where physically data should reside.

Michael Krigsman: Let me …

Ed Sim: We are a company for that, too. [Laughter]

Michael Krigsman: [Laughter]

Ed Sim: That’s probably the big idea, and we do use AI and machine learning to tell you where the data is, where it’s located, and we attach it to a unique ID before people kind of run scenarios on top of that, so…

Sean Chou: Got all your bases covered.

Ed Sim: [Laughter]

Michael Krigsman: That is clearly looking at all the different facets of this in order to cover, as you say, cover his bases. This issue of the data is so important, and would it be accurate or inaccurate to say that for companies that are thinking about AI, that it’s actually a data situation, rather than an AI technology situation or question? Is that accurate or inaccurate?

Ed Sim: I would say it’s pretty accurate, right? Because a couple things: I’m so bullish on the enterprise and leveraging machine learning in the enterprise, because once again, a lot of these folks don’t want to give their data to Google to train them, right? That’s Google’s model. Hey, I’ll give it to you for free, I’ll open-source it and [...] Facebook with that AI, but there is tons of value sitting inside of enterprises, right? And the more data you have, the better you can train, the better you can predict. So the question really is, I’m dropping something on-prem, and I’ve got healthcare data. That’s interesting. But somehow, I can tie that healthcare data anonymously with other healthcare data from other companies. Then you start creating a powerful data co-op and you can make better data predictions. But I think data is definitely a weapon, and it’s definitely a weapon in the enterprise space.

Keith Brisson: That’s definitely a weapon, but it’s not even all that you need. The technology and the skills required to process that data and turn it into something actionable are pretty technical and can be a distraction for most companies. So, for most companies, you can have the data. We also need the expertise to be able to transform it into something that’s actually usable within your business. And that’s something that tends to not make sense for companies to build themselves. I think that’s at least our premise is that we should provide that technology so that businesses can focus.

Michael Krigsman: That’s a really interesting … Oh, please, go ahead, Sean.

Sean Chou: I was just going to add, I think that the data is certainly essential for the majority of AI applications right now, and it's largely a function of where we are. When we narrow AI for machine learning, for developing neural networks, you need data. You need feedback to improve that neural network. But, as we start getting more and closer to more general AI, or even very well understood things like, you don't really need to train sediment analysis that large now because that's a more general application. So, you're already seeing some applications and outputs from general AI research that doesn't really need to be trained.

So, I think that that shift will occur over the next decade or two, where we right now need a ton of data, especially for domain-specific applications, and we're going to see more and more "general" general AI come out that won't need the training as much, or won't need as much data power.

Michael Krigsman: This whole …

Keith Brisson: I actually absolutely agree with that. Data is almost a crutch right now because the techniques around general AI have yet to be developed. So data is replacing the sophisticated techniques that are on the rise.

Michael Krigsman: It seems like it's quite a paradigm shift to some degree for people in the enterprise who are thinking about AI because they are tending to think through the lens of the technology, rather than - because it's so new - rather than necessarily thinking first and foremost about the business problem. And then the fact that so much of AI involves patterns and therefore requires that large volume of data, it requires a different way of thinking about how you go about solving problems. So there is an education process, I think, that is still having to take place, and the result of this is all of the hype in the software and technology industry because you can basically say whatever you want about AI. Everybody's got an AI, right? I've got an AI! [Laughter]

So, I was recently talking on this show with James Cham, who is with Bloomberg Beta, and he was saying we need a framework for making decisions inside the framework of the enterprise; investment decisions. So, Ed, let me toss it back to you. When you think about investment decisions that involve AI, what are some of the criteria and things that you think about?

Ed Sim: Well, I don’t try to make a specific technology decision on AI investing. I mean, first and foremost, we tend to be investing in technically-driven founders like Keith and Sean. So for us, it's deep domain expertise and understanding of the business problem they're solving. If they didn't come to me and say, "We're building an AI company," right, and they said, "We're solving this problem; it's a big, big opportunity," … And by the way, one of our core pieces of technology will be some use of AI, and then we fit it in through that a little bit. But that was kind of the main reason we fund it. So I think that's really, really important. It's just like every other tech trend out there. Back in the day, it was Java. You know, there were Java funds out there. Then it was everything with mobile. And now, it's AI.

But guess what? I view AI as just like water. I mean, it’s just like electricity. Every company in the next five years, every technology company is going to use some form of AI. And so, I don’t view that as separate [...] per-say. That’s why I say “applied AI.” What business problem you’re solving, and how you’re doing it 10x faster and 10x cheaper; if you do both, that’s an amazing order of magnitude improvement.

Sean Chou: Totally. Like is there a web company today, you know?

Ed Sim & Keith Brisson: [Laughter]

Sean Chou: But there were! I mean, ten years ago, or fifteen years ago, time flies. You know, fifteen, seventeen years ago, there were web companies. But there are no web companies today.

Michael Krigsman: Well, if you think about it, when Salesforce started, they said, “We’re doing this Salesforce automation over the web.

Sean Chou: Yup.

Michael Krigsman: … because it was new, and it was unique. And so, your feeling is that given some certain amount of time going forward, AI techniques will be incorporated just about everywhere, essentially?

Sean Chou: Yeah. I'm 100% convinced of that. And we're talking a lot about machine learning and human augmentation, but there's just a lot of simpler things like AI … Applying a lot of AI principles just makes a better product, because you are asking a person to do as it's thinking because the product can do more on behalf of the person. So, you're going to see a lot of very subtle importance about AI, because it's just better product. It just lowers the cognitive computing costs on the human's side.

Ed Sim: I love that Sean. I’m thinking that great AI is invisible to the end user. It just works! It’s really easy and it just works. There’s no friction involved, and I think that’s what great AI is. And that’s why Echo’s kind of cool. I mean, it doesn’t fully work all the time. You just talk to it and it works, most of the time.

Michael Krigsman: [Laughter]

Ed Sim: That’s what the great AI is. And that experience to bringing that to the enterprise, and the enterprise-level problems, that is pretty exciting for me. And no-one’s kind of doing that yet, and we hope to do bits and pieces of that with Catalytic and Init, but I think that’s a huge, huge opportunity.

Michael Krigsman: What are the …

Sean Chou: We work so hard for end-users to say, “Wait, what just happened there? [...]” Like, that doesn’t seem like it was really a lot, but they don’t appreciate how much work that’s going on behind the scenes to make this slow, magical moment happen.

Michael Krigsman: What are the inhibitors, or the obstacles that prevent this kind of adoption that Ed was just talking about in the enterprise? What needs to be in place to make it happen on a broader scale?

Keith Brisson: Personally, I just think it's not our time. You know, there are different companies tackling different components of it, and it's a matter of … a lot of it is integration, so as data becomes more connected between different parts of the enterprise, you enable new ways of taking these techniques and servicing value to people who [...] it.

So, to me, it’s just a matter of time; it’s a matter of these cool things being incorporated into the workflows where you’re not going to really see them.

Ed Sim: I would also say that there's an infrastructure answer to that, too. A lot of enterprises that we talked to; I'm sure we talked to Michael as well; are kind of re-platforming technologies, evaluating how to bring hybrid cloud into the enterprise. If you look at Pivotal, Pivotal just announces 72 million dollars last year from one-third of the Fortune 100 to help them think about infrastructure and cloud. So, once you have an agile kind of platform to build off of, that makes it that much easier to develop and deploy an AI plugin on top of their existing application. So, I think part of that too is just what the underlying infrastructure looks like.

Keith Brisson: Yeah. And, it’s getting there, you know? We talked with enterprises, too. A few years ago, all of their data was siloed in different systems with no interconnectivity. We talked to them today and they actually have internal APIs and ways of connecting data together which actually enable these kinds of applications. So, I think we’re right on the cusp of the point where enterprises have enough, or are getting enough connectivity between their different data stores, that we can [start] with applications really blossom into being every part of the enterprise.

Michael Krigsman: Keith raises a …

Sean Chou: I think one of the biggest inhibitors right now is just the lack of clarity as to what exactly AI is, because, on one hand, you have the extreme end of the spectrum that actually creates fear: the all-Terminator, Skynet-type of AI; all the way to, on the other end, where people have said, "Hey, I've just whipped together this thing, and it's AI," but a technologist will look at that and potentially say, "Well that's not really AI! You're just regular expression matching," right? So out of these extremes of what AI … what's being proposed and packaged as AI that lack clarity, and the buzziness of it all, actually makes it very hard for people to adopt, because they have to wade through all this crop to get to the real business value behind whatever the product is.

Michael Krigsman: Well, like I said before, it seems like virtually every technology company is selling an AI.

Sean Chou: Yeah!

Michael Krigsman: Hey, I have a couple of AI's. Do you want to buy one?

Sean Chou: Yeah! [Laughter] Yeah, yeah. That’s right.

Keith Brisson: Yes. You know, I think that probably the most extreme example is where they try to portray it as a single AI: this kind of all-knowing mind that can perform these superhuman tasks. But the fact is, generally what they have is a collection of machine learning-powered APIs or services that enable individual things, and they may sell it as this kind of super-knowing mind that’s going to form Skynet and Terminator, but the fact is, they’re not. And, as long as businesspeople recognize that those … that all-knowing mind may not exist, but the individual components can provide a real, concrete value, it shouldn’t be a reason to not adopt those technologies just because it’s not living up to that hype.

Sean Chou: Yeah. In fact, the AI industry, or people who have AI, almost do a great disservice by overhyping AI. And so, we try to, the minute we have a conversation with a customer or prospect, demystify what it is we do, specifically, and say, “This is very tactically how we use AI technology,” because we want to very quickly get to the value that we’re added, and not be kind of caught in this whirlwind of AI and this buzz of AI. I think if you let yourself get caught in that, and if you lead too much on it, you’re always going to disappoint people, because you’re not ever going to live up to the science fiction portrayal of AI, at least not for the next few years.

Michael Krigsman: Well clearly …

Ed Sim: The reality of it is, if you talk to enterprise buyers, they don’t say, “I need an AI;” I need AI, right? It’s like, “I have this problem, and if you can save me 30%, or do things faster, then it’s interesting to me. And if you leverage AI, very cool, but that’s not my checkbox,” right? And that’s not the checkbox item. So, I think we also need to think about the budgets: why they’re buying things and what ROI you’re providing. And in my mind, AI can help create incredible ROI, but you’ve got to apply it to something.

Michael Krigsman: We have just about four minutes left. And, maybe we can just go around the virtual room, as it were, and let me ask each of you for your advice to business buyers; to people in the enterprise, who are looking at these technologies and hearing about - again, every vendor is just hyping this to the max, whether there’s substance behind it or not. So, what advice do you have for people in the enterprise for sifting through the hype so that they’ll get something useful from AI? Who wants to start?

Sean Chou: Me. I can start very quickly. I would just say, stick with the basics, you know? Make the business case, look at the value that’s being created. AI, like I’m saying, is not a checkbox. It’s an enabler. Where you see people making claims about AI, I would say, you know, what’s the business value and if AI’s being used to dramatically multiply the business value, or if you can see incremental gains from any other non-AI types of solutions. I think AI’s going to evolve; what you’re looking for the potential to really move the needle. Not a 10% gain or a 20% gain. Maybe AI allows you to tackle new problems, or allows you to get 2, 3, 10x type of gains over a traditional solution.

Michael Krigsman: Yeah, that’s really interesting. What asking that core business question, “What are we doing; what’s the value; what are we trying to solve?”

Sean Chou: Yeah, absolutely. Stick to that.

Michael Krigsman: Keith, your thoughts on advice to folks in the enterprise who are hearing about this technology, and what should they do?

Keith Brisson: Yeah. My advice is to try to calm yourself down when you hear the hype. A lot of it is … It is hype, but there’s real value there. Like, this is a long-term trend that is just getting started, and it is going to transform the way business processes are tackled throughout the enterprise. And, really focusing on these media business use-cases like what Sean and Ed were saying is absolutely the right way to go. It’s not about technology, it’s about the business process. And, you know, I would encourage enterprises to be cautious about the hype, but really optimistic about what use-cases these can enable, because there are going to be entirely new categories like what Sean was saying, and there is real potential there to transform mass parts of enterprise workflows.

Michael Krigsman: So in other words, don’t buy into the hype, but focus on the real problems and practical solutions.

Keith Brisson: Absolutely! But, still be excited. It’s okay to be excited about where this is going because the long-term trend is there, and it is real. It’s just people are just a little ahead of it. So, that’s all.

Michael Krigsman: That’s a great point! And, finally, Ed Sim, your thoughts. You’ve been involved in folks in the enterprise for a long time. What’s your advice for people who are hearing the hype and trying to figure out what to do?

Ed Sim: If someone is selling you AI snake oil and that’s your initial pitch, run for the hills!

Sean Chou & Keith Brisson: [Laughter]

Michael Krigsman: [Laughter]

Ed Sim: [Laughter] The reality of it is, is that for example, you solve a business problem and think about what problem you have, and AI and some form of AI can help you solve that problem and help you do it much faster, and there are tons of companies in every category leveraging AI. So, really make sure you talk with a few different folks, whether it's large companies or startups - hopefully, startups, because I'm …

Keith Brisson: Agreed.

Ed Sim: And, the second thing is, as far as new categories, I mean, Security Scorecard, for example, created a security ratings market overnight, and that wouldn’t have been enabled without leveraging neural network technology, machine learning, rules-based classification - all that stuff - but they’re not going in and saying, “We’re an AI security company,” they’re going in and saying, “Hey, I’m going to help you figure out which third-party vendors that you have, that have not kind of attached their security systems, and might be risky for your company.” AI and machine learning never gets brought up. So, I think the companies that are the best ones are the ones that actually help to solve a problem, and they actually have amazing, amazing technology, but they’re not pitching that as a first kind of entry point.

Michael Krigsman: Great advice! Clearly, there is a unanimous decision here that the way to go is you solve the business problems and find technologies that will enable those problems to be solved in very dramatically better ways.

Everybody, thank you so much for watching! You have been watching Episode #222 of CxOTalk. Our guests today have been Ed Sim from Boldstart Ventures, Sean Chou, from Catalytic, and Keith Brisson, from Init.ai. I'm Michael Krigsman and tune in again next week. You can see all of our upcoming shows at CxOTalk.com/episodes. Thanks so much. Bye-bye everybody!

Artificial Intelligence and Privacy Engineering

Dr. David A. Bray, Chief Information Officer, Federal Communications Commission
Dr. David Bray
Federal Communications Commission
Michelle Dennedy, Chief Privacy Officer, Cisco
Michelle Dennedy
Chief Privacy Officer
Michael Krigsman, Founder, CXOTalk
Michael Krigsman
Industry Analyst

AI, machine learning, and predictive analytics rely on massive data sets. While holding the potential for great benefit to society, this explosion of data collection creates privacy and security risks for individuals. In this episode, one of the world's foremost privacy engineers explores the broad privacy implications of data and artificial intelligence. 

Michelle Finneran Dennedy currently serves as VP and Chief Privacy Officer at Cisco. She is responsible for the development and implementation of the organization's data privacy policies and practices, working across business groups to drive data privacy excellence across the security continuum. Before joining the Cisco, Michelle founded The iDennedy Project, a public service organization to address privacy needs in sensitive populations, such as children and the elderly, and emerging technology paradigms. Michelle is also a founder and editor in chief of a new media site—TheIdentityProject.com—that was started as an advocacy and education site, currently focused on the growing crime of Child ID theft. She is the author of The Privacy Engineer's Manifesto.

Dr. David A. Bray began work in public service at age 15, later serving in the private sector before returning as IT Chief for the CDC’s Bioterrorism Preparedness and Response Program during 9/11; volunteering to deploy to Afghanistan to “think differently” on military and humanitarian issues; and serving as a Senior Executive advocating for increased information interoperability, cybersecurity, and civil liberty protections. He completed a Ph.D. in from Emory University’s business school and two post-docs at MIT and Harvard. He serves as a Visiting Executive In-Residence at Harvard University, a member of the Council on Foreign Relations, and a Visiting Associate at the University of Oxford. He has received both the Arthur S, Flemming Award and Roger W. Jones Award for Executive Leadership. In 2016, Business Insider named him one of the top “24 Americans Who Are Changing the World”. He is currently the Chief Information Officer at the US Federal Communications Commission.


Download Podcast

Artificial Intelligence and Privacy Engineering

Michael Krigsman: Welcome to Episode #229 of CxOTalk. As always, we have an amazing show. I’m Michael Krigsman. I am an industry analyst and the host of CxOTalk. Before we start, I want to say “thank you” to Livestream for just their great, great, great support of CxOTalk. And if you go to Livestream.com/CxOTalk, they will give you a discount.

So, today we are talking about artificial intelligence; AI; and privacy engineering. And, we have two amazing people. Let me start by introducing Michelle Dennedy, who is the Chief Privacy Officer of Cisco Systems. Hey, Michelle! How are you? This is your second time back on CxOTalk.

Michelle Dennedy: It is! You can’t get rid of me, Michael!

Michael Krigsman: Well, I consider that to be a good thing. So, Michelle, very quickly, tell us about Cisco. I think we know who Cisco is, but tell us about what you do…

Michelle Dennedy: I hope so!

Michael Krigsman: … at Cisco.

Michelle Dennedy: Umm, so Cisco… We are now no longer a startup! Sometimes, I have to remind myself of that because there’s so much innovation going on here, but we are the heart and soul of the network globally. We support … I think we’re up to 130 nations [who] rely on the technology that we sever. And most importantly, for tech today, I report to a fellow named John Stewart who is our Chief Trust Officer. And so, for us, trust, privacy engineering, data protection, security, security engineering, and advanced research all live in one place in operations. So, we are as much of evangelist, forward-thinking innovators as we are operational staff really making this work for ourselves and our customers. So, it’s a fun place to be; kind of at the crux of Cisco’s network, and the gateway, really, to all of the networking that goes on.

Michael Krigsman: Wow! So, we’re going to have to definitely be talking about trust during this conversation. And, our second guest is somebody who regular viewers of CxOTalk are familiar with, because he’s been here a number of times; and that is David Bray. David is an Eisenhower Fellow, as well as the Chief Information Officer for the Federal Communications Commission. Hey, David! Welcome back to CxOTalk!

David Bray: Thanks for having me, Michael! And I guess you really can’t get rid of me since I keep on coming back. So thanks for …

Michelle Dennedy: [Laughter]

Michael Krigsman: And, again, lucky me!

So, I think, to begin, the title of this show is “AI and Privacy Engineering.” And, maybe we should begin by talking about what is privacy engineering? And then, let’s talk about what we mean by "AI." So, Michelle, what is privacy engineering?

Michelle Dennedy: Excellent. So, privacy, by design, is a policy concept that was first introduced at large… It was hanging around for ten years in the networks and coming out of Ontario, Canada with a woman named Ann Cavoukian, who was the commissioner at the time of Ontario. But in 2010, we introduced the concept at the Data Commissioner’s Conference in Jerusalem, and it was adopted by over 120 different countries to say that privacy should be something that is contemplated in the build; in the design; and that means not just the technical tools they can buy and consume, [but] how you operationalize; how you run your business; how you organize around your business.

And, getting down to business on my side of the world, privacy engineering is really using the techniques of the technical, the social, the procedural, the training tools that we have available, and in the really most basic sense of engineering to say, “What are the routinized systems? What are the frameworks? What are the techniques that we use to mobilize privacy-enhancing technologies that exist today, and look across the processing lifecycle to actually build in and solve for privacy challenges?”

And I'll double-click on the word "privacy." It does not mean having clean underpants, already using encryption. Privacy, in the functional sense, is the authorized processing of personally-identifiable data using fair, moral, legal, and ethical standards. So, we really bring down each one of those things and say, "What are the functionalized tools that we can use to promote that whole panoply and complicated movement of personally-identifiable information across networks with all of these other factors built in?" It's not something that you're going to paste onto the end easily. You're certainly not going to disclaim it away with a little notice at the end saying, "Hey! By the way, I'm taking all your data! Cheerio!" Instead, you're really going to build it into each layer and fabric of the network, and that's a big part of why I came to Cisco a couple of years ago. [It's] if I can change the fabric down here, and our teams can actually build this in and make it as routinized and invisible, then the rest of the world can work on the more nuanced layers that are also difficult and challenging.

Michael Krigsman: Okay. So, clearly, there's this key element of trust as you mentioned earlier. And David Bray, when we think about AI in this context of privacy, and of trust, where do they intersect? Where does privacy intersect with AI?

David Bray: So, I loved what Michelle said about this is actually something that's not just putting on encryption, which I think a lot of people will think is a panacea and it's not going to solve everything. It's worth going back to roots of when did the act come about in the United States. It came about when we started doing these things called […] data processing, or we were able to start correlating information, and the […] came something could be made of these correlations given your consent, too. And so, what Michelle said about building beyond and thinking about networks: That really gets to where we're at today, now in 2017, which is it's not just about individual machines making correlations; it's about different data feeds streaming in from different networks where you might make a correlation that the individual has not given consent to with […] personally identifiable information.

And so, for AI, if you think about it, it really is just sort of the next layer of that. We've gone from individual machines, networks, to now we have something that is looking for patterns at an unprecedented capability, that at the end of the day, it still goes back to what is coming from what the individual has given consent to? What is being handed off by those machines? What are those data streams?

One of the things I learned when I was in Australia as well as in Taiwan as an Eisenhower Fellow; it's a question about, "What can we do to separate this setting of our privacy permissions and what we want to be done with our data, from where the data is actually stored?” Because right now, we have this more simplistic model of, “We co-locate on the same platform,” and then maybe you get an end-user agreement that’s thirty or forty pages long, and you don’t read it. Either accept or you don’t accept; if you don’t accept, you won’t get the service, and there’s no opportunity to actually say, “I’m willing to have it used in this context, but not these contexts.” And I think that means Ai is going to really raise questions about the context of when we actually need to start using these data streams.

Michael Krigsman: So, Michelle, thoughts on this notion of context? Where does that come into play?

Michelle Dennedy: For me, it’s everything. We wrote a book a couple years ago called “The Privacy Engineer’s Manifesto,” and in the manifesto, the techniques that we used are based on really foundational computer science. Before we called it “computer science” we used to call it “statistics and math.” But even thinking about geometric proof, nothing happens without context. And so, the thought that you have one tool that is appropriate for everything has simply never worked in engineering. You wouldn't build a bridge with just nails and not use hammers. You wouldn't think about putting something in the jungle that was built the same way as a structure that you would build in Arizona.

So, thinking about use-cases and contexts with human data, and creating human experiences, is everything. And it makes a lot of sense. If you think about how we’re regulated primarily in the U.S., we’ll leave the bankers off for a moment because they’re different agencies, but the Federal Communications Commission, the Federal Trade Commission; so, we’re thinking about commercial interests; we’re thinking about communication. And communication is wildly imperfect why? Because it’s humans doing all the communicating!

So, any time you talk about something that is as human and humane as processing information that impacts the lives and cultures and commerce of people, you’re going to have to really over-rotate on context. That doesn’t mean everyone gets a specialty thing, but it doesn’t mean that everyone gets a car in any color that they want so long as it’s black.

David Bray: And I want to amplify what Michelle is saying. One of the things, when I arrived at the FCC in late 2013, we were paying for people to volunteer what their broadband speeds were in certain, select areas because we wanted to see that they were getting the broadband speed that they were promised. And that cost the government money and it took a lot of work, and so we effectively wanted to roll up an app that could allow people to crowdsource and if they wanted to, see what their score was and share it voluntarily with the FCC. Recognizing that if I stood up and said, “Hi! I’m with the U.S. government! Would you like to have an app […] for your broadband connection?” Maybe not that successful.

But using the principles that you said about privacy engineering and privacy design, one, we made the app open source so people could look at the code. Two, we made it so that actually, when we designed the code, it didn’t capture your IP address, and it didn’t know who you were in a five-mile-radius. So it gave some fuzziness to your actual, specific location, but it was still good enough for informing whether or not broadband speed is as desired.

And once we did that; also, our terms and conditions were only two pages long; which, again, we sort of dropped the gauntlet and said, “When was the last time you agreed to anything on the internet that was only two pages long?” Rolling that out, as a result, ended up being the fourth most-downloaded app behind Google Chrome because there were people that took a look at the code and said, “Yea, verily, they have privacy by design.”

And so, I think that this principle of privacy by design is making the recognition that one, it’s not just encryption but then two, it’s not just the legalese. It really is if you can show something that gives people trust; that what you're doing with their data is explicitly what they have given consent to; when they have chosen to have permitted with their data ... That, to me, is what's really needed for AI [which] is, can we do that same thing which actually shows you what's being done with your data, and gives you an opportunity to weigh in on whether or not you want it or not?

Michael Krigsman: Can I ask either of you a thought here? AI is really, at its heart, pattern matching. And, pattern matching is certainly not new, although it seems that we have built a cult around AI as if it is something new.

Michelle Dennedy: [Laughter] But it’s magic.

David Bray: Sure! It is what cloud was five years ago. [Laughter]

Michelle Dennedy: Yeah. [Laughter]

Michael Krigsman: So, since pattern matching is not new, and therefore, the foundations of AI are not new, and privacy is certainly not new, why should we even care about this topic in such an acute way? And also, the ethical implications. Why should we care about this?

Michelle Dennedy: So, if you want to start, David? I think we both have a lot to say, here. [Laughter]

David Bray: I will defer to you. Do you want me to? It’s up to you.

Michelle Dennedy: Okay. So I will dig in. First of all, I have to just underline anyone watching the playback, go back and watch Dr. Bray talking about “Yea, verily.” Okay? How many people? That’s like Robin Hood-speak in the government! So, yea, verily we see! So, I had to put a little purple underline about that.

But as far as why do we care, and why do we care now? And I’ve heard this in my entire career in privacy for two decades, either it hasn’t existed, or it’s so freaking hard that only a genius could get it. It’s kind of like, which null set is there.

AI, first of all, I think, taking a step back for artificial intelligence. You can tell that the way my mind is kind of matrixed. What are we talking about? When we say "artificial intelligence," are we talking about Skynet, which is like super-secret magic or are we talking about a really huge amount of dumb machines that are gathering stuff from either observations, sensors, or the inputs of human humans? And so, the quality degrades over time.

And then, you’re coming up with analytics. Are we back to statistics saying what is the trend and that can be artificial intelligence? If you think about weather mapping and how we decide which planes get to take off when? There’s a lot of artificial intelligence and analytics that come from sensors that talk about what’s the moisture in the air, wind pressure, what the weight on the plane is, how old is the plane; blah, blah, blah, blah, blah. All that data coming together so that someone can make a decision based on observations that we have not directly made in the moment, that can be a type of artificial intelligence.

And, it has impacted some lives, you know? Whether that plane makes it is a really important thing to me, particularly if I’m sitting on there or if someone I like is sitting on there. I may have a list of otherwise, but that’s an ethical concern.

David Bray: [Laughter]

Michelle Dennedy: I think the other thing is why do we care when we care now is the first 30-35 years of compute has been, "Can we make it work?" And I think the next 30-50 years is, "Can we make it work for us?" Are we the victims of the only platform [being] available is X, and therefore we’re using that and calling that trust? Or, do we now legitimately have a choice of the quality, the kind, the testing, of data and how it’s processed and when and where it’s processed?

And I think that we are on the cusp of saying “yes” to that answer. There’s enough information and capability in the networks that getting broadband out so that information can get to schools in rural areas, or teachers can learn; there’s an amazing program in Chengdu that Cisco actually helped after a terrible earthquake, and now we have the best teachers available. They broadcast their lesson plans the night before, and they broadcast them to over a million children in the outreaching provinces of China. That’s the power of the network.

And artificial intelligence can be a very big component of that. And obviously, there are other issues that we’re going to get into of, you know, there are concerns here. And we have to think about both the quantity and the quality of those concerns.

Michael Krigsman: David, we have an interesting question from Twitter. Scott Weitzman is asking a continuation of this thread. Scott Weitzman is asking, “With AI, is there a need for a new level of information security? And, should AI itself be part of this security?”

David Bray: So, I’ll give the simple answer which is “Yes and yes.” And now I’ll go beyond that.

So, shifting back to first what Michelle said, I think it is great to actually unpack that AI is many different things. It's not a monolithic thing, and it's worth deciding are we talking about simply machine learning at speed? Are we talking about neural networks? But that said if we don't spend the time unpacking all of that ... I think why this matters now is five years ago, ten years ago, fifteen years ago, the sheer amount of data that was available to you through the Internet of Everything; devices that are now being streamed to the internet; is nowhere near to what it is right now, and let alone what it will be in five years.

I mean, if we're right now at about 20 billion networked devices on the face of the planet relative to 7.3 billion human beings, estimates are at between 75 and 300 billion devices in less than five years. And so, I think we're beginning to have these heightened concerns about ethics and the security of data. To Scott's question: because it's just simply we are instrumenting ourselves, we are instrumenting our cars, our bodies, our homes, and this raises huge amounts of questions about what the machines might make of this data stream. It's also just the sheer processing capability. I mean, the ability to do petaflops and now exaflops and beyond, I mean, that was just not present ten years ago.

So, with that said, the question of security. I would modify Scott’s questions slightly and say it’s both security but also we need maybe a new word. I heard in Scandinavia they talk about integrity and being integral. It’s really about the integrity of that data: Have you given consent to having that be used for that purpose? So I think yes, AI could definitely play a role not just in making sense of is this data being securely processed? Because the whole challenge is right now, for most of the processing we have to decrypt it at some point to start to make sense of it and re-encrypt it again. But also, is it being treated with integrity and integral to the individual? Has the individual given consent?

And so, one of the things that I’ve heard is it was actually raised when I was in conversations in Taiwan. I want to raise the question of, “Well, couldn’t we simply have an open-source AI, where we give our permission and our consent to the AI to have our data be used for certain purposes?” For example, it might say, “Okay, well I understand you have a dataset served with this platform, this other platform over here, and this platform over here. Are you willing to actually have that data be brought together to improve your housekeeping?” And you might say “no.” He says, “Okay. But would you be willing to do it if your heart rate drops below a certain level and you’re in a car accident?” And you might say “yes.”

And so, the only way I think we could ever possibly do context is not going down a series of checklists and trying to check all possible scenarios. It really is going to have to be a machine that is actually able to talk to us and have conversations about what we do and do now want to have done with our data.

Michael Krigsman: So, the issue, then, is this combination of data plus compute power. And you add those together with the, can we say, advanced new pattern matching capabilities and techniques, and that’s why we have the privacy; the new set of privacy challenges? Is that a fair statement?

David Bray: Very much.

Michelle Dennedy: […]

David Bray: I would say IoT plus interconnectivity, plus machine processing; this is the storm ahead. Go ahead, Michelle. Sorry.

Michelle Dennedy: Yeah. Now that I know, I’ll just add one more element, which is the carbon-based unit.

David Bray: Yes.

Michelle Dennedy: Just so you know. [Laughter] We’re seeing cultures come together thanks to the network like never before. And sometimes, that’s wonderful. My daughter has a best friend in China, and someone else that she’s never met in Amsterdam, and that’s incredible and supportive and wonderful, and we all learn great things.

However, we are also exposed on a daily basis to the trauma of the world. Never before have we been able to witness mass problems, you know? When you turn on the BBC World, I didn't grow up with the BBC World. There were only two networks that were available to me. And now, we are bombarded with information, for better or worse […]. And we're making decisions, and we're seeing the world kind of recycle old ideas, and hopefully, process them in other ways.

So, when we’re making automated decisions, it’s absolutely critical that we understand that we are documenting, in some way, what those decisions are and what context. So it’s like, context on top of context, on top of context. And we understand that sometimes, as David was saying, there are certain periods where it’s like, “Do I want everything on this young, healthy person? Do I want every bit of my health and aspects of my health monitored? Maybe not?” Is that a decision we’re going to make in loco parentis. I threw a little Latin in there for you, David!

David Bray: [Laughter]

Michelle Dennedy: Yup. I’m here for you!

Are we going to make that, as a society, to say, "Listen. If only I had put sunscreen on my water stream when I was younger, I wouldn't be, like, holding my face up with bandages at this point in my middle age!" Or, are we simply going to let people choose and educate them enough to make good choices? We don't have the answers yet, and that's why I think it's interesting and exciting and innovative to try to build out controls and ethical tools as we're building this brave new world.

David Bray: And I want to add to what Michelle said about the importance of people, because just as we know that human beings do great things, mundane things like cat videos, as well as not-so-great things, too; so, too, will the machines. Machines […] are an amplification of what we share and send with it. And without naming the name of the specific app, recently there was a story this week, in which an app was bringing in people's photos and faces and it allowed you to make them “animé’d." The challenge is when you click the beautification button, unfortunately, the app's conclusion was beauty was lighter skin, which if any of us had ever […] that's atrocious towards…

Michelle Dennedy: Yeah.

David Bray: ... the human … That’s what the machine had been taught to think what’s beautiful. So, we need to recognize there is both the importance of privacy and engineering privacy by design, but also some sort of check to make sure that the machine is not going down a really bad path that is either incorrect socially, racially, whatever it might be. And we need to be aware of that.

Plus, just think about it. I mean, ten years ago, most of us did not have lots of our health data online. And so, we were not targets for having that data be stolen. But if you look at the recent cyber-trends, where the real interesting attacks are trying to go after is actually after healthcare data because that is a huge value, unfortunately, in the dark web. And so, there are also going to be questions of even if you do share it and you give permission, are we now creating a new target or a new risk attack surface that we have not thought about as a society?

Michael Krigsman: Michelle, I have a question for you picking up on David’s comment just now; this issue of bias. So, with machine learning, we give the system datasets, and if those datasets have some inherent bias, then the AI system will pick up that same bias. And so, are there privacy engineering considerations that come into play with the respect to the inherent potential for inherent bias in a given system?

Michelle Dennedy: Yeah. And this is something I’ve been really thinking hard about lately and talking [about] to much smarter people than myself, which isn't hard sadly. But, there's a woman; gosh, I'm going to forget her name now; Elish; she wrote a paper called "Moral Crumple Zones," and I just love even the visual of it. If you think about cars and what we know about humans driving cars, they smash into each other in certain kind of known ways. And the way that we've gotten better and lowered fatalities of known car crashes is we actually use physics and geometry to design a cavity in various parts of the car where there’s nothing there that’s going to explode or catch fire, etc. as an impact crumple zone. So all the force and the energy goes away from the passenger and into the physical crumple zone of the car.

Now, the analogy falls apart fairly readily. So, don’t throw your Twitter knives at me. But, I think Madeleine Clair Elish is her name – I'm sorry, Madeleine, I think you're brilliant. She's defending her Ph.D. this month, so let's hold hands out for Madeleine – but she's really working on exactly what we're talking about. We don't know when it's unconscious or unintentional bias because it's unconscious or unintentional bias. But, what we can do is design in ethical crumple zones, where we're having things like testing for feeding, just like we do with sandboxing or we do with dummy data before we go live in other types of IT systems. We can decide to use AI technology and add in known issues for retraining that database.

I’ll give you Watson as an example. And Watson isn’t a thing. Watson is a brand, right? So the way that the Watson computer beat Jeopardy contestants is by learning Wikipedia. So, by processing mass quantities of stated data, you know, given whatever levels of authenticity that pattern on. And, it could really simulate a genius person.

What Watson cannot do is selectively forget. So, your brain and your neural network are actually better at forgetting data and ignoring data than it is for processing data. So, we're trying to make our computer simulate a brain, except that brains are actually good at forgetting. AI is not good at that, yet. So, you can put the tax code, which would fill three ballrooms if you print it out on paper. You can feed it into an AI type of dataset, and you can train it in what are the known amounts of money someone should pay in a given context?

What you can't do, and what I think would be fascinating if we did do, is if we could possibly wrangle the data of all the cheaters. What are the most common cheats? How do we cheat? And we know the ones that get caught, but more importantly, how do […] get caught? That's the stuff where I think you need to design in a moral and ethical crumple zone and say, "How do people actively use systems?" The concept of the ghost in the machine; how do machines that are well-trained with data over time experience degradation? Either they're not pulling from datasets because the equipment is simply … You know, they're not reading tape drives anymore, or it's not being fed from fresh data or we're not deleting old data. There are a lot of different techniques here that I think have yet to be deployed at scale that I think we really need to consider before we're overly relying [on AI], without human checks and balances, and processed checks and balances.

Michael Krigsman: So, let me ask David. What can we actually do about this? Because I think it's one thing and relatively easy to talk about it. But, what are the implications for policy? For the private sector? For the people that are building these systems? For the corporations that are using AI tools and collecting these large datasets? And for data scientists? What do we need to do?

David Bray: So, I think it’s going to have to be a staged approach, because exactly as Michelle said; I’m going to throw out also some Greek: “experiments” and “expertise” both come from the Greek word “experior,” meaning “out of danger.” We are in dangerous times, and we’ve got to do some experiments to figure out the expertise to move forward. I would recommend as a starting point: begin to have … You almost need to have the equivalent of a human omnibudsman – a series of people looking at what the machine is doing relative to the data that was fed in.

And you can do this in multiple contexts. Either a, it could just be internal to the company, and it’s just making sure that what the machine is being fed is not leading it to decisions that are atrocious or erroneous. Or, if you really want to gain public trust, share some of the data, and share some of the outcomes but abstract anything that’s associated with any one individual and just say, “These types of people applied for loans. These types of loans were awarded,” so can make sure that the machine is not hinging on some bias that we don’t know about.

Longer-term, though, you’ve got to actually write that omnibudsman. So real quick, what Michelle said, we need to be able to engineer an AI to serve as an omnibudsman for the AI itself. So really, what I’d see is not just AI as just one, monolithic system, it may be one that’s making the decisions, and then another that’s serving as the Jiminy Cricket that says, “This doesn’t make sense. These people are cheating,” and it’s pointing out those flaws in the system as well. So we need the equivalent of a Jiminy Cricket for AI.

Michael Krigsman: Okay. There’s …

Michelle Dennedy: We have range! We’ve had Disney, you know, we’ve got ancient Greek, I like it!

David Bray: You need to bring in Pirates of the Caribbean and we’re set!

Michelle Dennedy: [Laughter] Little extra eyeliner for me. Make me Johnny Depp.

David Bray: Excellent!

Michael Krigsman: Okay. So there we have that word again; that word, “trust.” What does trust keep coming up as a thread throughout this conversation?

Michelle Dennedy: So, I’m going to go all Renee Brown on you. [Laughter]

David Bray: Go for it!

Michelle Dennedy: And if you don’t know who she is, you really should! She’s so awesome! She’s out there rising strong and all sorts of good stuff.

I think at the root of trust, and why we keep rotating back to that word, even in computer science, where typically, a lot of people gravitated to this field because they’re either afraid of humans or just prefer machines. Not that we don’t love them all! I have Sheldon here just, you know … We love you!

David Bray: [Laughter]

Michael Krigsman: [Laughter]

Michelle Dennedy: Umm … [Laughter] But, I think it’s really important to … How do I put it? Just, to build this stuff in is essential and to build it in as coming from the people that are attracted to a field that may not be human, it’s so interesting once you introduce the notion of trust.

I’d break down trust into two factions. One is, I’m on a mountain, and I’m hanging onto a blade of grass, and you’re the dude with the rope. I trust you with my life. Why? Because I trust you more than my ability to fly. You’re kind of the only game in town. Then, there’s the Renee Brown kind of trust, which is I think where we’re heading with these very complex compute systems and the ones that feel simple because they are garage apps, and they're easy to download and consume and throw away. But, they're really adding to the complexity of the information footprint that surrounds you in the network; and that is trust that comes over time. Trust that says, “For better or for worse.” Like, there are certain platforms; why people buy them when they’re 1.0, I cannot comprehend. They will blue-screen you every time. Wait until 2.0, people! We know this! We’ve got 30 years of experience! My trust is, I trust that you’re going to release it too soon. And I trust that some sucker’s going to test it for me and I’m going to get to it when it’s 2.0. And that happens in a lot of technology, and including cars and all sorts of things. So, we’re buying a boat, right? Never buy a new boat.

Umm, the other kind of trust is the one we’re really trying for. You know who I am; you know I'm going to perform for you over time; you know that when I make a mistake; I'm going to admit it, and correct it to the best of my ability. And, I'm going to have something or someone that is either a direct advocate for you like the Chief Privacy Officer, or a proxy, which is going to be working with other people that come in and test our systems. You know, we've got consultancies all the time that test our ethical and our trust frameworks. Are they really too self-centered? Are they only looking at our shareholders? Or, are they also looking at the quality of care that we’re giving to all of our customers and employees?

So those are the two kinds of trust that I think they can be broken down once again as kind of, “I’m like a broken record.” What does it mean? What is the problem we’re trying to solve? How do we break it down into things that we know we know how to do? And let’s look really hard and things we don't know how to solve for, and either retrain for it, have teams around it, have something else as a proxy when you can't solve it completely and perfectly.

Michael Krigsman: David Bray, let me ask you something here relating to this. Yesterday, I was in New York. I was running an event, and one of the participants on stage was a senior person from IT who works at JetBlue. And, you talk about trust, he was explaining how, in his world, there is no room for technology error, and therefore the trust level just needs to be right up at the top. How does this dimension of trust come into play?

David Bray: So, you set me up perfectly, because I wanted to amplify with what Michelle said. When I was doing my Ph.D., I was focusing on how to improve organizational response to disruptive events, both the technology and the humans, because I like both of them and the interesting messiness that occurs when you get technology and humans together because there are all sorts of pathologies that arise. Trust was key and I would define trust based on the academic literature as the willingness to be vulnerable to someone, or something you can't control. And so, the JetBlue example is, yes, when you're on a plane when that wing flies off, or the autopilot starts soaring into the ground; […] you’ve lost trust because you’re now vulnerable to the actions of something that you did not have direct control over.

Now, there are three predecessors that if present, it’s been shown, humans are now willing to trust a person or a thing. If they believe that the individual, or the machine, or the system is benevolent, and so they have a good interest in mind for the person; if it’s competent, so it actually is skilled at what it does; and then finally, that it acts with integrity. It’s not going to do when you’re not looking, something that you’re not expecting, and only behave on its best behavior when you’re looking.

If those three things are present, then you’re willing to be vulnerable, then you’re willing to trust. So I think when I think about AI, how can we begin to [instrumentalize] and show benevolence? How do we reveal that to the public if the public buys into it? Competence is a little bit easier. Integrity is the hard one. It circles back to that conversation where, again, there are experiments in Europe, in Scandinavia in particular, where this idea of how do you show integrity? Because let's think about it. The professions, going back to the 19th Century, doctors and lawyers; the reason the professions are able to credential and self-police themselves, is because they do actually find people that are behaving badly, the un-credentialed, […] I'm going to take their license away. You'll get debarred, or something like that.

What are the equivalent of the professions for AI, which is if the community determines that an AI is not behaving with integrity, we’re going to take your license away or we’re going to debar you. Because I think the public is willing to let the private sector self-police itself insofar as it sees the self-policing as effective. Otherwise, they’re going to be looking for other options.

Michael Krigsman: I love this! And, so I have to ask the brilliant privacy engineer among us.

Michelle Dennedy: Uh-oh!

Michael Krigsman: Michelle, how do we manage this integrity issue? Figured we posed a difficult question to her. [Laughter]

Michelle Dennedy: [Laughter] Yeah! Well, and I love the breakdown is exactly, you know, on-point for how we’ve even organized around this at Cisco with our Chief Trust Officer, and putting us … You know, we have arms and legs in public policy, in the world of legal support, etc. But this is not a compliance function. This is an integrity-building function. Did we have the right people? Are they trained? Are they constantly, actively listening to vulnerable clients? And those clients are not just money-making machines who need networking or clouds, or collab tools. These are people who are serving and creating experiences. These are people who have families, you know? We have 70 thousand Cisco employees, and they have families. And I take every single day that I work here as a fiduciary obligation to make sure that these families are … people are going home to their families with integrity.

And I think part of that is really fascinating in the study of the ethics of … When you think about ethics versus morality, and this, you know, to anybody who’s really a scientist, I apologize for getting your terms about wrong. I think of morality as "killing people, bad." That's a pretty universal one, I think! I hope! Killing people, bad! But, if you get into "What do I own as far as data?" this is where my ancient roots of [patent] litigator come out and say, "I think about personally-identifiable information many ways as similar to other versions of intellectual property.” It’s a story that exists in someone’s mind, in someone’s database, in someone’s diary somewhere; together, the three of us jointly own the truth that we were here today and had this conversation. Everyone who’s following us on Facebook Live or Twitter are part of this conversation.

Each one of those little breadcrumbs is an element of personally-identifiable information. Who owns that? Well, if you’re in the Western world, there’s notions of intellectual property. If you’re in the U.S., we’re having kind of a change, a sea-change of what do we mean by being…. Are we the ethicist and the moral beacon of the world, or are we looking within our territorial borders anymore? If you are in Asia and certain other collective communities, for three thousand years, they look at intellectual property as selfishness. We look at it as stealing. “Why are you takin’ my stuff?” They look at us like “Why would you prevent everyone from benefitting from innovation?”

A society that’s split and you’re still trying to prove integrity, what you have to do is be very open about what is your ethical model. And if the model is you get to control the informational stories that are told about you to the greatest extent possible, without degrading the ability of everyone else to live their life with integrity and self-determination, and for the organization to continue to be around to protect that data, that is once again … This mixed sort of complex use-case-driven model of integrity. But I think part of it is even when we don’t know what the answers are, admitting that, and saying that we have organized; we have invested; we are publicly confused working with external parties that are old customers to continue asking questions on what fields like integrity … And talking about feelings at large.

You know, [at] Cisco, we build that network, remember? We, right now, are the electronic currency, if you will. The current that runs underneath all of human activity these days; it’s a huge responsibility. And as we’re getting smarter in the networks and the demand is for us to curate more and more of that information along its way, we’re also going to be the curators of the world’s currency of data. And so, the notion of integrity, in how you build in every single step of the way, from the policy to the buildout, to the quality models, to the organizational structures that deliver that; all of that matters. And all of that has to be orchestrated together in one, kind of beautiful package […].

Michael Krigsman: We have literally […] four minutes left, and it feels like this has gone by in a snap. So, David Bray, you have this broad overview of AI and technology, and I've even heard you quote from the Federalist Papers. So, what should we do?

David Bray: Well again. We’re going to have to experiment. I don’t think any one person has all the answers. I definitely don’t intend to. That means we need to be listening to all sectors and all members of the public because, as Michelle said, there are huge variances in perspectives both in the United States, but also around the world. I mean, we didn’t even dive into Europe and what’s going to be going on with DDPR, and there are going to be huge questions about how you can even begin to show what an AI is doing with its decision making without showing the data, too.

But I will leave with two, main thoughts. First, as you mentioned, the Federalist Papers, I love to say, "What is government but a great reflection of humanity? If all men were angels, no government would be necessary." Let's just replace the word "government" with "civil society" and "public service," and let's add, instead of "men and women," let's do "men, women, and AI." What is AI but the greatest reflection of humanity? Because it is! It's us being reflected back, and that may or may not be a way to say, "Hold up. Wait. Is this fair? Is this right? Is this not biased, or is this prejudiced?" So, I think AI can begin to be used as a tool to say, "Are you aware of these biases? Are you aware of these concerns that you may not have been aware of otherwise?" There's some good there.

And then two, to sort of take what Michelle said about ... I mean, the world has massive differences in philosophies, and I've tried to figure out … In three thousand years, philosophers still can't agree. But what I would say as to one undercurrent I see is, "Do unto others as they give consent and are willing to be committed to be done unto them." I think that, to me, at the end of the day … So, we have to develop tools that allow them to express their consent, express their permissions, and then have it so it’s not always asking you. Because we can’t answer to these questions every five minutes. But, have it be sort of our own, personal open-source AI bot that is representing what we give consent to, what we give permission to in the world ahead.

Michael Krigsman: And, it looks like, Michelle Dennedy, you’re going to get the last word here.

Michelle Dennedy: Oh, dear!

Michael Krigsman: In about one minute, David said, “Men, women, and AI.” So, please, tell us about that world.

Michelle Dennedy: [Laughter]

Michael Krigsman: And you only have a minute, by the way.

Michelle Dennedy: I only have a minute! Well, so I will end on a high note, hopefully. I will add, and I think they were implied in his list; children, and this new generation coming up. I am the mother of an eleven-year-old, and a fifteen-year-old who think they know everything about the world already! We have our judgment, and we have what we've learned from the last three thousand years of history to impart to our little ones. They are growing up in a very different environment. They are growing up with far more technical might than we ever did before. And, I think if there's one thing I would say to the world, it's "Don't count them out." If you think your kids don't want to curate and decide when they are a certain persona, you're wrong. If you think that your kids aren't interested in how their information is processed and how it's used for and against them, you're wrong! If you think that they aren't out there marching with their crazy, silly signs and doing all the things that we're doing to really express a new type of democracy, you’re out of your mind!

I’m looking at the younger generations to really take a rethink and a reset of what are the specs and requirements for the hard-coded systems, and how do we allow for flexibility over time and maturity. And as we discard some of these tools that we thought were so nifty when they first came out, some of them will be like the fat, jiggling machines of the 1920’s. They’ll just look silly to us.

Michael Krigsman: Okay….

David Bray: […] up a character from a 1999 Super Bowl ad. I think it’s going to be some that are like that.

Michelle Dennedy: Exactly! Exactly. And we’ve got Grumpy Cat.

David Bray: Yes, exactly! [Laughter]

Michelle Dennedy: Like a selfie.

Michael Krigsman: I love that! And, are those the only real-life emojis that you have, Michelle Dennedy?

Michelle Dennedy: No! I've got … This is the Order of the Flying Pig, and I would like to present this to Dr. David Bray. The Flying Pig was years ago; someone told me that we only needed to do privacy for 10% of my time to support my customers, because it was going away. And I said, "Well, when pigs fly, we will do privacy all the time." So, I present you virtually, the Order of the Flying Pig, David, because we're not done doing privacy, ethics, or AI.

David Bray: Very honored to be inducted. Thank you!

Michael Krigsman: And, with that, it is time to conclude, sadly, Episode #229 of CxOTalk.  And, what an amazing conversation! We’ve been speaking with Dr. David Bray, who is an Eisenhower Fellow, and the Chief Information Officer of the Federal Communications Commission; as well as Michelle Dennedy, who is the Chief Privacy Officer of Cisco. And, I will extend an invitation to both of you to do this live. We need to have this conversation live on stage someplace, in front of an audience.

David Bray: With the Flying Pig. As long as we bring the Flying Pig, too, I’m all for it. Thank you, Michael.

Michelle Dennedy: She’s in! [Laughter]

David Bray: [Laughter]

Michael Krigsman: [Laughter] Everybody, thank you so much for watching. Check CxOTalk.com/episodes to see what’s coming next. And, like us on Facebook and of course, subscribe to us on YouTube. Thanks a lot. Take care, everybody. Bye-bye!

Chief Digital Officer: Lessons from a Former CIO

Christian Anschuetz, Chief Digital Officer, UL
Christian Anschuetz
Chief Digital Officer
Michael Krigsman, Founder, CXOTalk
Michael Krigsman
Industry Analyst

With the Chief Information Officer role in transition, business expectations of the CIO have also changed. In this episode, we talk with a seasoned CIO, Christian Anschuetz, who left that position to become Chief Digital Officer of Underwriters Laboratories. The discussion explores the Chief Digital Officer role and offers advice to both CIOs and their organizations.

Christian Anschuetz is the Chief Digital Officer at Underwriters Laboratories. He has been the Chief Information Officer of Underwriters Laboratories since November 2008. Mr. Anschuetz is responsible to establish IT strategies, goals and priorities and to provide senior leadership on key technology initiatives in the areas of enterprise resource planning, business process automation, computer systems validation, and electronic communications. Mr. Anschuetz served as the Chief Information Officer and Executive Vice President of Americas at Publicis Groupe SA, where he was responsible for the strategic management and delivery of IT support to over 17,000 associates in more than 100 unique lines of business. Prior to Publicis, Mr. Anschuetz served as Vice President and Director of Operations at BCom3. He began his professional career in a broad range of progressive management roles these included; Senior Consultant and Information Security Thought Leader for Sprint Paranet, and Senior Partner/Founder of UpTyme Consulting. He holds a Bachelor's Degree in Economics from the University of Michigan, Ann Arbor, and a Bachelor's Degree in Computer Information Systems from Strayer University. He was a decorated United States Marine Corps officer and a veteran of the First Gulf War.

Download Podcast

Chief Digital Officer: Lessons from a Former CIO

Michael Krigsman: Welcome to Episode #223 of CxOTalk. I’m Michael Krigsman, and I am your host. I’m an industry analyst, and we have a really interesting show. We are going to be talking about the role of the Chief Digital Officer, and our guest, Christian Anschuetz, works for a company called UL that everybody knows under the name “Underwriters Laboratories.” So, I have to imagine that having been founded in 1884, the company is different today than it was way back then.

Christian Anschuetz: Oh, it is so different than it was back in 1894. Hugely diversified, it is now a global leader. We're in over a hundred countries worldwide, thirteen thousand people to this day; it's a fantastic company with just a super, absolutely superb mission, which again is all about safety, safer living, [a] safer working and living environment.

Michael Krigsman: So, you were the CIO at UL for many years. And, then you transitioned recently into the Chief Digital Officer role. So, let’s begin by talking about that CIO role. So, what was your mandate as the CIO?

Christian Anschuetz: Well, I think I was just like, you know, every CIO. My job was to help create a contemporary technology, a platform if you will, that would allow the company to be successful in the marketplace.

Michael Krigsman: And, what are some of the challenges that you face? I mean, it’s a really tough job. And, I’ve seen you talk a lot about the role of IT in terms of supporting innovation at the company. So, I think that’s a particularly interesting aspect as well.

Christian Anschuetz: Well, you know, I think that everybody has a role in the space of innovation. And, I definitely think that technology, whether you’re in IT or in a line of business that’s associated with technology, you have to lead from there, because you simply are already in that cutting-edge space. And, I think we’re uniquely positioned as leaders in technology to be aware of new and emerging trends, and take advantage of them for our respective businesses.

Michael Krigsman: But, I guess, you know, the challenge that many CIOs face is bringing innovation back inside the organization, and getting out of just supplying the infrastructure, right? And, people use the buzzword “becoming a partner with the business.” So, maybe we can kind of explore what that is, and how do you go about doing that?

Christian Anschuetz: Umm, yeah. So, maybe you’ve got to bring innovation in. You know, I’m a firm believer in the idea of cross-pollination. I think that you really have to innovate by creating a […] so, you really have to spend about two-thirds of your time outside of your comfort zone, meaning outside of your industry. You learn from what others are doing and find connection points. And then, innovate through … understanding what others are doing, and bringing those into your industry, bringing those into your company. Otherwise, what you end up have to happen, Michael, is potentially having … So we see this all the time, right? It's an industry of "me too." If all you're following is the same players in your market, the same players in your industry, you're going to keep doing what the same industry is doing. And, how innovative is that? Or, is it perhaps more interesting to bring something from outside the industry altogether, and create something altogether new? Maybe, the new category takes you and your firm outside of your niche.

Michael Krigsman: That’s a very interesting point. I guess the question, then, becomes how do you do that? I mean, do you talk with startups? How do you bring external innovation ideas inside, and especially into IT in a way that will affect the broader business outside of IT?

Christian Anschuetz: So, is the question how do you do that?

Michael Krigsman: Yeah.

Christian Anschuetz: That’s, you know, that’s kind of the magic of it. Well, you know, I think so much of it comes down to a fundamental leadership conversation, right? So, first of all, you’ve got to lead by example. You have to be able to do that yourself. You have to be willing to be really uncomfortable, right? And push yourself in these new and different areas and hopefully inspire people to do the same.

When you bring these different ideas in, you have to hopefully make the connections and show that in these intersections, in these different things that you can possibly do with the business, you can maybe create an inspiring vision that [could] have people go, “Wow! This is fantastic! This is something I want to be a part of!” I guess the point of what I’m trying to make, Michael, is you can’t tell people what to do in this space, but you can inspire them to want to be innovative. You can inspire them to want to look outside their comfort zone, you can inspire them to want to look up outside […].

Michael Krigsman: And so, can you give some examples from your experience at UL of how you did this? I know it’s a leadership issue, as you were describing, but I think it’s one that many people find very difficult, or there would be more of it.

Christian Anschuetz: Uhh, yeah. I think it is very difficult, and I think; well, let’s talk first about the last part; we said that there would be more of it. You know, what’s your impression, Michael? Are most firms struggling with disrupting themselves, even though it’s obvious that all firms are going to be disrupted?

Michael Krigsman: I mean, is that a setup question? I think disrupting oneself, whether it's a … Look, as people, it's hard to disrupt and rethink how we are, what we do, how to improve ourselves, and companies are comprised of people. So, absolutely it's very difficult for most companies, and very few companies are actually disrupting themselves. I think that's really hard.

Christian Anschuetz: Yeah, well why is that, do you think?

Michael Krigsman: Hmm. The tables are turned. The interviewee becomes the interviewer. Again, I think the reason is that it’s easier to stay stuck doing what we know. So, in business terms, we have sources of revenue. And, we have processes. And, we don’t want to risk upending or disrupting those sources of revenue. So, we tend to do that which we’ve done before, which we know has worked in the past.

Christian Anschuetz: Yeah. Michael, I think you’re exactly right. And, I’d add another dimension to it, actually. And it goes back to what you were saying about businesses not wanting to disrupt the revenue streams, or disrupt their current models. I think there’s another part to it, too. Another part is that I don’t think people want to disrupt themselves. And you know, when it comes right down to it, we can talk about IT and digital and everything, and you know all day and all night, and think about it in terms of technology, but in the end, it really does come down to people. Just because it’s digital doesn’t mean we take people out of the equation. In fact, digital is actually more powerful when you consider people as part of the equation.

What the reality is, is I think that most people struggle with disrupting themselves. I mean, change is hard. I mean, you know, there's a reason we call growing pains "pains," right? Because it's hard to grow into new and different areas. And so, I think it's really important for us to tend to the wants and the needs and perspectives of the people that we're affecting when we're having these conversations in order to really help bring in these innovations into these disruptions and make them really disruptions that are opportunities as opposed to disruptions that are perceived as distractions.

Michael Krigsman: So, you're saying that the key is to engage the people who are quote-on-quote going to be disruptive or disrupted, in order to make them part of the change process.

Christian Anschuetz: Yeah. I think the key is actually to look at them as less of people that are going to be disrupted, and more people that are going to then actually become disruptors themselves. They’re going to become part of the disruption. Umm, you know, at least that’s the perspective of a firm that’s trying to disrupt itself.

Michael Krigsman: And is that what … Is UL trying to disrupt itself?

Christian Anschuetz: Yeah, of course. Well, we definitely are. We're a hundred and twenty-year-old firm that likes to think of itself as a hundred and twenty-year-old startup. And we do want to disrupt ourselves. Yeah, that’s right.

Michael Krigsman: Well, I guess for a firm … any firm that’s been in business for a hundred and twenty years has gone through many changes. And so, can you elaborate right now on what are … What is the focus of that disruption at UL?

Christian Anschuetz: Well, you know, UL is just a fantastic company. I think you have to understand a little bit about us and let's just start with the "why," again. And so, the purpose of UL is … Our mission and our purpose is to make safer, more sustainable, and more secure … Well, [a] safer, more secure, more sustainable world. It’s a mission for humanity, right? And, we've accomplished that mission in the past by helping organizations test products to meet standards. Standards are … sometimes we write, and sometimes there are standards where we help participate in their development. And when a product passes the standard, that means that product is safe, it's sustainable, it is whatever … It's over the threshold for whatever reason that standard exists. And in many cases in our tradition business, that's about safety, right?

And yet, the thing that’s fascinating about us is that our mission is something other than testing. Our mission is about safety, sustainability, and security. And, nowhere in that mission statement does it say we just test. And, it’s very interesting, because the one thing that this company has, and that is very unique, and so we are a leader in the trust industry. We are trusted, we’re a third party, we’re hugely independent. Our integrity we hold incredibly dear.

And then, a firm's that know of us, and this is so many of them. Over seventy thousand manufacturers worldwide. That's our customer base. And, they know this about us. And, when we have the opportunity to engage in dialogue with them and say, "You know, what are the real, deep problems that you're trying to solve?" It often is bigger than simply testing their products out and help them get to market. There are way bigger opportunities for us to perhaps pursue. And, we're disrupting ourselves by thinking about ourselves in pursuing these higher order problems, as opposed to just the transactional testing activities that we do.

We’re a leader in science research. We spend more on r&d, at least to our knowledge, than anybody else in our industry. And, we are constantly figuring out and learning about these new and emerging technologies and all the while figuring out how we can maybe disrupt the status-quo as we learn more about everything from, you know, new and emerging alternative power sources, EV, hack for that case drones, new app trays and forays in cybersecurity. I mean, what makes the world safe today is very, very different than what made the world safe in the past.

Michael Krigsman: That’s quite interesting. So, your underlying mission remains constant: safety, security, sustainability; that trust that you were talking about. Your underlying mission remains constant. However, the way that you, can we say, deliver that mission; that’s the thing that changes and is disruptive. Is that an accurate way of saying it?

Christian Anschuetz: That’s wholly accurate. And, you know, that’s what’s beautiful about our mission, Michael. If you think about our mission, it’s really not bound by a lot, right? I mean, making the world safer, more sustainable, and more secure, that gives us a lot of room to maneuver, right? And in that maneuvering, it’s helpful [sic] we can maybe reinvent ourselves.

Michael Krigsman: Yeah. That's a very interesting way to think about it. I think many companies don't have that sense of constancy or consistency about their core mission. And so, the disruption becomes a more complete type of change. But, it sounds … But, so you have that constant mission and when you, therefore, are thinking about disruption, the execution, the delivery of that mission, how do you then go about it? How do you then think about that transition, that transformation?

Christian Anschuetz: That’s a big question. So …

Michael Krigsman: Yeah. It’s tough. These are tough questions.

Christian Anschuetz: Yeah, they’re really tough in that, you know … It depends on what we are … Let’s just speak in the abstract; let’s talk about any firm. It depends on, I think, what the firm’s trying to transition or transform itself into, right? And, I think that is, you know … I’m a big believer in “Start with why.” Our “why” is clear. Again, our “why” is a mission for humanity, you know. What we do then, and how we do it, is sort of that order. So you start with “why,” you go to “what are we trying to do,” and then we determine about how exactly we do that. So, it kind of depends on the “what” a firm is trying to disrupt themselves, and transform themselves into before you can probably, at least before I could […], perhaps say how you might go about doing it.

But, I want to circle back to a previous comment and part of our discussion beforehand. You know, so much of this has to do with, again, people, right? We have to be absolutely deliberate and focused on making sure we bring people along for the ride. It’s so, so critical, Michael. And, I will tell you: if you were to ask me some of the differences between like a traditional CIO or maybe a CDO role, they’re both important roles and certainly, one is not better than the other. They’re just different, right?

I think that CIO role is really more typically, typically more about internal, you know, transformation, efficiency; can be in a contemporary firm, internally. A CDO role has, you know, has to trust that a lot of that is happening internally and then project it externally, and bring the customers in. So I think the CDO role is typically, typically more of an externally-facing role. But regardless, when we are affecting like the transformation either within your firm, or you’re trying to create new values outside the firm, you really need to be considering people all along the way.

With regards to the CDO, because it may have a tendency to have an external impact which we change the internal dynamics and how the company sees itself, maybe even how – definitely how it runs itself, right? How it actually delivers this new value, start these new things.

The scope of the responsibilities tend bigger, right? So, one is internal, and one is maybe more external, at least in this definition, right? And, but the CDO role is really all-encompassing, at least in my opinion. And, you know, this is where the soft skills become even more important […] because you really are responsible for changing the external perspective on […], and then you have to change the internal perspective, perhaps, on exactly what the firm does to the value that it creates.

And so, again, I’ll go back to what I think the CDO role [is], and you actually manage transformations really involve people and organizational change management. It’s that saying – I’m stealing it from a contemporary of mine that said that, you know, “The hard results you get are really coming from the soft skills.” And I do believe that’s true for the CDO role. Both roles. All these leadership roles, for sure, but definitely the CDO role.

Michael Krigsman: So, in practical terms, how is your role; how is your work as Chief Digital Officer different from what you did and what you focused on as Chief Information Officer?

Christian Anschuetz: Well, it kind of follows that same path that I was just on. I mean, the CIO role is really much more internally focused around internal operations, and the CDO role is much more of a customer-facing, customer-discovery, customer-exploration role. Again, going in front of customers and saying, “Okay.” You know, what are the really big problems that you’re trying to solve? And doing this out of the context of how they normally see you as the firm. Remember, relationships are contextual, right? So if you and I only know each other in a certain context, and we keep talking about the opportunities to work together in new and different ways, it will always be influenced by the context in which we know each other. Is that a fair thing to say?

Michael Krigsman: Yes, of course.

Christian Anschuetz: Well, when you want to go into these customers, and you want to discover these bigger opportunities, you have to first pull yourself out of that context that you’re known for, and probably talk to someone that’s different from that customer, it doesn’t have that same context. I mean, the day-to-day context of how they do business with us today.

Now, this is why, you know, for the company now, I’ve been speaking in generalities, the company that I’m with now, UL, [has been] talking about … With the permissions that we have in terms of this leader and the trust industry, and this independence, high-integrity firm, we have the opportunity and the latitude, in so many cases, to move outside of typical interactions we have with our customers and engage in different ways; simply because we carry those traits with us. We’re the […]. And so, then we can engage in a different conversation and start having explorations around different, perhaps even bigger problems that we can solve for them. And, again, perfectly in conjunction and support of our mission and our purpose.

Michael Krigsman: So, when you talk about, again, going back to this consistency of mission and purpose, to what extent is this change and disruption affecting your underlying business model and the operations of UL?

Christian Anschuetz: Well, I think that has yet to be seen, Michael. I mean, we’re a relatively – I’m relatively new into this role, and you know, that said, the company has been working to improve itself and diversify itself in accordance with our customer needs for a long period of time. We had a very big disruption for any firm. You know, I sometimes wonder, I mean: When GE was, you know … decided to go to GE digital and really kind of create this industrial internet, this Predix platform and all that, when did they know that’s what they’re going to do?

Michael Krigsman: Yeah, what an interesting question. I mean, I think to … We’ve had a few people from GE on this show. We’ve had Ganesh Bell, who is the Chief Digital Officer for GE Power and Water – they have a different name, I think. And we had Linda Boff, who is GE’s Chief Marketing Officer. And, I think it became apparent to them that the market was changing, and GE needed to have a different kind of relationship with their customers. And so, they then re-thought, “Okay, what kind of technology platforms are they using? What is their business model? How are they selling? How are they pricing?”

And so, for example, instead of selling you a jet engine, they’ll … They own the jet engine, and they’re essentially licensing that jet engine to you, and you can pay on the basis of usage, obviously.

Christian Anschuetz: Jet engines, who would have thought. Right?

Michael Krigsman: Exactly. So, the question of how do you recognize when it’s time to change. I mean, at UL … And I want to remind everybody that we’re talking with Christian Anschuetz, who is the Chief Digital Officer of UL. And, I think everybody knows UL by the name “Underwriter Laboratories,” which was their original name before rebranding. And so, how do you, at UL, […] recognize, and when did you, and what are the signs that say, “Hey, we need to do something different?” It’s a really tough; it’s a really interesting question.

Christan Anschuetz: Yeah, and it's a tough question. I'm not sure if I can put exactly my finger on it and give your audience, your esteemed audience a really great answer. We do know that there is a need for us … our entire industry knows that we're in a position where we can be potentially disruptive, right? And the question is without knowing exactly what that disruption will be, there is a very simple question, and it's one that hopefully all companies, and all leaders are asking themselves: "Do we want to be the disruptor of ourselves, or do we want to sit by, sit back, and wait until someone disrupts us and then moves the initiative?" And, I think we … You know, UL I can speak for specifically, in this case, we want to keep that initiative. Now, why give up that initiative when we can own it?

Now, exactly what that disruption’s going to look like, exactly what will happen; we aren’t sure. Yet, we do know that the only way we’re going to seize the initiative is to act and to do something. And something is? Michael, hopefully someday we’ll talk and we’ll go, “Wow! That was crazy a year ago or two years ago.” Whatever it was. “How did you know you were going to get here,” and you know, we’ll probably reflect back and say, “Well actually, we didn’t, and here are the series of milestones we get,” and then suddenly, “This is the epiphany was this is what we’re going to do and this is how we’re going to change,” and create and entirely new category of business. Something out of what is our traditional industry which is TIC, testing, inspection, and certification.

Michael Krigsman: Well, it’s definitely not a straight line.

We have a few questions from Twitter. So, let's jump on those because they're pretty interesting. So …

Christian Anschuetz: To the best of my ability.

Michael Krigsman: To begin, Arsalan Khan asks, “It sounds like, to some extent, the CDO role is like a consultant to external clients.” I’m sure it’s not a consulting role, but in fact, there’s probably an element of that.

Christian Anschuetz: It’s actually a really great comment. And I think, you know, maybe I would have been pretty far from using the word “consultant,” just the way I think of that word sometimes. Umm, I do think there’s something to that statement, though, because one of the things we have to do that goes back to the whole context thing – I think one of the things we have to do when we’re talking to our customers, when we’re really thinking about the businesses we want to be in and the problems, the key, the problems we want to solve; we can recursively ask “why,” right? Keep asking, “Why are you doing this? Why are you having this problem?” I know “why” is a personal word. You know, “What makes this an issue for you,” until you finally get to, you know, the root cause; you know, the root problems that, you know, the company’s real customer base is experiencing.

You know, our perspective. They engage us for many, many different things. UL's a hugely diversified company and very different than it was a number of years ago. The core of our business is still we test the product against standards and when they pass, we help issue a mark, we tell the agency we're testing for that it met the performance criteria, whatever, right? But when you start asking, "Why do you need the tests," and "What makes you require this certification," until you keep asking for […] It's the organization's turn to try understanding that there's just a general lack of understanding with regards to firms of what they have to do to really, to safely, in accordance with compliance and regulations, put their products in a specific market, right? And testing is a byproduct. That comes down to the “how” you actually do it.

But, you could wind back and keep asking why until you get to the whole … a totally different problem statement that if you attack the “there” or the “why,” then what would you do today could be, you know, it could be relevant; it could be relevant in a different way. I suppose it could be rendered […] and I think that’s unlikely in this particular scenario. But, I think there are the things we can resolve, but you have to …

The consulting question is good, because you have to go in there, and you have to do essentially customer discovery sessions. What are the real pain-points? Other than the context that they know you and that you know them?

Michael Krigsman: And, Arsalan Khan has a very interesting follow-up to the point that you were just making. And he says, “So, yes, it’s good to know… We have to know customer pains and their concerns, and so forth. But, if we only listen to our customers, then Ford would have made just faster horses, not cars.”

Christian Anschuetz: Well, that comes down to the whole design theory, right? You can go and you can listen to just what they say and that’s the Ford story, you know, “Instead of building a car, would they have built a faster horse?” But, what the customer’s really saying when you recursively ask “why” enough is that they actually needed to get from point A to point B faster. They had to do it without a certain amount of maintenance. They didn’t like using; I’m totally making this up [laughter]; they didn’t want to wagons, they needed something with a certain amount of capacity. They didn’t want to sit side-by-side with somebody. In other words, the question might have been more about, or the challenge might have been more about diversity in mobility than it would have been about a faster horse. And if you listened enough, you might have heard something different than a faster horse, too.

I totally get where that statement’s coming from now. I mean, I get it, and I believe in that. But, I think when you listen to them, you have to listen to what they say, you have to really understand what they mean. Those can be two different things.

Michael Krigsman: That’s a key point. So, it’s not just listening to the words, but it’s trying to divine being empathetic, I guess you could say; being empathetic to what do they really want? Listening recursively, as you were describing earlier?

Christian Anschuetz: Yeah. What do they want, and what do they really need? And if you look at some of the best disruptions, I mean, you’re talking about things that people didn’t even know that they wanted. I love the example of Uber. I know it’s kind of tired in so many ways, but just think about it. People just always took for granted that you had to stand sort of dangerously close to the curb and wave your hand waiting for a cab, and by God, hopefully, it wasn’t rush hour, or it wasn’t raining, you know? Or otherwise, you were kind of out of luck. And that, though, was just the way it was, right? Of course that’s just the way it is, it’s how the business works, that’s how … We got rides from Point A to Point B until someone said, “Wow! You know, there’s another need there.”

And actually, did they have to ask the customers or did they just have to observe? And, I think that's observation is key, and that kind of goes to that second thing. You can listen to what they say, but you've got to really follow the meaning. And, the meaning can be divined by any number of different ways, but observation is certainly one of them. I think it's probably the key one.

 Remember, most of what we get from people is less about the words they say, it’s about how they say ‘em, right? It’s the nonverbal cues. And then just if you believe that, right? […] And there’s all the science to back that, that makes it very clear. If you back that, and you really kind of add, then, the sort of subtle, nuanced, observation piece and you say you observe their behavior, well that’s when you get into design thinking and you start understanding why some companies are just better at disrupting than others. They do more than just listen to words.

Michael Krigsman: It’s quite interesting: design thinking as a systematic means to do that kind of deep listening that you’re describing in order to get to the surface of what the customer ultimately really cares about.

Christian Anschuetz: Yes.

Michael Krigsman: We have another interesting question from Twitter. Marc Orelen asks a burning question that I think is on all of our minds, which is: Why do we need a Chief Digital Officer? Why are these … Why is the CIO and CDO role separate? And he says, “wouldn’t the ideal be a customer-focused CIO?”

Christian Anschuetz: I think that’s a great question and a great point. So, you know, it’s so funny. I got the CDO role just a short while ago. I’ve been operating in the capacity for a while as the CDO. But, I’m still the CIO. So, what’s the difference, right? No sooner than I got the role that I stumbled on an article by Forbes. It was January … It was this year, I think, in January. Forbes was saying, “Say goodbye to the CDO role.” And I read it, and I’m like, “Wow. That stinks. I just got the job.” [Laughter]

But the point of it was, and it was a really good point, is that if firms stop thinking about there being business strategy and digital strategy, and it’s just a contemporary strategy and the businesses are run with a very contemporary mindset, and it’s very agile around technology; it’s very inclusive of people and their involvement in technology, then you don’t need a CDO.

Michael Krigsman: So, I’ve heard people say that eventually, the CDO role may go away as the digital mindset, the digital understanding, kind of defuse through an organization; that the CDO role, we could say, is a transitionary role.

Christian Anschuetz: I think that’s right, you know? And I’m less than, I’d say, some sort of expert in this. I do think that’s right, though. But, let’s be honest with ourselves, and look about at the firms that we all know. And I’m speaking in general here. I think that having a CDO role in a company like; I’m just picking an example; like a Google, for example, probably makes a little less sense than a company like, say, maybe like a Ford Motor company, right? Both fantastic companies; and by the way, I drive Fords; love Fords.

But, you know, I think that there is this transition, as you said, and as firms … Firms don’t just overnight become this sort of digital entity, right? It took Ford a while to understand that they didn’t just do cars; that they did mobility. And then understanding what it takes to be mobile players in the digital world is still something that they’re embarking on. And so, having a CDO role that is sort of ushering in that understanding, this sort of contemporary culture, this contemporary understanding, this contemporary application to their business I think takes a certain amount of time.

So, counter to the Forbes article, which said "Say goodbye to the CDO role," was another article by McKinsey that talked about the CDO as a transformer-in-chief. And, you know, I prefer the latter article to the former. By the way, they're both great articles. But I think that's why you actually need the CDO role, at least right now, because I think we're in a state of massive transformation. And again, every industry is going to get disrupted and since we're all rather unclear as to how we do it; I mean, the very basis of why we're having this conversation, the questions you're asking. How are you going to know? How are you disrupting yourself? What are you doing about it? Because most of these questions are very difficult to answer for most firms. I think that's why the CDO role exists.

Michael Krigsman: Well, as you said earlier, it’s very difficult to disrupt ourselves as individuals, and it’s very difficult to make the changes needed to disrupt ourselves as companies.

We have another really interesting and, I think a pretty deep question, actually, from Sal Rasa, who says, “Is the CDO role a community relationship responsibility, a community relationship management responsibility, designed to inform change management decisions?”

Christian Anschuetz: I think that’s a big part of it. I really do. I go back to the statement about the people, and not leaving the people behind. That is all about change management, and I think that that is a really big part of it. Now, that said, there is an external portion of it that goes back to these adjacencies that we talked about. You have to be bringing the people on, but you also have to be an explorer, and you have to be utterly unafraid to go into new and different areas.

Jeff Bezos, I love one of his quotes, and he’s a very quotable person, right? He made a comment that’s a quote, and I think I’m attributing this properly. If I’m wrong, I apologize, but he said that “At Amazon, we're not afraid to be misunderstood." And, I think what's behind that quote is that they are okay to go in new and different areas, and have a lot of people scratch their head and go, "Why the heck are they doing that?" But they're doing it as part of their exploration. Now Louis and Clark didn't make a beeline directly from the east to the west. It wasn't a perfectly straight line and we made that comment earlier, right? You know, a lot of people that I'd say, "Well why did they scale that mountain?" Well they actually didn't know they had a choice, or it looked particularly great, or perhaps, it gave a whole new vantage and a whole set of opportunities that lay beyond it.

I think that there’s people aspect to the CDO role, I think that’s critical, I think this exploration portion of it, and bringing the people along in that exploration; again, making them potential disruptors themselves is actually very, very critical.

Michael Krigsman: […]

Christian Anschuetz: And yet again, another really [good point], you do have … I remember when we were starting this conversation, you said that "Christian, just think we're going to be sitting here talking around a table with a bunch of very, very smart people." You're making that comment, and clearly, the audience and the questions they're asking is making your statement very, very true.

Michael Krigsman: Oh yeah. Now the audience of CxOTalk is quite amazing.

Now we have another really interesting comment from Shelly Lucas, who is with Dun and Bradstreet. And, she makes the comment that she thinks you are ahead of your time as a Chief Digital Officer because many digital leaders are focusing on the science rather than the people on the culture. And I interpret that to mean not just the science, but focusing on the technology platforms that enable this, as opposed to the people in cultural issues.

Christian Anschuetz: Well, thank you. You said it was Shelly?

Michael Krigsman: Shelly.

Christian Anschuetz: Well, thank you, Shelly. That's very kind. You know, I was in IT long enough to know, I mean, IT could implement the best system, and you fail to get the people on board with it, and you're going to have an adoption issue, you're going to have, well, we all know the stories, right? You can implement the best system and … By the way, a little IT joke: How do you make people love their old system? Implement a new one, right? And that’s because if you fail to bring [Laughter]… It’s true! It’s so true. It’s a joke, but it’s totally true.

Michael Krigsman: [Laughter]

Christian Anschuetz: Ummm.

Michael Krigsman: Spoken by somebody with a long history in IT. I’m sorry, I didn’t mean to interrupt. Please, go ahead.

Christian Anschuetz: But it’s totally true, and you know, so I learned at a relatively young age, and I’ve been trying to get better at it, and it is a bit of a struggle. But I’ve learned that you can only get down the path as far as you want to go when you have a lot of people in support. So, you’ve got to bring them along. And I go back to this topic of leadership at the end, but what is the obligation of leaders but to create a compelling vision and inspire people to fulfill that vision? And if you are unable to do that, then how would you ever really help to disrupt yourself and disrupt an entire industry? Because you're not going to disrupt it with just technology. You're only going to disrupt it with your people plus some technology.

Michael Krigsman: So the technol- … I mean, the way I talk about it often is the technology provides enabling capabilities, right? It lets you do things that you couldn’t have done before. So, for example, a software platform that lets you collect data. Well, you need, if you’re a digital company, you’re going to be relying on lots of data. Merely having that technology platform doesn’t mean that anybody is going to use it or do anything valuable with it.

Christian Anschuetz: You’re a wise man. That’s exactly right. How many great technologies were just simply the wrong technologies even though they were perfect, but they came out too soon. They came out too soon, so they were still wrong, right? And so, you know, was it because the technology was at fault? Was it because society or the audience was unready for it, or was it a combination of the two where the technology was right for too little time spent in making the audience understand why this was actually, you know, a really great value. I think there are probably a bunch of different answers depending on use-case to look at.

Michael Krigsman: So, how do you convince the organization that change that it needs to undertake; this kind of change; and then, can we go back to UL specifically and talk about the nature of this change process at UL?

Christian Anschuetz: Uhhh, sure. So, what’s the question you kind of want me to dial in on there? Is it change process specifically you want?

Michael Krigsman: Well, I think the … And by the way, we have about five minutes left, so as we wind down, what are the lessons or the takeaways about driving disruption; self-disruption; disrupting your own organization? How do you even begin?

Christian Anschuetz: Well, Michael, I think you begin and you might be surprised to hear this from a company that prides itself in integrity and independence. It starts with transparency. You know, we ask our colleagues and they’re getting better at this, and we’re just really kind of starting off. Our colleagues, you know, what are the directions that they think we should go? What is the company that we can, and we should be? Again, unconstrained by anything other than our unique mission and purpose; again, […] for living and working environments, right? And our imagination. What could this company be? Getting them involved. I’ll tell you that’s what I think is one of the most key things I can do. Again, I know it’s soft, it has very little to do with inventing some whiz-bang, high-tech solution, but it’s been an important lesson for us, I think, is to involve our staff.

I think the other thing is, again, that thing we talked about already which is changing the context of our conversations with our customers. They know us in a certain context, they give us permissions to have different conversations with them than we traditionally do, so seizing those permissions, having a different power station, and really try and find the sort of root of desire, or the problems that plague them. And, that you have the opportunity to help them address and create new value for them and that portion of the company […]

Michael Krigsman: What about the role of senior management? You know, you’re talking about the grassroots side, but don’t you have to also go from the top down as well?

Christian Anschuetz: Well, you know, again, the senior management, that leadership, it's the vision, it's inspiring people to follow that, and then, of course, there's modeling, right? There's an old … You know, I was in the Marine Corps, and the Corps taught you a lot about leadership and this concept of leading by example. And allowing yourself to be less than perfect; allowing yourself to fail and even celebrating this failure, so getting a management team on board is saying, "Hey, we're going to explore," and some of our exploration – perhaps even the majority of our explorations – are going to end in dead ends. Being accepting of that I think is critical, because that unfetters your organization. It makes them less scared to move in those areas with these roads less traveled, and become potential disruptors themselves. Because, if you're afraid that a dead-end is going to be a blemish on your career, on your history, I think that you're actually stifling yourself. I think you have to free up, again, you have to free up your people, and to the best of your ability, just free them up from that particular fear, and help them have courage. Well, there will be some fear, but a little less fear, a little more courage, and I think senior management's critical.

Michael Krigsman: Well, I guess that’s a … one of the most important and fundamental lessons. We have just a minute left, and Christian, I know that you are a vet, and I know that you’re very supportive of vets, and would you like to take a minute and tell us about some of your activities in relation to that?

Christian Anschuetz: Aww, thank you. Thanks, Michael. Yeah, I mean, just a quick plug. I'm part of an organization called Project RELO. It's a fascinating organization that uses transitioning veteran instructors to teach corporate executives the art and science of leadership. And, that's done in a very unique fashion. In partnership with the United States military, we do pseud-military operations with this collective of executives and veterans and build a deep understanding that hiring our veterans is more than a social good, it's simply good business. If you want to learn more, check out projectrelo.org.

Michael Krigsman: Project reload, r – e – l – o – a – d-dot org.

Christian Anschuetz: Uhh, Project r – e – l – o-dot org. RELO.

Michael Krigsman: Got it! Okay! Check out projectrelo.org.

We have been talking with Christian Anschuetz, who is the former Chief Information Officer and now the Chief Digital Officer of UL, which everybody knows as Underwriter Laboratories. Christian, thank you for taking the time to be here with us today.

Christian Anschuetz: Thank you so much.

Michael Krigsman: We have more shows coming up, and they are great shows. Next week, we’re speaking with the CEO of Coursera, and he used to be the president of Yale University, so that’s going to be an interesting one. Check out cxotalk.com/episodes. Thanks everybody for watching, and we will see you next time. Bye-bye!

Automation, AI, and Business with Michael Chui (McKinsey) and David Bray (FCC)

Dr. David A. Bray, Chief Information Officer, Federal Communications Commission
Dr. David Bray
Federal Communications Commission
Dr. Michael Chui, Partner, McKinsey & Company
Dr. Michael Chui
McKinsey & Company
Michael Krigsman, Founder, CXOTalk
Michael Krigsman
Industry Analyst

Data and automation have the power to transform business and society. The impact of data on our lives will be profound as industry and the government make greater use of techniques such as artificial intelligence and machine learning. Explore this important topic with two world experts.

Dr. David A. Bray began work in public service at age 15, later serving in the private sector before returning as IT Chief for the CDC’s Bioterrorism Preparedness and Response Program during 9/11; volunteering to deploy to Afghanistan to “think differently” on military and humanitarian issues; and serving as a Senior Executive advocating for increased information interoperability, cybersecurity, and civil liberty protections. He completed a PhD in from Emory University’s business school and two post-docs at MIT and Harvard. He serves as a Visiting Executive In-Residence at Harvard University, a member of the Council on Foreign Relations, and a Visiting Associate at the University of Oxford. He has received both the Arthur S, Flemming Award and Roger W. Jones Award for Executive Leadership. In 2016, Business Insider named him one of the top “24 Americans Who Are Changing the World”. He is Chief Information Officer of the Federal Communications Commission.

Dr. Michael Chui is a partner at the McKinsey Global Institute (MGI), McKinsey's business and economics research arm. He leads research on the impact of information technologies and innovation on business, the economy, and society. Michael has led McKinsey research in such areas as Big Data, Web 2.0 and collaboration technologies, and the Internet of Things. Michael is a frequent speaker at major global conferences, and his research has been cited in leading publications around the world. His PhD dissertation, entitled "I Still Haven't Found What I'm Looking For: Web Searching as Query Refinement," examined Web user search behaviors and the usability of Web search engines.

Download Podcast

Automation, AI, and Business with Michael Chui (McKinsey) and David Bray (FCC)

Michael Krigsman: Welcome to Episode #219 of CxOTalk. I'm Michael Krigsman, an industry analyst, and your host. Today, we have such an interesting show. We're going to be talking about data, automation, machines, machine learning, AI, and the role of all of this in business today and what it means for the future.

Our guests are David Bray, who is … Well, David has been on the show a number of times; and David, why don’t you introduce yourself?

David Bray: I am David Bray, I refer to myself as a digital diplomat and human flight jacket, otherwise known as Chief Information Officer at the Federal Communications Commission. I should also mention that I'm an Eisenhower Fellow to Australia and Taiwan, which means I met with their government and industry leaders in both countries about what their plans are for the Internet of Things; and then briefly just mention how ten years ago, I met Michael Chui at the Oxford Internet Institute. We were actually working together on how one could best do what was called "distributive problem solving": how you could bring human and technology nodes, reach better outcomes and organizations, and so that's why it's so great to be here with Michael again talking about artificial intelligence.

Michael Krigsman: Fantastic! And, I met Michael Chui, who is with the McKinsey Global Institute through David. And, just after we arranged for Michael to be on this show, I saw him on CNBC in the most interesting segment. So, Michael Chui, welcome to CxOTalk, and please tell us briefly about yourself and what you do at McKinsey.

Michael Chui: Sure, delighted to. Like David, I was once a CIO of a public sector organization, but it was much smaller. It was a municipality. But now that I’m a partner in McKinsey Global Institute, I need some of our firm’s research. It’s part of the larger McKinsey company, a management consulting firm. At least some of our research on the impact of long-term technology trends were including some of the distributed problem-solving that David mentioned, but also artificial intelligence, robotics, and AI.

Michael Krigsman: Michael, you have been studying data analytics. You came out with a very rich report; a lengthy, deep, and interesting report; last December. Please share your view with us on data and digital disruption.

Michael Chui: Well, one of the things that we've been studying for quite some time is the potential impact of using data and analytics to change organizations and society. It's said, for instance, disrupting either industries or organizations … Our first publication was actually five years ago. It's the report on Big Data, and to an extent, the report that we published in December is a sequel. It's a good thing in movies, and sometimes it's a good thing in research, too.

One of the things we identified back in 2011 was the tremendous potential of using data and analytics to really change the game. And we looked at the public sector in Europe, we looked at health care in the United States, we looked at manufacturing and retail, and location-based services and said: In each of these domains, if you use data and analytics, either as a basis of competition, or as a way to increase the efficiency or effectiveness of what you’re doing, or change business models; we saw the potential for all kinds of good things to happen, both for the organizations themselves, whether or not it’s retail or they’re trying to sell more, but also a health care provider trying to improve healthcare outcomes of a country or an organization, etc.

We saw all kinds of potential. We said, “That could potentially happen within ten years.” And five years on, we said, “Let’s see how things are going. Let’s see how much value has been captured out of the billions of dollars of potential value.” And by the way, that’s not just profits, it’s improved healthcare outcomes, better services provided to citizens, etc.

And quite frankly, what we found in this last piece of research amongst the many findings that we discovered: When we had that sort of “Look into the rear-view mirror,” we said that the report card was actually quite mixed. Some organizations have really expanded their ability to use data and analytics, and some sectors have moved farther than others. And to be frank, those are sectors in which you had, in many cases, a digital native competitor you had to compete against. So retailers had moved along farther.

There are other sectors and organizations which, quite frankly, have been, on average, farther behind, capturing only about 10 or 30% of potential value. Unfortunately, some of those have been public sector. But again, there is great individual variation.

The other thing we found is the spread in performance between not only sectors but individual organizations, and I know David's been doing terrific things as the US FCC; and again, some of the organizations that have been doing the most have really extended their lead versus median or even lagging organizations.

Michael Krigsman: So, David, your view is a public policy view, especially when you wear your Eisenhower Fellow hat, so any thoughts on this from a public perspective?

David Bray: So, I agree 100% with what Michael said, that different sectors are embracing the opportunity provided by data analytics, artificial intelligence; and that it does seem to be, for those sectors where there is a digital native incumbent or an organization that is either a startup or already present, that has embraced the digital regime; that pressures the rest of those organizations to move along.

The public sector, on the other hand, you have the challenges of not just in the United States but around the world, governments are facing pressures to do more with less. Here in the United States, we had sequestration; but also talking to Australia, when I met with them, their public sector leaders were facing the possibility of a recession. In Taiwan, the economy was not growing as fast as it had previously. And so, here you have shrinking budgets, but at the same time, you are being asked to transform how you do your work; and so, it’s the challenge of how do you leapfrog from legacy investments in IT? You can’t be a startup, because you have to keep things running, and you have to keep on serving the public. At the same time, if you keep on doing things on-premise with legacy IT that’s on average five to ten years old, you won’t be able to get where you need to go.

And so, at the FCC, when I arrived three years ago, it was an interesting situation where they had nine CIOs in eight years, which I said was a great sign for CIO #10 that things are just going great; and I quickly surveyed that they had more than 207 different IT systems that were on average more than ten years old - we even had some that were approaching nineteen or twenty years of age - and they were consuming more than 85% of our budget, just to maintain those systems.

And so, that's where I said, "In two years or less," at the time, it was rather ambitious [proposal], and I think I had a lot of people that were a little surprised, "In two years or less, we want to have nothing on-premise at the FCC. We want to go straight to either public cloud, or a commercial service provider, because you cannot capture the benefits of data analytics, artificial intelligence, the Internet of Things, and making sense of the data that's coming from them if you are still tied to legacy IT."

And the good news is, two years later, we did it. We reduced our spending from 85% to 50%, and in a lot of ways, it was just setting the scene for getting ready for making sense of all these widespread data sources, making sense of what can be brought in from machine learning and AI.

Michael Krigsman: Michael, how do organizations make the decision to invest, and where should they invest? How do AI and machine learning come into play in a practical sense, as opposed to all the media hype or science fiction? How do we become practical about it?

Michael Chui: Well, a few things. I think one of the things that have happened over the years that we've been doing research on, as well as more organizations,  have started to understand the potential of data analytics, and then those applications to these techniques - AI and machine learning - is that awareness has certainly grown. In fact, amongst what's called "executives," whether they be public sector or private sector, it's actually … They're starting to understand that, in fact, data and analytics are either becoming a basis of competition or a basis of providing the services and products that your customers, citizens, and stakeholders need.

Thing is, we've reached that level of awareness, but as David and I have talked about in terms of actually what has been able to be captured, what sorts of value has actually been able to be generated, comes about for a number of reasons. And I think as we looked at organizations all around the world if you ask and executive or leader there, "Are you thinking about data? Are you thinking about analytics, and are you doing anything?", and almost everyone says, "Yes." And many would say, "Oh, we've got a successful pilot where we conducted this experiment. We've invested in the technology, we've invested in hiring some data scientists and analysts, etc. for software and hardware; on Cloud, or on-prem, what-have-you, while doing that transition."

What we found oftentimes: While there are oftentimes real technology challenges which take real investment, and time, and energy; and as a technologist myself; it’s very interesting to talk about those things. Oftentimes what we find the real barrier is, is around the people stuff. It is how do you get from an interesting experiment where there’s a business-relevant insight? We could increase the conversion rate by X percentage if we actually used this next product to buy an algorithm and this data, we could reduce the maintenance costs, or increase the uptime of this whole good. We could, in fact, bring more people into this public service because we can identify them better.

But, getting from that insight to really capture value at scale, is where we've started to find oftentimes organizations are either stuck or falling down. And, it really has to do with how do you bag that interesting insight, that thing that you capture, whether in it's in the form of some sort of machine learning algorithm, whether it's other types of analytics, into the practices and processes of an organization, so it really changes the way things operate at scale? To use a military metaphor: How do you steer that aircraft carrier? It's as true for freight ships as it is for military ships. They are hard things to turn.

And that organizational challenge of understanding the mindsets, having the right talent in place, and then really changing the practices at scale, I think that’s where we’re seeing a big difference between those organizations who have just reached awareness and maybe done something interesting, ones who have radically changed their performance in a positive way through data, analytics, and AI.

Michael Krigsman: I want to remind everybody that you’re watching CxOTalk. And, right now, there is a tweet chat that is taking place on Twitter, of course, using the hashtag #cxotalk. You can ask questions to our guests directly and they will answer.

David Bray: [Laughter]

Michael Krigsman: [Laughter] Well, we hope they’ll answer.

So, David Bray, Michael Chui was saying that the barrier to adoption is the people. Now, in the realm of AI and machine learning, how does this particular issue play out? Is there anything that’s unique about AI and machine learning that we need to be considering when we talk about adoption and the proliferation of these technologies in the enterprise in a meaningful way?

David Bray: So, that’s a really great question. First, I would say that I emphatically agree with what Michael was saying, that the real secret to success is changing what people do in an organization, that you can’t just roll out technology and say, “We’ve gone digital, but we didn’t change any of our business processes,” and expect to have any great outcomes. I have similarly seen, both in the private sector, and in the public sector, here in the United States, here in the Federal government, but also in other countries like Australia, Taiwan, and other places in Europe, where they’ll do experiments that are isolated from the rest of public service; and they say, “Well look, we’re doing these experiments over here!”, but they’re never translating to changing how you do the business of public service at scale.

And to do that requires not technology, but understanding the narrative of how the current processes work, why they’re being done that way in an organization, and then what is the to-be state, and how are you going to be that leader that shepherds the change from the as-is to the to-be state? For public service, we’re probably lacking conversations right now about how to dramatically deliver results differently and better to the public.

Now, for artificial intelligence, in some respects, it’s just a continuation of predictive analytics, a continuation of Big Data, it really is nothing new in terms of the fact that technology always changes the Art of the Possible, this is just a new Art of the Possible. I do think there’s an interesting thing in which it could offer a reflection of our own biases through artificial intelligence. If we’re not careful, we’ll roll out artificial intelligence, populating it with data from humans, [and] we know humans have biases and we’ll find out that the artificial intelligence itself, the machine learning itself, is biased.

At the same time, we could actually use it to say, “Look for biases and past outcomes, past decisions, past performance with this organization, and let us know where things weren’t exactly either as equitable or as beneficial as it could be. So AI could either A) be a dangerous tool where it just reflects and augments and amplifies human biases, or it gives us a chance to look in the mirror and say, “You know, you’re being biased when you make these decisions or these outcomes.” And I think that’s a little bit more unique than just a predictive analytics bias or Big Data.

Michael Krigsman: Michael, let’s drill down now into AI a little bit more deeply, and machine learning. In your research, what are some of the business areas that are today most well-suited, and where do you see this going?

Michael Chui: So, we did do some research in terms of trying to understand where there’s the greatest potential from some of these technologies. Like one of the interesting things that we discovered was … Well first of all, our hypothesis was as we look across about ten different industry sectors and then a dozen different types of problems in each, we expected, quite frankly, that much as we find for other techniques, the most of the value will be concentrated - 80% of the value coming from 20% of the problems, or something along those lines.

When we surveyed about 600 different industry experts, every single one of those problems we identified, at least one expert suggested it was one of the top three problems that machine learning could actually help improve. And so, what that actually says is the scope for potential is just absolutely huge. There’s almost no problem where AI and machine learning potentially couldn’t impact and improve performance.

A few things that come to mind: One is a lot of the most interesting and recent research has been in this field called “Deep Learning,” and that’s particularly suited for certain types of problems with pattern recognition, oftentimes images, etc. And so those problems that are somewhat similar to image recognition, pattern recognition, etc. are some of those that are quite amenable and interesting.

So again, in terms of very specific types of problems, predictive maintenance is huge. The ability to keep something from breaking, so rather than waiting until it breaks and then fixing it, the ability to predict when something's going to break not only because it reduces the cost, but because it's cheaper, in fact, to try and keep something from breaking rather than paying someone to fix it. The more important thing is the thing doesn't go down, so if you bring down a part of an assembly line, you bring down the entire factory or oftentimes the entire line. So being able to avoid that cost is the same kind of thing as a jet engine on a wing of an airplane, etc.

And so, to a certain extent, that is an example of pattern matching. When you have all of these sensors, which are the signals that actually reflect that something’s going to break and you should go and do some predictive things? And so, we find that across a huge number of specific industries that have these capital assets, whether it’s a generator, a building, an HDC system, or a vehicle, where if you’re able to predict ahead of time before something’s going to break, you should actually conduct some maintenance. That is one of the areas in which machine learning can be quite powerful.

But the other thing is, again, taking this idea that you have one problem with one area, you look for analogous problems. If you take health care as another case of predictive maintenance on the human capital asset, then you can start to think, "Well gosh! I have had the Internet of Things, I have sensors on a patient's body; can I tell before they're going to have a cardiac incident? Can I tell before someone's going to have a diabetic incident, that in fact, they should take some actions which could be relatively less expensive, and less invasive, than having that turn into an emergent case where they have to go through a very expensive, painful, and urgent care type of situation? Again, can you use machine learning to be able to do prediction? And those are some of the things that we're starting to see in terms of problems that can potentially be better solved by using AI and machine learning.

Michael Krigsman: David, a practical question from you, and then we have a few questions from Twitter. So, as a CIO, how much are you thinking about AI, machine learning, and predictive analytics in the operations of your organization?

David Bray: So, right now, I actually have an ask out to all the eighteen different bureaus and offices of the Federal Communications Commission to identify a bureau or office challenge or problem involving the public that they would like to have machine learning and artificial intelligence brought to bear. Woe be to the CIO that tries to force a solution onto a bureau or office that's not ready for it yet. And so, this is trying to see if they are receptive if they can spot something. And maybe it is identifying where we can provide it, as just Michael mentioned, preventative maintenance of services that can actually benefit the organization and benefit the public, making sense of comments that we receive.

We did actually, back in 2014, make public comments we received on a specific issue that involved four million comments, with the idea that we actually wanted to allow the public to use tools to bear, to make sense of them: sentiment analysis, understanding what was either a "for" or "against" proposition. And I think in some respects, public service has the opportunity that it's not necessarily in competition with any organization, we could actually make our data available; recognizing we need to protect privacy; but once we protect privacy, making that data available to the public sector and the private sector to make sense of it. We don’t have to do it by ourselves. And so, I think the opportunity for artificial intelligence and machine learning is what are those things - it’s a little bit harder at a national level - that will benefit the public?

I think a lot of things are going to happen first in cities. I mean, we’ve heard talk about smart cities. There, you can easily see where if you can actually have preventative maintenance on a road, or better providing of power, and actually monitoring it to avoid brownouts. I think actually the real practical, initial, early adopters of AI and machine learning are going to happen first at the city level in some respects, and then we’ve got to figure out how we can best use it at the federal level.

Michael Krigsman: We have some questions from Twitter, and one is from Bob Russelman. This is a really interesting one, and he’s asking about the impact of automation and AI on human employment. And I think when we talk about AI, robots, and autonomous systems, this is one of the big questions that come up. So Michael, what are your thoughts about that?

Michael Chui: Sure. So, about a month after we published this Age of Analytics report, and about one month ago, we published another report - and by the way, these are freely available on the web - which really looked at the potential for automation to affect employment and the global workforce.

So, a couple of things: One of the things that we did in this research was to look at not only every occupation, because we think it's quite rare that, in fact, you'll be able to remove someone out of a job and put an AI or robot in there that will do everything that they did. They actually conduct a number of different activities in any job. So, we looked at things at the level of individual activities, and scored them against eighteen different capabilities that could potentially be automated - everything from fine motor skills, navigating the physical world, cognitive tests such as problem-solving, sensory activities, and even understanding and producing natural language. One of the highlight findings is that about 50% of all the activities we pay people to do in the global workforce could potentially be automated by adapting currently-demonstrated technologies; which sounds scary, but … Wow! Fifty percent of the things that we pay people to do. But that's not going to happen overnight.

And again, part of our analysis was understanding what those timeframes might be. Now, we can’t predict the future, so we developed some scenarios with really wide bands around them. When you think about the requirements, I said theoretically, 50% of these activities could be automated. Really, it takes time to integrate those capabilities technologically and create individual solutions. And then beyond that, you have to create a business case, because what I didn’t say was this would cost less than it does for a person to do it. So again, compare the cost to develop and deploy these technologies against the cost of using people for doing the same things.

And then finally, the natural curve of adoption of any technology, which often takes eight to twenty-eight years after the time that something's commercially available to the time it reaches a plateau in its eventual full adoption, then it might take something like forty years or ten presidential terms. At least, that's the middle point of all scenarios that we modeled out before 50% of current activities are even automated.

What’s interesting is that level of change in what people do is not unprecedented. If you look back at 1900, about 40% of the US workforce was about [...] and agriculture, and about seventy years later, about 2% was. So what that actually says to us is that in fact, we need to find new things for people to do, as automation comes into play, so that people are complements of the work that machines are doing. And in fact, we need that quite badly because of aging. We need everybody working plus the robots working to have the economic growth that we need.

It's been done before. I'm a sunny Californian, so I'm hoping this can be done, but it will require real effort to make sure we actually find new activities for people to do and find ways to make sure they get paid to do those new activities as machines work alongside human beings.

Michael Krigsman: So very clearly, then, this technology has the potential to drive a major social upheaval! Michael, that’s essentially the implication of what you’re saying!

Michael Chui: Yeah. I think the question is what word do you want to use? I think “shift” is a different word than “upheaval”, and a different word than “disruption.” But, what we are saying is that all of us, because again, it’s not 50% of jobs, it’s nearly 100% of jobs will have a significant percentage of their activities that will change. How can we all have the flexibility, have the training, have the retraining, so that we’re enabled to do new things as we help use machines to improve our productivity?

David Bray: I would like to add to what Michael is saying because I agree. It really is about augmenting human capabilities as opposed to replacing human capabilities. We almost should be talking instead about artificial intelligence, we should be talking about augmented intelligence. And as we talked about earlier, what machine learning and AI is really good at are things that are pattern recognition and repetitive in nature.

So in some respects, I don't know what we humans want to do to things that are repetitive, rote, that are the same thing over and over again for hours at the end of the day. What this is really doing is freeing us up to focus on those jobs that are going to be nonroutine, where there is no pattern that is present; or even, in fact, where the machine tips and cues us and say, "I've identified something that fits outside of the pattern. You should pay attention to it. I don't know why it's happening, but that's going to require a human to take a look at it." It's almost freeing us up to focus on those things that require more creativity.

Now, that said, it does require us to ask interesting questions, which is 1) What skills should we be training; not just current students in school, so they can be ready for this future of working together in augmented capabilities, but also retraining these same workers so that they can be ready for this future that is not necessarily going to be rote and repetitive work for them, but instead is going to be about, “What is the non-routine work? What is the diagnostic work when a machine tips and cues you to pay attention to it?” We really do need to look at what the future is of pairing humans plus machine, working together, and what does that look like, as well as what new patterns of work will emerge as a result.

Michael Krigsman: And what about the ethical issues of this? It's so fascinating to me because we've got essentially a technology, or set of technologies and techniques that very quickly have cultural, social, and educational implications; and therefore, that immediately takes us down the ethical pathway. So, what about that?

David Bray: So …

Michael Chui: [...]

David Bray: Go ahead, Michael.

Michael Chui: Go ahead, David!

Michael Krigsman: [Laughter]

David Bray: You first, Michael!

Michael Krigsman: [Laughter]

Michael Chui: [Laughter] A couple of things. You know, let me just build on something that David said before in terms of the need for augmentation. You know, one ethical issue to bring to bear is what is it that we’ll need to make sure that the next generation actually has better lives than this generation? For the past fifty years in the biggest economies in the world, about half of the economic growth we’ve seen has come about because of increases in employment, about half of it because of increases in productivity; the ability to use machines and other management innovations to do more with fewer hours.

In the next fifty years, we’re basically going to lose half of our sources of economic growth. Why? Because countries are aging. The US is aging. China’s workforce is actually decreasing in size, and that’s a billion-and-a-half people. In Japan, it’s already happening. And so, unless we have everybody working, plus the robots working, we simply won’t have enough economic growth for the next generation to have better lives than we do. And so, that’s an ethical question already. It actually suggests we need to accelerate the use of automation. But that means that I think, Michael, [it’s time to] to get to your question. As David mentioned before, we have had, and this is true not only for AI but all technology, we have had a lot of our values and the technologies that we developed.

So you know lots of people talk about self-driving cars and this trolley problem. You know, if a car turns one way, if it turns the other way, it kills the people in the car: What should be done? That's a particularly stark and interesting philosophical discussion, but long before we start to need to worry about those in a really deep way, because to a large extent, the cars are not automating the ability in practice to do philosophy. They're incorporating algorithms about what they're seeing on the road.

I think more importantly, as many of these technologies; particularly machine learning, which is more about training computers rather than programming them; understand what it is that data you have in your training set was perhaps the most important that you had. As David said, sometimes that training set is biased in terms of the data that you selected. And that’s where this idea of not just using data, and using analytics, but using it well, is what’s most important. And I come back to this thing about: It’s not just the data analytics, it’s about the people who use it.

And that's a lot of what goes into being a good data scientist. How do you make sure that you understand, you know the provenance of the data, the biases that come about because you collected data? One of the most famous ones that a lot of us who spent time in data talk about is the ability to use a mobile device and in Boston, in order to determine where there are potholes. People drive around and the accelerometer notices a bump, and it says, "Oh, there might be a pothole there." And one of the things that, at the time was true, was there's a bit of a bias in that sample set based on who had smartphones at that time. So again, one needs to understand that biases come about, it's really an ethical issue about what training set you're using in order to train a machine learning algorithm.

Michael Krigsman: We have a couple of people on Twitter who have asked the same question or made the same comment. And I want to remind folks who are watching on Facebook, if you want to be part of the discussion right now, hop over to Twitter using the hashtag #cxotalk. You can watch on Facebook and chat over on Twitter.

So, Neil Raden and Bob Russelman have both raised the comment that in this new world of job training, what kind of skills are going to be needed for people to be trained and to adapt?

David Bray: So, I will actually pull into that question what Michael just said about biases. I think it is being aware of both your biases and other people's biases, and how that impacts what the machine does. I think it's something that if you're lucky, maybe you pick up either from your own childhood upbringing or from your schooling that I don't think we currently have significant forces focused on. I don't even know what the subject would be other than critical awareness of being aware of your biases and being aware of the biases of others, and how that impacts outcomes involving a machine and involving an organization. And so I think that's a new thing that doesn't exist, and in some respects, a machine can actually reveal to us.

I also think it's going to be about cognitive offloading of certain things, and being able to turn off the day. I can easily see someone getting so wrapped up in the fact that the machine doesn't have to sleep, the machine doesn't have to eat, and they end up fourteen hours later still involved working with a machine and not turning things off. And so you're beginning to see that already where people are saying, "You know, after about nine o'clock, ten o'clock at night if you email me, I'm not going to respond. I'll pick it up the next morning." I think being able to cognitively offload some of your work and recognize that a machine's going to keep on working in the background isokay. But, you, as a human, need to take care of yourself. That's also a skill. It's almost like how we deal with physical education for kids. We may need to equally do some version of cognitive relaxation awareness as to when do you turn off your device, and that you're not 24/7.

Michael Krigsman: A lot of social questions here. We’ve got only about ten minutes left, and one of the topics that I really wanted to talk about is: Michael speaks about the concept of “radical personalization.” I think that’s very important. Michael, would you tell us about that?

Michael Chui: Well, I think one of the things that we've often discovered when looking at data and analytics: Those of us who like data … You know, we look at averages and means in particular, and what we found is oftentimes the averages hide some of the most interesting insights. And so, being able to understand distributions has always been important when it comes to data; to use a marketing term, this idea of "segmentation." In fact, not every customer wants the same thing. Not every citizen wants the same thing. Not every citizen is going to benefit from the same sort of intervention, etc. And that’s one of the things we’ve known for many, many years.

But now that we have the technological capability to not only look at three customer segments based on demographics, or ten behavioral segments, really being able to help an individual based on what their needs are. You know, from a healthcare perspective: really understand, for example, their genetic makeup and then be able to customize something for an individual, a “segment of one” as the people in marketing say. I think that’s a capability which is now coming to the fore. One of the things that we know is that just thinking about people as individuals is something we naturally do as human beings, but being able to have our machines be able to do that as well is a lot of value to be created.

It does bring to mind, again, coming back to your question about ethics and values, how you ought to deal with the privacy question, because when you have enough information to be able to customize a service, a product, or a person, that means you do have some pretty interesting information about an individual. And so, you have some questions about how you want to handle that. But as soon as you are able to understand and handle that; provide that individual citizen, or customer,  or employee with the understanding of why their data is being used and how; then we can start to provide, as you described, a radical personalization. It is one of the things that we described in our report from December as being a potentially disruptive force. Because, many organizations really are set up to deal with groups, as opposed to individuals. And, when a competing organization comes about and say, “Well, I can provide you with exactly what you need in a very customized way,” that can really change the game.

David Bray: And I think that is going to be a fascinating area for the public sector to try and wrestle with, especially in republics and representative democracies. Historically, the public sector has provided the same service without any personalization to everybody, because we're trying to be equitable. And, we don't want people to say, "Well, they got preferential treatment or they got something special." But I think as consumers and citizens alike become almost expecting that they're going to get personalization from the private sector, and then they're going to look at the public sector and say, "Why aren't you treating me like an individual?", that is going to be a real thorny issue of how do we allow the public sector to do personalization of the services to you, but still, have checks and balances to make sure nobody is getting preferential treatment or biased treatment?

Or, it may very well be that people don't want to reveal the information necessary to give the personalization. And so, that's where actually I think for public service, and it may apply to other organizations as well, we almost need to say the Golden Rule of, "Do unto others as you would have do unto you," and tweak it slightly for what Michael was saying to be, "Do unto others as they will commit, and maybe even like you to do unto them." And so, we maybe have to figure out how can individuals in the public express what they want to be permitted done to their data to a government, what they would like to have done with it, and recognize that's going to have huge variability across nations and across the world.

Michael Krigsman: We have got a bunch of questions coming in from Twitter. And, we don’t have that much time left, but here’s an interesting one from Chad Barbier, and he asks: What applications are you finding that automation is working well for today? Anybody?

Michael Chui: I'm going to mention a couple of things. Some of the types of activities that we found have the greatest automation potential are physical activities in predictable environments. And so, a classic case of that is an assembly line. So, we're starting to see a lot of robotics being used in those types of situations. What's interesting is that robotics decrease the cost; we're starting to see them used in services as well. For instance, at home, I have a robotic vacuum. Some people say that until we figure out the problem, we call it a robot; afterward, we call it a "dishwasher." And so, I think on the physical side, that's happening.

On the more cognitive side are two other types of activities: collecting data and analyzing data. And many times, I think people who are watching or listening will recognize this: How many times are there systems where I have to look something up on this system, and type it into a computer, or cut and paste, or copy and paste, etc.? There are a set of technologies described as robotic process automation. They’re not robots, they’re software robots, but they automate some of these processes, which are, as David says, are these boring things where I’m just taking something from this application, copying it, and pasting it into this one, and all those really rote, simple, and super annoying things. We’re seeing more and more organizations try to deploy those types of software robots to take away that really annoying work from human beings.

Michael Krigsman: And David, your thoughts on what is working well today, in terms of automation.

David Bray: So, there actually was a competition about two years ago on Craigle to see if anyone could write an algorithm that could grade a purse for a third-grade teacher; so, find the same sentence mistakes and grammar mistakes. And for about sixty-thousand dollars, someone actually wrote an algorithm that succeeded in doing that. And so, amplifying what Michael was saying, I think it is … My interests, particularly because I'm in public service, are what are those things that we can do to remove the rote, repetitive work from individuals so they can focus on unique problem-solving things they need to do. So, I think it is making sense of large amounts of data to find errors, to correct things, to give recommendations back, and then to tip and cue a human to pay attention. Those are the things right now that I think are working today.

I think the challenge is there are a lot of cases [where] those systems that can make sense of patterns, and cap tip and cue humans, don’t have access to a sufficient amount of data on things that are useful to the public - whether it’s because we need to make sure we protect privacy, or because that data’s right now is not in a format that can actually be used by the machine, [etc.] I think we need to have better conversations about what are the top maybe three challenges we want to solve as a nation, and then identification of what data, as well as algorithms, we can bring to bear. But that technology exists today to find interesting patterns, to find things that are missing, and to make corrections.

Michael Krigsman: We’ve got just about five minutes left, and Michael, would you share a distilled summary with us of your thoughts about where this is going in the near-term, and practical advice that you have for managers and business leaders who are looking at this changing landscape and feeling a little bit confused about what to do?

Michael Chui: A couple of quick things. You know, one is we talked about data a lot, and I think one of the things that we found, and my colleagues who are helping various organizations around the world find, is that there is usually value just sitting on the table; because in most cases, organizations have access to a lot of data, whether it’s data within organizations or external or open data. And, a very small percentage of the value gets captured from that data that is already sitting there. So, Number One: Figure out what you can do with the stuff that you already have access to.

And then, the second thing which is actually the harder thing: which is that because data analytics, AI, and machine learning can actually add value to almost any process, the hardest thing oftentimes for an organization is to figure out what it should do first. And, that really just requires you to map out where you can do things, and then prioritize the things that create the most value, and you can capture most easily.

And finally, the last thing I’d say is you’ve got to solve the technical problems, but the hardest problems that we’ve talked about several times are how you move an organization. And, that requires just leadership, and so working on the leadership side to move an organization to use these technologies well is what’s important.

Michael Krigsman: And David Bray, your thoughts on how you move an organization, as Michael was just saying, to be able to take advantage of these technologies in the right way?

David Bray: So, sort of looping back how ten years ago, Michael and I were researching distributive problem-solving networks, I think you need to recognize that no one person is going to know 1) all the data that is of value within the organization, and no one person's going to know the processes that can best lead themselves to being adaptive and improved. So, almost, in some respects, you want to crowdsource it within your organization, and you want to champion saying if anyone can come to me with an interesting pitch on the inside that says, "Look, if we brought this data, and this data together, we'd have these insights. And then, we could tackle this process, and you almost treat it like an internal venture capitalist.

That shifts the role of the CIO from being responsible, for being top-down, and having to supposedly know everything in a rapidly changing world, to being almost a human flight jacket and champion of anyone who can bring interesting data to bear, that can inform how the organization can do better and improve those processes. And I think that’s required because this is changing so quickly, and at the end of the day, we are changing what people are doing. You are changing how they work, and they’re going to feel threatened if they’re not bought into, “I’m okay with changing this process because I see the better outcome that will come as a result.”

And so, I think that’s almost an imperative for CIOs to really work closely with their Chief Executive Officers and say, “What I will do is, I will effectively serve as an internal venture capitalist on the inside, for how we bring data, to bring process improvements and organizational performance improvements - and work it across the entire organization as a whole.

Michael Krigsman: Well clearly, these new technologies, data automation, AI, and machine learning, have the dual component of the technology and then the organizational implications. And while that’s true of any technology that hits the enterprise, it seems the potential implications seem even greater in this case.

You have been watching Episode #219 of CxOTalk. We’ve been speaking with David Bray, who is the CIO of the Federal Communications Commission, and Michael Chui, who is a partner at McKinsey with the McKinsey Global Institute. Gentlemen, thank you so much for taking the time today!

Michael Chui: Thank you, Michael!

David Bray: Thank you, Michael!

Michael Krigsman: It has been a great discussion and I invite everybody to come back next week because we'll be doing it again with another great show. Thanks for watching. Bye-bye!

Disruption in Education, with Rick Levin, CEO, Coursera

Rick Levin, CEO, Coursera
Rick Levin
Chief Executive Officer
Michael Krigsman, Founder, CXOTalk
Michael Krigsman
Industry Analyst

Companies like Coursera are changing education dramatically. From higher education to vocational and skills training, online courses offer high quality instruction at lower cost than ever before. On this episode, we talk with an online education pioneer to learn about the impact of technology on modern education.

Rick Levin is the Chief Executive Officer of Coursera. In 2013, he completed a twenty-year term as President of Yale University, during which time he played an integral role in growing the University’s programs, resources and reputation internationally. He was named to the Yale faculty in 1974 and spent the next two decades teaching, conducting research, serving on committees and working in administration at the university. Rick served on President Obama’s Council of Advisors for Science and Technology. He is a director of American Express and C3 Energy. He is a Fellow of the American Academy of Sciences and the American Philosophical Society. Rick received his Bachelor's degree in History from Stanford University and studied Politics and Philosophy at Oxford University, where he earned a Bachelor of Letters degree. In 1974, Rick received his Ph.D. from Yale University and holds Honorary Degrees from Harvard, Princeton, Oxford, and Peking Universities. 

Download Podcast

Disruption in Education, with Rick Levin, CEO, Coursera

Michael Krigsman: Welcome to Episode #224 of CxOTalk. I’m Michael Krigsman, and I am an industry analyst and the host of CxOTalk. We have such an interesting show today. We are going to be talking about changes in higher education, and in corporate learning, and our guest is Rick Levin, who is the CEO of Coursera. And, Rick was the president of Yale. He was the longest-standing president of Yale University. Then, he retired and began this, can we say, a second career, Rick? Welcome to …

Rick Levin: Absolutely!

Michael Krigsman: ...CxOTalk!

Rick Levin: Yeah. It was a complete change.

Michael Krigsman: So, you were retired from Yale, and you’re at home, and doing the things you do, and you decided to become CEO of Coursera. How did that happen?

Rick Levin: Uh, well, I was actually on a sabbatical away from New Haven. My wife and I came out to Stanford to take some time off after twenty years of service as President, and a couple of things: One is I got approached by someone who was a senior advisor at Kleiner Perkins, one of the investors in Coursera, and he said, "You know, you've got extra time, Rick. You've got to come over and help us with Coursera!" And I was a big fan of Coursera. Yale was a partner, and we did online learning experiments at Yale in the past. So, I said, "Sure," and I got into discussions with Daphne Koller and Andrew Ng the co-founders about serving as a kind of consultant for them right after the first of the year in 2014. But, no sooner had I done that that John Doerr, the famous leader of Kleiner Perkins was all over me to say, "No, no. What you really should do is not be Senior Advisor but come be the CEO and help build the company." It was easily tempting, because it's such an amazing company with such an important social mission! We're really doing great work.

Michael Krigsman: So, tell us about Coursera. It’s an amazing company with a great social mission, so tell us what is Coursera, and underlying the activities of the company, what is that mission?

Rick Levin: Okay. Well, the mission is easy. The mission is to, you know, transform the lives of people by giving them access to the world's best education. So, we are a platform that offers courses from a hundred and fifty of the world's best universities. We distribute it worldwide. We have [been] using advanced technology to enhance the learning experience, and we have reached, since its inception in 2012, have reached over twenty-five million learners worldwide. Three-quarters of them outside the United States. And, you know, we offer courses across the whole spectrum of human knowledge, but what we found in terms of shaping a business model was people are happy to watch the videos for free in all of these courses, but it's only in sort of career-relevant fields that they're willing to pay for credentials. So, the model is, the courses are there, anybody can watch the videos, but if you want to make it tested and you want to get a sort of certificate, that you completed the course, you pay a very modest price.

You know, we’re on subscriptions for some of them now, forty-nine dollars per months. Others are priced at seventy-nine dollars a course, it’s very reasonable. And, this is drawing people from all over. So, that was the basic start, and we’ve since branched out to do some other things, but the core business is built on free, massive, open online courses that are open to anyone and then people who want to pay for certification are starting to use these credentials as sexy badges in the workplace.

Michael Krigsman: You partner with educational institutions, so well-known professors are putting their courses online with Coursera.

Rick Levin: That’s right. We have a hundred and fifty university partners, including some of the top universities in the United States such as Yale and Stanford, and University of Pennsylvania and University of Michigan, and many, many others; and then top universities all around the world: University of London, and the five or six of the Grande Écoles in France, and top universities throughout Asia, Singapore, Hong Kong, and so-forth. So, we have great content, and because we partner with the universities rather than go try to hire freelance professors, we essentially get quality assurance, because universities don’t want to put second-rate teaching up for global learners to sort of experience, so they really give us their best faculty, the courses have extremely high ratings from their learners. The average course is a 4.6 on a five-point scale. So, we’re really proud of the product. It’s really great. It’s really great content.

Michael Krigsman: Rick, what is the incentive for the universities and for the professors to partner with you, because if we think about it from an economic standpoint, you’re charging under a hundred dollars for a course.

Rick Levin: Yeah. Well, the incentive was we split the revenue with our partners, and the number, the scale is such that the revenue becomes meaningful. I mean, we're growing quite rapidly, we've doubled revenue year-on-year the last couple of years, and it's beginning to be the case that some of our university partners are, at this point, making meaningful revenue. And we think there's a huge growth ahead of us.

Michael Krigsman: So, this is about scale; that’s one very important aspect of this.

Rick Levin: That’s super important. I think, you know, obviously, at those low prices, it wouldn’t be meaningful if we were talking about a hundred students in a class. So the scale is super important. And, I should say that universities’ incentives are not strictly financial, and neither are ours. That is, if you’re a professor at a great university, you know, [you’re] more or less there with the dual mission of advancing knowledge through your research and disseminating it through your teaching. But, you know, if you think about it, historically, universities are, in a way, squandering these talented people and the fruits of their knowledge by wasting it on only fifteen people in a seminar, when they could be teaching fifteen thousand online. So, we see it as a way of actually helping universities to expand their mission beyond their own campus boundaries to educate great numbers of people, which is completely consistent with what they were set up to do, which is to disseminate knowledge.

Michael Krigsman: Is there a disruptive aspect to this in the way that Amazon has changed retailing, or Uber has changed the transportation business. So, is there a disruptive dimension to this as well?

Rick Levin: Well, there are disruptive dimensions, and let me try to characterize it. But first of all, the difference between us and Uber is, of course, we're hiring the existing taxi drivers to teach the courses, not a new labor force. So, we're using universities and working with universities to reach learners. So, they're our partners, not our competitors. So, we're not, in that sense, not really disrupting higher education. At least not now.

Now, the disruptive impact we’re having actually is on the labor market because these credentials are now being taken seriously by employers as certifications that people have competencies. You know, if you look at LinkedIn, you’ll see tons of people who post their Coursera certificates in their LinkedIn profile, and that is now becoming a signal to employers that it will continue to grow as more employers start to hire to these credentials. And, we’re seeing it. We’re seeing a number of big [companies] in Silicon Valley; employers saying, “We’re not going to look only at where the person went to university, but also what further education they’ve availed themselves of in their working careers.” So, in that sense, we are creating a new currency in the labor market and it’s highly valuable.

Michael Krigsman: So, this type of disruption in the labor market must have ultimately upstream effects in terms of the type of activities, experiences, courses, that traditional universities offer, right? So, there's this unbundling. Clay Shirky talks about the unbundling of the experience.

Rick Levin: Yup. Absolutely. I mean, in our first couple years, we just took whatever content the universities wanted to provide us. So, we build a broad catalog of courses across the spectrum of liberal arts and sciences, and business, and technology, and so forth. What we discovered is that the propensity to pay for content was really concentrated in business courses, in computer science-related courses, and data science, which is a big emerging field in which there's a tremendous labor shortage. And so, we saw that, and then we started actually soliciting content in these areas from our university partners. So now, it's a mix. A good fraction of the new content we introduce is actually stuff that we went to the universities and put out an RFP and asked them to supply to us because we knew there was a market demand for it.

Michael Krigsman: One of the interesting aspects of this is how … What were the conditions that needed to be in place for Coursera to be founded, right? There needed to be technology, economic conditions, so what enabled Coursera to come alive?

Rick Levin: It's a great question, and it really does have a lot to do with technology development. Back at Yale in the year 2000, I helped to form a joint venture between Stanford, Yale, and Oxford, not, by coincidence, my three alma maters, and we pulled up to create online courses. It was the first wave of the internet revolution, and the day was just too early. The video streaming was really jerky and not effective, and it just didn't have enough bandwidth, and most devices didn't have enough bandwidth. You know, we didn't have mobile phones that played the courses in those days. So, technology wasn't really there to make it hugely scalable and effective.

Then, there was a second wave in the middle of the first decade in the century when MIT, and Berkeley, and Yale, and a number of other schools just started putting lecture courses online by filming what happened in the actual classroom. And that got a lot of viewers, but that open educational resource movement got a lot of audiences. But, it wasn't really a very edifying educational experience because it didn't have the interactivity. And so, fast forward to 2012, and really you have sort of a simultaneous discovery around Coursera, at MIT, at Harvard, where they founded edX at about the same time as Coursera, and at Britain where the Open University founded FutureLearn just a little while thereafter.

All of these realizing: now the bandwidth is there, now the capabilities are there, now we can actually have interactivity real-time, we can propagate - you know, [we] have the server capacity to host all this, I would say, huge amount of data, you know; 16-20 hours of video content and all kinds of interactive quizzes and stuff. These are actually pretty substantial requirements from a storage and computational point of view. But, you know, by 2012, it was all there. And, it’s not surprising that sort of several of these companies sprung up right at the same time.

Michael Krigsman: How do you differentiate … When you look at the market for this type of learning; and what do you call this type of learning? I mean, we all know the term MOOCs, but what do you call it? And, when you look at the market, how are the different companies differentiated and what’s unique about Coursera?

Rick Levin: Okay. Well, there are a bunch of questions; let me try to disentangle them. First of all, we call this learning, not the type of learning - that is to say, we think it's for many, many types of subjects. Online learning is equivalent to, or even superior to, in-classroom learning. So, let me just unpack that for a minute: [...] We don't yet have the capacity online to do exactly what's done in like a live seminar with 12 or 15 people, which is intense back-and-forth with a live professor asking questions, using Socratic methods … and argument, take feedback and criticism from their peers, and so forth. But, that's an amazingly powerful way of teaching and develops a kind of capacity for critical thinking and argumentation, and you know, rational discourse that I think the online technology is yet to deliver on.

On the other hand, if you talk about mastering content, if you want to learn to code in Python, or if you want to master some of the new techniques of machine learning that are fueling the data analytics operations in so many companies and governments around the world: content mastery we do extremely well, compared to live teaching. I mean, how many people … Have you ever sat in a math course and been infused in the first five minutes, and therefore, we're at sea for the remaining forty-five minutes in a lecture? I've certainly had that experience. And, you don't have to have that experience online because we have six-minute segments, you take a quiz at the end; if you're stuck and you don't get it, go back and watch it over. You might even get guided as we are developing some of the pedagogy of, "If you're really having trouble with this, why don't you try so-and-so's course which is a little easier and will help get you prepared for this course?"

So, there's the rewind button, there are the frequent quizzes, and actually, there's also the opportunity to adjust the speed of the lecturer's voice to anywhere between 50% and 2x the normal rate of speech. And believe me, that's actually pretty cool, too because when you really get something, it's easy for you. Speed it up! And when it's hard, slow it down.

So it’s a lot of ways in which the learning is better, and there have been a lot of studies that have shown that for content mastery, online learning is equivalent, or even better than classroom learning, and actually the best of both worlds is when you blend them; it’s when you watch videos offline, take quizzes, and then come into class and have a discussion. That tends to really … that’s really the optimal experience.

So, that was the first part of your question, “What about learning?” And then you asked, “What are the competitors,” which, edX I mentioned. They’re very similar to Coursera in that they work with university partners, they have, just as we have, they have top-notch institutions. They have headline institutions of Harvard and MIT that were the co-founders. And, you know, actually, the main thing that differentiates us from them is our audience is about two and a half times bigger. We have about 25 million registered learners; they have about ten. We have a broader course catalog. We have about two thousand courses; they have about a thousand. But, it’s very … I mean, it’s great stuff. They have great material just like we do. They are also working with top universities.

Michael Krigsman: We have some interesting questions from Twitter, and I’m just going to take them in order. So, Terry Griffith asks: When you were talking earlier [...] about gaining quality by partnering with universities; high-quality courses, material; she makes the comment that she’s looking for employers to also play a larger role. And she’s an academic. And I know that you are working with employers, and so what about the role of employers in education and learning?

Rick Levin: Okay. So, there are two dimensions to that, that are worth talking about. One is industry-provided content to supplement our university-provided content. So, companies are actually making courses. And, we started to work with a number of companies to do courses that effectively use proprietary technologies. So, the material is essentially not competitive with what our universities do. So, for example, you can now take courses on Coursera offered by Google, to become a Google Cloud developer; or courses from IBM that introduce you to the Watson Internet of Things technologies. We also have some Intel courses coming out of the platform, and PWC is doing some work on the visual presentation of data. How do you give a great Powerpoint talk? Very practical. So, we are complementing university teaching with a more practical, applied courses from universities. I think that will grow from companies … That will grow.

A more exciting development to me is, you know, about a year and fifteen months ago or so, we looked at our data, and we realized, "You know, we've got 50 thousand course registrations from people with an Amazon.com email address, and 45 thousand from IBM, and 30 thousand from Cisco," and we realized people are using these courses in the workplace and connecting with their work. And so, we started early last year going to companies, testing the idea, "How about buying packages of these courses to help solve your problems in training your labor force?" And I have to say this has hit just an enormous resonant chord with Chief Learning Officers in major companies. They see that what we can do is fill a gap that they just can't do. I mean, they do live instruction on lots of things like company policies and compliance, and so forth like that. They use purchased videos for some of the same work, but they're like short-form videos. And, many companies use a Lynda.com or Skillsoft, which are large catalogs of short-form videos that teach you a single piece of skill, how do pivot an Excel spreadsheet or something like that.

In our courses, which are essentially four weeks long, 12 to 16 hours of video content, they saw an opportunity to really start to upgrade the skills of their workers, and we've now got an enterprise business going where we're selling these university courses into companies and they're really finding it very valuable. A couple of use-cases: One is very specific training. You know, like at Bank of New York Mellon, they're onboarding all their new software engineers with our specialization from Hong Kong University of Science and Technology in full-stack web development so everybody has a common language. That's one example.

The other use-cases at the extreme are, "I just want to be a good employer, and I want my millennials to have access to learning and professional development, so I'll give them a wide range of options of courses to take and encourage them to do it." Then, both cases are at work.

Michael Krigsman: So, are you, in this case, replacing the in-house university [to one] that might be focused either on employees or very often companies - say technology companies - provid[ing] education and training for their customers on their particular products?

Rick Levin: Yeah. Well again, you know, is it disruption or absorption? Will the in-house universities now use our content in favor of some of their own locally-created content, or together with it? I think that is the most common use-case, that companies that have these internal universities, they’re going to continue to do things with their own live staff. I do think they will be substituting more and more online courses for live courses. It’s very economically effective for them, and you know, it makes a lot of sense.

Now, it’s also true that there’s still a case for high-level in-person training that many companies use. Business school professors come in and give customized content to top levels in an organization. We think that’s highly complementary with what we’re doing, because our business school partners can basically take the top of the pyramid and we take the base of the pyramid as target audiences, and I think that’s going to work well.

Michael Krigsman: I have the strong sense as you’re talking that your focus is on this partnership and finding ways where you can be complementary to whether it’s corporations or universities as opposed to trying to displace what they’re doing.

Rick Levin: Yeah. That’s very good. I agree with that. I think that is what we’re doing.

Michael Krigsman: We have some additional questions, and we have one from Shelly Lucas on Twitter. And, Shelley is asking, “How is artificial intelligence or machine learning affecting human learning?”

Rick Levin: So, you know, in many ways, actually. I mean, we’re using machine learning algorithms for a whole variety of purposes at Coursera. One, of course, is to drive the recommendation engine and matching learners to the right content for them. And, I think I’ve mentioned before we both do that through the standard kind of recommendations like Amazon would deliver. We see from what you’ve done on our website that you’re probably interested or you might be interested in the following courses. That’s a sort of Level One.

But Level Two is we’re really developing now a whole content-mapping of our courses in a very fine and granular way using machine learning to basically map out what videos contain what topics and what content, in a way that we can then direct learners more specifically to fulfill the needs and learning objectives that they have. So, guiding people to the right content is a big use-case for machine learning.

Giving people the right assessments and helping them get the right kind of feedback is another thing we’re using machine learning for; and some of that is not rolled out, but we’re doing some really exciting work actually on optimizing assessments for you so that you’re not discouraged and can continue to make progress in a course.

Michael Krigsman: So, there’s a technology component. Is the technology piece as important and as large as the educational content piece? How do you think about those two?

Rick Levin: Great question. They really obviously go together. And we can enhance the quality of the content with powerful technology. I do think that if you ask, "Why has Coursera resonated so much, and why have we grown so rapidly?", you'd have to say that the power of our university brands is super-important for that. I mean, so the content; the perception and the reality of the quality of the content probably come first at this point, but who knows where the technology will take us? I mean, it's changing every day and there could be some great breakthroughs.

Michael Krigsman: And, on this topic of technology, we have another interesting question from Arsalan Khan, and I’m glad that he asked this because I was curious about this as well. What kind of analytics and tracking do you do of the courses; of the videos; of the assessments? And he’s also wondering, do you share these results with your partners so that they can improve their content creation?

Rick Levin: Super great question, and it’s exactly what we’re doing. So, with respect to universities, yes. We give them tracking capability. They’re giving the grades to the learners. Now, granted a lot of the grades are the result of machine grading algorithms. They’re not that much human intervention. Most of them are open, large-scale classes. But, the professors have definitely used the data to improve their courses.

So, I’ll give you an example. Drawing from personal experience, I mean I taught microeconomics at Yale for many years, and I taught at the sort of level where you first start to use calculus, so it’s pretty hard for a lot of students to make the leap and actually do this subject. And sometimes I would ask a midterm exam question and almost everybody would flub it. And the question is … the question then becomes, “Is that because the question was poorly written, or is that because I wasn’t getting across and people really didn’t understand what was happening in the lectures?” If I want to change that in a residential setting of campus, I had to wait until next year to pick one of those two hypotheses and try it out and see what the results are. So, it might take me two or three years to iterate what is the best treatment of this subject, or set of treatments and exam questions.

On Coursera, you can do that within a matter of weeks! You can change your quiz questions, you can improve performance, and if that doesn't work, you can go in and re-record a six-minute video, or four six minute videos to cover the block of material and then test that, and see if that improves results. So, the feedback from the learner data is incredibly important, and it really does improve pedagogy.

Michael Krigsman: This has to be one of the most fascinating and incredible parts of the entire online course experience, because it means that you’re able to iterate in terms of that content, ensuring quality content, and ensuring that the content matches the needs of learners by getting direct feedback from learners as a group very quickly.

Rick Levin: Exactly. It’s great. And by the way, that tracking capability that we give to our university content creators, we’re also giving to our enterprise customers so that they can see how their employees are doing and make appropriate interventions where they need to.

Michael Krigsman: So, at Coursera, what does “innovation” actually mean? So when you talk about innovation at Coursera, what form does that take?

Rick Levin: [Laughter] I could step out of the studio right now and right on this floor at our offices in Mountain View, we’ve got 75 engineers who are super bright, competitive with the very best technology firms, just working on learning experience on assessments, on how to sequence material to make it better and in terms of some of the offerings we’re doing now, I’ll come to this later, but we are offering degree programs now on Coursera from our university partners. And that’s a big area of innovation right now because we’re moving into the area of live synchronous interaction as well as the asynchronous massive interactivity that we have now. So, there’s a lot of innovation going on.

Michael Krigsman: So, again, you must have the technologists working closely with content developers, or again, how does that work? How does that interplay to make content?

Rick Levin: Well content, the actual, you know, the professors and instructional designers and so forth are largely in the universities. We do give a lot of support. So obviously, we create the technology platform which gives them, the universities, the tools and gives them things they can do in their courses that maybe before technology they couldn't. I'll come back to that. But, we also have a set teaching and learning team who are pedagogy experts, and they are sort of helping to basically do the quality assurance on the courses. So, all of the courses that we have to go into our larger-volume products, larger revenue products, which are bundles of courses we call "specializations," so, like sequences. All of those courses are beta-tested before we release them. We have a population of two or three thousand learners who have agreed to be beta testers, and so we get a lot of feedback. So, we actually are helping to make sure that the concepts are clear and the courses are well-structured. So, we do all of that. We built technology and sort of human intervention on the teaching quality side.

Michael Krigsman: It sounds like one of the key themes is you having interaction and providing feedback both to your higher education partners, as well as your corporate learning partners…

Rick Levin: Right.

Michael Krigsman: … based on that data.

Rick Levin: Yeah. Yeah. That’s right.

You know, I should mention one other sort of use-case for our material that is growing also rapidly, which is… This gap that existed; what people call the “skills gap,” which I would define as, “There’s a lot of unemployed and underemployed people who are looking for jobs on the one hand; or on the other, there are high-skilled jobs that are going begging, and remain vacant because they can’t find enough qualified people. And, one way to solve that, of course, is to give better skills to the people who need them and who then could get the better-paying jobs and the high-quality jobs. A lot of those jobs are in computer fields, and also in data fields in this era of machine learning and big data, where we have a superb curriculum.

So, one of the things we’re doing now is we’re working with governments on workforce development programs. I mean, one small pilot in the state of Maine, for example, with unemployed and underemployed workers there, where they’re taking our technology-oriented courses in order to get entry-level jobs in that sector. And they’re even getting live facilitation of their learning in collaboration with the University of New England. And then, we’ve got larger-scale programs in a number of countries around the world, and a very large program about to launch in Pakistan, that will be tens of thousands of people needing training. And we have similar programs, smaller scale, in Egypt, Malaysia, Mongolia, Kazakhstan, and Singapore. So, that’s a huge need. And, when you think about who the Trump voter is and what many of them need is they need a new opportunity. The people in our society who have not had … constant or declining real income for the last thirty years. A lot of those people could pick up new skills through online technology and help to get better jobs.

Michael Krigsman: Well, certainly, this issue of skills training in our global economy is so important. And so, please share your thoughts on the economic aspects of this, the social, cultural aspects of this, and what can be done? What are the policy implications here?

Rick Levin: Now, well I think these … You know, in our country, we don’t really have federally-administered training programs at any scale, but there are a lot of state workforce development programs that could be, I think, greatly enhanced by the use of online materials which are so scalable and so low-cost, relative to making everything be live instruction. So, I do see a big scope for that.

I also see scope for government policy at the federal level. We have, on the books, in the US tax code, a lifetime learning tax credit that allows you to, for vocational training in adulthood, after your eligibility for the higher education opportunity tax credit is used up, you can use the lifetime tax credit. However, because it's a tax credit, and it's not refundable, forty-five percent of US households at the bottom of the income distribution can't use it. It seems really weird. It's designed to upgrade people's skills. And then, if your income exceeds 65 thousand dollars, you also can't use it. So, we've got a tax provision that's actually hitting maybe 10-15% of the population, which seems a little nuts. And so, I think a very important reform in the upcoming tax reform would be to just make this a refundable credit and open it up to many more people. And, that would allow people on their own to get access to very high quality, online materials, as well as get to take a course in their community college live.

Singapore has a program just like this, where they have … You can get a $500 credit, and you can use it to take a live course at an educational institution or to take an online course that's certified and accredited by the government. We put over 600 of our courses through the accreditation process, and we're actually the largest beneficiary of that program in Singapore, offering more courses and having more people involved than any other provider.

Michael Krigsman: So, this kind of tax, rather than a tax credit, as you’re describing, what would you call it? A rebate, or …

Rick Levin: I’d say, no … From an economics [perspective], what if you got a tax credit that’s a refundable credit, which meant if you didn’t pay taxes, you got the money back? It’s similar to the earned income tax credit.

Michael Krigsman: And, this would obviously be beneficial to folks who are in need of skills training. We hear all about globalization. So, your work, then, with governments; do you touch on this, both in the United States and in foreign countries as you’re working with various governments? What is your relationship [with them] and how do you see those relationships?

Rick Levin: Umm, we have not been working at the policy level. I'm just suggesting this lifetime earning tax credit as an economist noting that this is something we should be fixing. But, the relationships we have; we have a team, a business development team that's out talking to governments to sort of elicit their interests in the use of our courses for workforce training. And, as I said, it's resonating. We've got about a half dozen countries in the developing world who are now engaged.

And, there’s a lot of great use-cases. We have a small program with a potential to scale, which is here in this country with the Institute for Veterans and Military Families and the idea here is to give our courses to people in the last six months of their military service to help them prepare to transition into the civilian labor force. And so, we’re teaching both computer skills, and actually hotel and restaurant management to veterans involved in this program, or soon to become veterans. Yeah.

Michael Krigsman: So, the breadth of what you’re teaching now, and your plans for the future, are really quite broad; quite wide.

Rick Levin: Oh, yeah. I mean, look. The opportunity here for society, not just for us, but we’re the leading edge, but it is enormous opportunity to use technology to help people improve their lives and improve their economic opportunity. So, I do think that, you know, we’ll see a proliferation of these policies; of this kind of use over time to create a batch.

Michael Krigsman: We have another question from Twitter, which is, “What are the most popular courses that Coursera offers?”

Rick Levin: Good question! So, that’s interesting. If you do it by just enrollments, it’s a mix of career-related courses, and courses that are more based on just practical interest and curiosity. So, for the skills-development side, the most popular courses are first-course machine learning taught by Andrew Ng, one of our co-founders, which he’s updated some and is about to actually do some more new material. But, that’s hugely popular. It really is probably the best foundation you can get in this field of machine learning available anywhere. I mean, he’s a superstar teacher at Stanford, and it’s really a great course. So, that’s one of the top four or five.

Introduction to … It's called "Python for Everyone," a Python programming course taught by a very popular computer science professor, Dr. Chuck, at the University of Illinois… Michigan, I'm sorry, the University of Michigan … is extremely popular.

On the other side, a couple of courses that have had tremendous popularity. There's one called "Learning How to Learn" by a woman named Barbary Oakley offered through the University of California at San Diego. And, this is a neuroscience course that talks about how the brain works and yet, draws very practical lessons about how to study effectively. So, it's very cool. She has this metaphor about how active and passive ways to the brain work, and how you have to combine them. So, that leads to tips like never study for more than twenty-five minutes without taking a break. And there's a whole lot of scientific evidence to back this up. Very entertaining course.

Those three are probably the top three courses in terms of popularity.

Michael Krigsman: And we have another related question coming from Twitter. We have just about five minutes left. And this is from Pao-, hope I’m pronouncing his name correctly, Paola Guevara, and Paola asks, “What is most inspiring to you about online education?”

Rick Levin: Oh, that’s easy. Every meeting of our company, we have all-hands meetings typically once every other week. And, at the end of the meeting, someone in our company will get up and present the story of one of our learners, and they’ll often do it just be reading email exchange. Sometimes, they’ll connect it by Skype with those learners and create a video for the company to see. But, the stories are just amazing. I mean, people that have completely transformed their lives because of their access to this material. I mean, we had a story recently from a Syrian refugee in a camp in Turkey about how this has just opened up a whole new world to him. We’ve had a family of a dyslexic young man tell us about how this child couldn’t relate in a live classroom. He was terrified, and he couldn’t function. And all of a sudden, he’s just taken off and blossomed as a human being, and he’s completely like 25 of our courses, and it’s changed his life and the family’s life.

You know, people getting amazing economic opportunities, from stories as mundane as "I graduated from a British university as an English major, and of course there were no jobs for me. But, I got out, and I took Coursera courses on data science and now I got a great, high-paying job as a data analyst. So, that's a mundane one. The more radical one is "I'm a battered woman. I escaped a brutal marriage in Bangladesh and took work and business courses on Coursera in order to prepare myself to start a business; and now I have a successful bakery.

I mean, these are amazing stories. That’s what inspires me.

Michael Krigsman: Seems like you’re having a good time.

Rick Levin: Well, it’s a fantastic thing to be able to create a company that we expect will be a sustainable, viable, you know, profitable enterprise and at the same time, has an enormous social impact. It keeps all of us here really going.

Michael Krigsman: And, in our last couple of minutes, where do you see online education going?

Rick Levin: I think it’s going to be huge. And, I think what it does is it changes the paradigm - that education is no longer something you do K - 12 followed by four years of college; that it’s not a one-and-done operation; that education is a lifelong pursuit, and that in your twenties and thirties, and even forties, as you are still moving up the ladder career-wise, you’re taking courses all the time either in your workplace or on your own, on your mobile phone. By the way, we’ve been talking about that, but this is accessible. I mean, our mobile app is … You can do just anything you can do on the web, you can do on mobile.

So, I see that people are doing career-related activities in the early stages of their careers, and then folks that are well along like you, who will want to take an astrophysics course just for the heck of it because you're curious and would love to learn about what's going on. So, I think it serves people at every stage of life with every kind of need. And, to me, as a person who spent 43 years at Yale University as a student, teacher, and president, the idea that universities can now contribute to people and have a lifelong relationship with learners and not just a four-year relationship, I think it's immensely powerful. And, I think universities are going to eventually learn that they have a whole new, very important social function.

Michael Krigsman: With profound implications economically and for societies here and certainly, in other countries as well.

Rick Levin: Absolutely.

Michael Krigsman: Well, fantastic! You have been watching Episode #224 of CxOTalk. And, we have been speaking with Rick Levin, who is the CEO of Coursera, and previously, he was the longest tenured president of Yale University in its history. Rick Levin, thank you so much for being with us and taking time today!

Rick Levin: Thanks for having me! It was fun.

Michael Krigsman: Everybody, come back next week. We have more shows, and check out CxOTalk.com/episodes, and subscribe on YouTube. Thanks so much. Have a great day, everybody. Bye-bye!