Data and Analytics in the NFL

Michael Krigsman: You're operating on this enormous stage.

Paul Ballew: Really dominating viewership in TV and all of those things. There's nothing like it in the world, to be honest. The biggest event of the year – all those things.

Michael Krigsman: That's Paul Bellew, Chief Data and Analytics Officer for the National Football League. Paul, how are you doing a week or so before the Super Bowl?

Paul Ballew: Doing well, Michael. Looking forward to next weekend. Hopefully, everybody is doing well on your end as well.

Don't forget, everybody. Tune in next Sunday. About 14 hours of coverage of the Super Bowl, so there's a lot of content to digest, but all good. It's been a great season for us and hopefully, we've got COVID behind us, at least a little bit further, and nothing to complain about, for certain.

About Paul Ballew

Michael Krigsman: It must be pretty extraordinary to work at the NFL. But before we jump there, just give us a sense of your background. You have the most amazing background in data and analytics.

Paul Ballew: I've had the privilege of growing up in the field before the field even existed. If you go back to the origins of the field, it's always been anchored around can I generate insights to affect a decision and drive a better outcome.

We are now a field, and we're a field because the science can now be scaled. It can be impacting so much of the decision-making environments of every organization, whether you're trying to drive efficiencies in manufacturing, trying to connect with your customers better, trying to test new products and services more relevantly and quicker to get insights. All of those things that were so laborious and so difficult two decades ago have now become so much easier.

For me, I've been able to see it in manufacturing, financial services, retail, now in sports, also in government. It's been quite the journey and I like to tell everybody I've had a front-row seat to this amazing field which is now driving so much in the way of transformation across all industries and all sectors.

I get up most days and kind of think of myself as the luckiest person around because we didn't exist 30 years ago, and I've got to have a front-row seat for all of it during this period of time.

Michael Krigsman: You've worked in very large organizations like Ford, GM, and the Federal Reserve Bank. What's it like to be working at the NFL, especially a week before the Super Bowl?

Paul Ballew: Well, I can't lie. If you love sports, which I do, it's just fun. There's no getting away from it. Every day, we get to talk about football. We get to talk about sports. We get to talk about ratings and viewership and all of those things going along with it.

But on the other side, there's also a very serious dimension for us to do what we're doing in data and analytics. The league has a number of things that we're systematically focused on tackling. The league is at a point where data and analytics is a critical engine to what we're doing today and, more importantly, what we're going to do in the next three to five years.

When you think about our remit, our remit includes things such as supporting player health and safety because addressing that issue systematically for the league is important for the league and critically something we've been focused on for a while. On top of that, we get up and we focus on officiating. We focus on revenue generation and connecting with our fans in a more meaningful way.

We're involved in all the operations within the league and, increasingly, with our clubs, so it's a lot of fun. There's also a lot of heavy technical substance to what we do every day, and there's a lot to do.

So, if you like to get up in the morning, you love sports, you have a passion for data and analytics, and you have a lot to do, that's a good combination. Those are all the things that I enjoy doing, building organizations. But data and analytics combined with sports, I get up most mornings and kind of go, "Really? How did this ever happen?" [Laughter] I consider myself to be very blessed.

What is the role of data and analytics in the NFL

Michael Krigsman: Data and analytics, we know is important to sports, in general, but where does it fit into the NFL? I should mention that you are the NFL's first chief data and analytics officer, which indicates the importance to the league. What's going on with data and analytics at the NFL?

Paul Ballew: Our remit is we're an enterprise asset. We support all facets of the league. We support it in the context of building a central data organization, which has all the dimensions of the central data organization: data management, data engineering, data governance, data acquisition.

Our involvement in technology and our data and analytics strategy, working with our IT partners, we have that remit.

We also have the remit for all the data science and analytics teams: viewership optimization, revenue, fan, all of our personalization work in collaboration with marketing, what we're doing in football operations, what we're doing in player health and safety.

I give you that context because it gives you some sense of how the league is doing data and analytics. It's viewing it in a way that I think should be viewed by any organization that's serious about it.

That is, you want data and analytics to support the operation comprehensively. You want it there to support the immediacy of business questions and needs today. But we also view data and analytics as a critical enabler for the future of the league, both at the league level as well as with the 32 clubs.

What I mean by that is, think about what's going on in the world. Personalization is a must-do given the digital revolution that's there. The movement towards direct-to-consumer that is powered by personalization is not something that's theoretical anymore. It's in front of us.

Continuing to improve your content and the relevance of your content, which includes minimizing injuries as well as improving the game and the enjoyment of the game, is absolutely essential. Then, of course, all the activities around how you support that in the ecosystem are either immediate needs today or they also face transformations in the future.

Organizations that really embrace data and analytics at scale do so when they take that perspective. They realize that we're not a science project. That we're not a one-off initiative. We are an absolutely critical element to the entire organization and what the organization is focused on to make their business better, to connect with their customers or fans more deeply.

The other thing I would mention, of course, is the league is getting up now in the morning and realizing that this technical element (supporting data and analytics) is a must-do to harness. When we talk about things that we throw as almost flippant statements like artificial intelligence, it's no longer an R&D activity. The technology is at a point where you need to consume it. You need to deploy it.

But then the question is, how do you do it? How do you scale it? How do you get maximum impact? Without a data and analytics organization, it's very hard to make that come to life.

Michael Krigsman: Since you are the league's first chief data and analytics officer, I make the assumption that this is a relatively new set of muscles that the league is learning to build and flex.

Paul Ballew: Absolutely, but the league has also done some very good things in advance of us bringing this organization together. Starting a few years ago, they brought together parts of the venue analytics team to really help the league.

We've obviously pioneered things like next-gen stats, which were the sensors in the pads and the sensors with the officials.

We've laid the track in terms of our cloud environments and the technology around it, so there is something to build on.

As I like to say, and we talk about all the time, we were at that 1.0, 2.0 level of data and analytics, which is good because you have some building blocks. Now we've got to get to 5.0, 6.0, 7.0.

What's really behind that is two key elements for us. Number one is you have to build the data organization that's scalable and properly governed.

The words I always use is you want to make it scalable, repeatable, and governed. Without that organizational construct of bringing the data side of the shop together – something 20 years ago we wouldn't talk about, but now we do – you just can't execute and have the impact you're looking for.

Secondly, you want analytics to be driving the organization comprehensively, not just on the revenue side of the shop, not just employer health. You're looking at it across that and getting the synergies, developing the talent, leveraging the organization systematically. That's what we're focused on doing.

There are parallels to other organizations that I've had the privilege of setting up in the last few decades. There is always uniqueness, but those parallels are if you're going to do this at scale in a properly governed way with maximum impact, you've got to put the piece parts together, invest in the people, invest in the technology, and, importantly, invest in the collaboration with the business partners.

Data governance in sports and the NFL

Michael Krigsman: When you talk about putting the pieces together with the right kind of governance, can you describe, give us some insight into the kinds of pieces that are relatively unique to the NFL because I think all of us are interested in sports to some degree and the NFL and the Super Bowl are such enormous brands? Take us behind the curtain a little bit on that.

Paul Ballew: When I talk about governance, sometimes governance also has become shorthanded. What we mean by that is there are different components of governance.

• There's data governance, which has data quality, privacy, and consent.
• There's analytic governance, which is all the validation work and the monitoring as it pertains to analytic solutions driving outcomes because it affects decisioning environments now and workflows.
• Then, of course, there's the governance around how we're using data, insights, and analytics to make decisions, which is always with the business partners.

For the NFL, we have some unique areas. Player health and safety is a very unique area for us. It's not to say that safety hasn't been with me in prior lives. It has in automotive, but it was heavily tied to the product and the impact of airbags or other technologies to reduce severity of crashes and accidents.

In the case of player health and safety, it's a very complex, multidimensional element. How can we systematically reduce the probability of injury occurring in a sport that's a contact sport? There is an epidemiologist's view on this. There's a biomechanical view of this. There's a football operations view of this.

We think of it as, we call it, a tripod of all of those multidimensional views coming together to determine how we can continue to improve safety while still making the game compelling. That's the great balancing act. It's a very unique area for us and it's an area that we collaborate with other organizations because of just how sophisticated and complex it is.

I would also say that there are dimensions to the NFL that are pretty unique as well. We have relationships with clubs. "Who owns the fan?" for instance, is an interesting question. We're committed, through our organization, to helping clubs in the league deepen fan engagement and building that personal connectiveness.

Well, of course, you have to execute that through an organizational structure that is pretty unique. You've got 32 individual clubs that are individual business entities. You have the league. Then you also have partners.

How you systematically get the insights with the permission of the fans to engage them meaningfully and then do it in a way that you're orchestrating this relationship across all these channels, partners, and so on requires a different approach and, I would argue, a dexterity to personalization that is different than perhaps a retailer who has their own interesting challenges with regard to personalization. In this case, it's a complex orchestration component that has to be navigated along with what we're doing on the data side and the analytics side.

Those are just two examples. It is fascinating. When you come in from the outside, you have this image of this very large, incredibly successful organization.

Let me attest; it is a phenomenal organization that has so much impact and so much reach. But then you find the nuances that are there that, if you're going to stand up an organization to support it, you really have to account for, adjust for, and build the capabilities around.

Michael Krigsman: When you talk about something like personalization, for example, you are ultimately responsible for the end-to-end from the data collection, through the analytics, to delivering something useful back to the league or to the individual teams. Is that correct?

Paul Ballew: That's correct. If you're going to be successful in data and analytics, you get up in the morning and realize there are two big missions you always have to achieve on behalf of the organization you're helping.

Number one is organizational efficiency or effectiveness. In the case of the NFL, that's providing what assistance we can on the content, officiating player health and safety.

The other thing you have to be successful on in data and analytics is helping organizations connect with their customers in more meaningful ways because that results in higher sales, higher retention, opportunities to cross-sell, and so on.

We have the exact same challenge in the NFL. We need to deepen relationships with fans. To do that in this world, the opportunity data and analytics bestows is the opportunity to connect with them personally.

I call it being customer fan-centric. You've got to be careful with centricity as kind of an overused word, so maybe it's better to talk about sensitive, so you can talk to the right person at the right time through the right channel – and so on.

The way we approach it at the league is we think of it on multiple dimensions. We determine who, what, how, and then we measure the effectiveness to continuously build upon it. Data and analytics is centered around determining who and why.

Our marketing partners in the clubs are centered on the how: How are we going to execute it? What's the content? What's the channel – and so on?

Then we come back around and measure the effectiveness of that interaction. If you build that system systematically, it's game-changing. That's what we're focused on doing.

Then of course, with the world we're living in, you better do it because direct to consumer and people cutting cords and all of these other things really require you to have that level of connectedness with the individual because the traditional ways of connecting with them were everybody watched TV, of course, are changing pretty rapidly.

Data collection and storage at the NFL

Michael Krigsman: We have an interesting question from Twitter that I think relates to what you were just talking about. This is from Arsalan Khan. He's a regular listener. Thank you, Arsalan, for watching. He asks great questions. He wants to know about the data, the kinds of data that you're collecting. How do you decide what data to capture and where there's going to be ROI from that data because it seems pretty clear you have an enormous set of data sources that you can choose from. How do you make those decisions?

Paul Ballew: The good news is, data collection and storage, the cost has dropped rapidly. Fortunately, most organizations are no longer in the data purging business. That was actually very prevalent a couple of decades ago, "We don't want to keep this data."

I want to just remind everybody the U.S. government has been keeping data since 1790, so we should have a very similar approach.

Having said that, with regard to new data sources, we have a data acquisition team. The data acquisition team is there to help us evaluate the value of new data sources.

We have an environment by which we look at the potential use cases. Sometimes you have to buy the data. Sometimes there are other costs in acquiring it.

We're also making sure that any data we acquire, we work very closely through our privacy team to ensure that we have permissible use and related activities going along with it.

It's a great question, though, for one other point. That is, there is always this balancing act. The data science teams want every data element in the world you can possibly provide them. I've never met a good analytics professional who isn't hyper curious and has never met a data set they don't love, which I encourage because I want them to love the data set, I want them to be hyper curious, and I want them to do all those things that you want the investigative part of our jobs to do.

On the other side, our data team – the other part of my organization – gets up in the morning and goes, "Oh, I've got to acquire this data. There has to be an ingestion layer. I've got to curate it. I've got to store it. I've got to put the right access controls," and all those things. They're not negative about it, but they are trying to prioritize.

What we've done over my career is to put a data acquisition team as part of that data organization to help bridge that. The requirements of the end-user – it could be the analytics community, could be the business partner – are then met into a data organization that has to support those needs. It's an interesting journey to be on.

Once again, 15 years ago, I doubt we would have put a data acquisition team in place in any organization. Maybe it was 15 years ago we started to experiment with it, but the world has changed.

That's the way we look at it. Better data is what we're all about. These expressions around big data, an expression that I just absolutely hate and detest, and these quips around 90% of all the data has been created in the last 10 years. You don't know.

Just to be clear; we're able to capture data today. I imagine Aristotle and others thought they were also generating data and insights, so I wouldn't go down that path. I'd always think about it as better data and the ability to use it to support needs is what this journey is all about.

How to choose the right data to meet business needs?

Michael Krigsman: How do you ensure that the folks inside the NFL, as well as in the teams, are focusing on the right sets of results from your data acquisition and analysis as opposed to just being inundated with tons of stuff that looks interesting, might be nice eye candy, but really ultimately is not that important?

Paul Ballew: A few tricks along the way of being around us for a couple of decades. One thing I do is establish a business intelligence function, what we call a center of excellence, and other organizations as well.

Those foundational insights matter. But if we're not careful – the crude way of describing it – we'll just vomit data and reporting. That's what happens.

Just because there's more data doesn't mean it's better. It's critically important to have a group that's there supporting foundational insights.

The second element to this (and I think it really gets underestimated, Michael) is the collaboration with the business has to be one of the pillars of any data and analytics organization. That's both the problem formulation on the upfront side of this as well as the consumption of the capabilities on the backend and the measurement.

We shouldn't just build it, throw it over the wall, and assume good things will happen. That's not the way the process works. We're internal consultants. That's really what we are, and so you have to embrace that. You have to embrace the change management.

You're seeing, by the way, many data organizations now putting in place business transformation teams and process engineers who go along with data engineers and data scientists.

Now, one of my bosses decades ago was so far ahead of the curve on that, he had a data science support team. We didn't call it that so much. It was decision science support at that point in time.

Decades ago, he kept reminding me (early on in my career) how important this was. Of course, I was a purist and a theoretician saying, "Well, if I come up with a great insight, everybody in the world is just going to beat a path to my door."

He was so far ahead of this journey, I'm now very proud to say that not only was he right but he taught many of us good practices. We have to embrace that transformation component, which involves spending time with the business to understand what they're trying to solve for, their activities, then helping them consume what we're doing, and hopefully, get away from some of the shiny objects because, of course, dashboards now are ubiquitous and we can generate all sorts of things.

I like to say the dashboards of the next decade are the Excel of the last decade where everybody and their brother has a link to Excel spreadsheets. We've all learned how to click and drop. We've all learned to write macros or all those other things. Just because you can do it doesn't mean you should do it. [Laughter]

Michael Krigsman: How does this then relate to the kinds of problems that folks inside the NFL, as well as your player clubs, need to address in order for them to get the outcomes they want?

Paul Ballew: I'd always put it in a couple of buckets. First, foundational insights matter; what happened, why it happened, what's going on. There's a lot to be said for that.

You're in this journey of helping to inform people that are making decisions. Doing that correctly should never go away from any data and analytics group.

I know sometimes there are practitioners in our field that say this whole reporting foundational insight stuff is tampering and taking away from what we really should be doing, which are these very complex questions. Just to be clear; I've never seen a successful data and analytics organization that doesn't do a good job on providing foundational insights and taking latency out of foundational insights.

Business leaders want to know what's happening, why it's happening, or at least hypothesizing why it's happening faster with greater precision. So, you have to do that. That's the hygiene, foundational price you pay to be in our field. I think it's also good for us because it helps us keep our fingers on the pulse of the business.

I have a team at the NFL that specializes in media and viewership optimization. Well, one of our biggest jobs is to interpret the ratings and the viewership that happened over the weekend. Much of that is what happened, why it happened. I don't know if you find any profound insights in there, but what you do is you help business leaders understand and provide context.

Now, on the flip side, there's a big part of what we do that involves very complex questions. My example on player health and safety, that's a very complex set of questions in terms of what else should you do to impact what's going on with player injuries. It's a great example.

Personalization is another one. Just because we get up in the morning and we have some information on an individual, to really build a whole test environment and learn continuously through feedback looks how individuals are responding to stimuli we're providing to them is pretty complex.

We've got to do both. We've got to get the data right. We've got to get the visualizations right. We have to do all of those things. If you do those things, you're then helping the business leaders in the decisioning activities they're doing soup to nuts.

That's our job at the end of the day. We're a support function.

Michael Krigsman: Ultimately, you are – I'll use this term – infiltrating through the organization in order to support really every business function, operations function, and (in the case of the NFL) sports function, the actual playing functions as well.

Paul Ballew: Yes. I always pride ourselves when we cross two lines. One, do we help the organization comprehensively? Secondly, when we become the trusted advisor.

The trusted advisor, when you really think of that concept, it's really what this is all about. It means we don't have to force ourselves onto the table or at the table. We're at that table because we're valued.

We are internal consultants. That's what a data and analytics organization is if you're in a for-profit entity. Maybe not if you're in another entity. But within that structure, although my tenure at the Fed would say, very similarly, we were internal consultants. It's just a different outcome, mainly monetary policy or regulatory support. That's our job and that's what we're paid to do.

If you think about a consulting organization, consulting organizations that are very successful, what are they? They're trusted advisors. They are the go-to for leaders to go leverage and take full advantage of.

Data and analytics at the Super Bowl

Michael Krigsman: We have some questions coming in on LinkedIn and Twitter. On LinkedIn, Mona Khanna says, "Good luck with Super Bowl 56!" which leads directly to what is the role of data and analytics and your team in the Super Bowl?

Paul Ballew: We're there on the ground providing support, especially on the data side. Our data team, most of my data team, is there working with our IT partners behind the scenes to make sure everything runs without a hiccup. So, we're there.

We're also there from a viewership and a media standpoint, all the measurement and diagnostics, and understanding how we're being portrayed and consumed from a content standpoint.

Then, of course, at the league level, there are all the other things going on such as next-gen status and tracking and adjusting of that information and leveraging of that information. That's more forensic, afterward, but we do leverage all of that to continue to make the game better.

We're there. We're behind the scenes a bit. I don't know if I'd call us the intel yet of sort of we're the processors behind it. We have a lot of roles going on during all of that, and we're really excited about it. We just continue to grow in terms of the impact on the league, and we're very proud of it.

We're also at the Pro Bowl, by the way. I've got my team out there, so there's quite a bit going on.

Data sharing with teams in the NFL

Michael Krigsman: We have another question now from Twitter. This is, "How does the league share data with franchises and other partners, what kind of data and algorithms do you share, and do the franchisees need to share their data back with you at the league?"

Paul Ballew: We're going through a journey right now of deeper and deeper partnerships with the clubs. We provide a service layer for them today of data enrichment and analytic services, targeting, and related functions. Those services are ramping up very quickly and very systematically.

We are continuing to partner on the data unification standpoint, bringing the fan data together. But that's more collaboration. That's not a mandate that those activities have to occur. The way we view ourselves on behalf of the clubs is it's a partnership and we're a service provider.

What has been so exciting for me is that the clubs have a deep passion in collaborating with us, in working with us. Going forward, I expect that our services to the clubs will allow us to get to the unified view, will allow us to engage with fans at the league level with the clubs in very close coordination and that, going forward, we'll continue to provide deeper and deeper insights to the clubs.

It's been a fun journey. It's been a lot of enjoyment for me to get to know all 32 clubs and the different journeys they're all on individually because they all have their own data and analytics organizations. But the collaboration is great, and I would describe it more as a collaboration.

Michael Krigsman: I have to say, as you're talking, I'm really struck by the broad complexity of what the league is involved in, what your team is involved in, and then you add in all the clubs on top of it. It's a very complex operation, and you're operating on this enormous stage.

Paul Ballew: It's amazing, coming in from the outside. You have an image, and you see this in any organization, but certainly with the NFL. It's such a brand, such a platform, such a dominant live event, and really dominant in viewership in TV and all of those things. There's nothing like it in the world, to be honest. The biggest event of the year – all those things.

You come in. You walk in, and you have certain assumptions regarding where we're at, activities, and how it functions. Then you learn a few things, and you do realize that an organization of this impact has a degree of complexity to it. But it's a degree of complexity that makes sense, once you're behind it and you understand the history of the league a little bit more.

Again, being on the outside, you understand a little bit of the league. I grew up with football and all of those things, and sports are a passion for myself. But then you get behind it and you go, "Wow! Yeah, there are 32 separate entities, and there's the league over here, and then there are partners over here."

The partner component is complex as well because you've got sponsorship partners, but you've got network partners. You've got all those components that go along with it. It does require a little bit of organizational dexterity, but it's been very enjoyable to understand the inner workings.

You also quickly understand the impact that the league has on individuals and the commitment we have to make the game better going forward. All of us get up in the morning with a high degree of passion for the game. It's one of the things you do see is the passion for the game and making it better is there all the time.

It's quite enjoyable because, in my life, I have two sons. Sports were a common bond for us when they were very young and still a common bond to this day.

My eldest son, by the way, works for the Lions. He actually beat me to the NFL. He's a data engineer, and he works for them. He's been there a number of years. Now we get to even collaborate closer.

How to work with customers and partners

Michael Krigsman: We have another question, a really interesting one from LinkedIn. This is from Anshuman Das. He says, "As a trusted advisor to the organization, what are some of the challenges you face when you're dealing with these data and analytics issues with the marketing team at the NFL or among the clubs?" I think we could probably broaden that. It's really a broader question of fan experience, what you're doing, and what are the challenges associated with it and the data.

Paul Ballew: It's always an interesting journey to go through with any business partners. The blessing we have at the NFL is our business partners are fantastic, especially on the marketing side. I couldn't ask for better marketing partners. I almost pinch myself on that every day as well.

Having said that, our job is to be an objective organization. That always requires a level of collaboration as well – that's what I keep coming back to today – because we are going to provide facts and information that sometimes go against what people are hoping for, what they believe, or other things.

The important part for us is we want to invest the time to understand their context, their needs, and why they believe it. Then our job is to help them achieve their objectives, but our job is also to report out. It does require the human side of this to invest time in.

I'll go back to my career at GM. One of the roles I had at GM that the CEO relied on me for is that I was a very objective, cross-functional part of the organization. That required, in many cases, to share with him elements of what was going on that weren't bringing the most positive news. In fact, in many cases, given what was going on in the automotive industry, it was a very stressful time.

I spoke to him a few years back. He was thanking me profusely for playing that role. We have an exceptionally good relationship, 15 years away from working together. For him to compliment me on that, I think that points out when we do it the right way. And doing it the right way requires that.

The league, again, you do have 32 independent entities. You've got the league. You've got marketing. You've got our media group. You've got all these other groups.

The way we've approached it is to turn it into a team sport. Our activities are structured around what people in software development would think of as product teams. We believe in that as well.

There's a lot to be said for those collaborative type of organizational constructs, and it really helps a data and analytics organization to think of themselves as operating most effectively in that sort of construct.

My advice is invest the time, build the relationships. There is nothing that will substitute for a good relationship in any collaboration.

Michael Krigsman: On this exact topic, we have another comment, another question from LinkedIn. This is from Avi Singh Malhotra. He says he caught a phrase you used: customer sensitivity. He loves it. "Could you share an example to bring this concept to life, especially in context of how a consumer gets engaged for the coming weekend?" How do you engage your consumers with this concept of customer sensitivity?

Paul Ballew: Let me maybe give a little context. A few decades back, there was so much underway in terms of "I want to be customer-centric. I love my customers. I put my customers first." All of these sort of managerial clichés.

We really started to probe on it a bit. Very few organizations can really be customer-centric just because of trade-offs and organizational structures and so on. If you're really customer-centric, you treat every customer like they would be your only customer and every decision first starts with what's best for them, even if it's highly detrimental to your organization. It's very hard to do it.

But you can be customer sensitive, and many organizations levering data and analytics are increasingly customer sensitive. What does that mean? Well, it means don't carpet bomb them with emails and emails and emails and outreach and outbound and so on. Don't do that.

Make sure that you're communicating with them in a respectful way, in a way that allows you to communicate to them when they want to be communicated to, how they want to be communicated to.

All of us have those spam filters on all of our emails because we just get saturated. Well, that's a very low-hanging fruit around being customer-sensitive before you even get to the hyper-personalization elements where you're being respectful with their permission around triggers because you've identified some change in what's going on in their behavior, not to mention next best conversation and all these other elements that go along with it.

I just point that out because one of the downsides to the digital revolution and the technology revolution is there are a whole bunch of things that we can do today that we shouldn't do today. There's a really important differentiator between can and should.

When the cost of reaching out dropped as much as it has where we no longer have to do snail mail and put a stamp on it that costs us $0.25 back then or so on, and we can now do it by a press of a button, we've lost a big chunk of that.

The next round of personalization is to get away from that mindset and realize that it's to be customer sensitive, which means, "Talk to me like I want to be talked to and when I want to be talked to. Don't make me have to put a spam filter on or block you or do those other things."

We're going to see the same thing with smart devices as well because we're now getting to the point where the cost of being able to contact somebody through that device is dropping. How are you going to manage that is a big part of that.

We spend a lot of time with our marketing organization in a collaborative way that we call getting the orchestration layer right, and that's a big part of this journey.

Michael Krigsman: You just spoke about personalization and being careful with the data. At the NFL, how do you formulate drawing that line between what you can do and what you should do (when it comes to privacy and data with personalization)?

Paul Ballew: Yes, we have an organization focus on data governance, which is focused not only on quality but also use cases. Not only use case approvals from a legal standpoint, but a use case approval on partnership with our business partners to ensure that we're going down the appropriate path.

Michael Krigsman: You've baked this into your data process. It's fully baked in.

Paul Ballew: Yes. It's part of the ecosystem and how we set ourselves up.

How to determine the economic value of data?

Michael Krigsman: We have another question from Twitter. This is again from Arsalan Khan. He really asks such great questions. He says, "Do you assign financial value to data not only in terms of cost but potential opportunities?" He follows up saying, "How does your CFO see the data versus your COO?"

Paul Ballew: I prefer to always look at it as economic value and not just revenue or cost savings. Revenue and cost savings are part of it, but we should be impacting a variety of other things.

If we're deepening in helping marketing deepen engagement with fans, that is an economic value that is more than just sales or LTV. It has economic value in terms of what we can do to better connect our partners with customers or fans (in this case). So, we do view it a bit broader.

We also have spent quite a bit of time with the education of the organization helping them understand some of this is just foundational. You shouldn't be looking at a data and analytics organization where everything has to have an ROI associated with it. In fact, organizations that do that under-invest in data and analytics, eventually.

We should hold ourselves accountable to deliverables. We have to be careful with that. When you see organizations that do that, they turn data and analytics into projects instead of organizations.

When was the last time the finance organization or the HR organization had to put pen to paper to show the ROI on their organizations? There is this interesting balancing act we have to go through and hold ourselves accountable, drive towards economic value and maximum business output, but be very careful you're not on a project-by-project funded basis because you'll never scale.

In terms of our CFO, he's great. We have a good relationship. He's doing the job of a CFO. He's having me justify the investment. He's having me ensure that we're connected with our business partners. He's having me make sure we're focused on prioritization.

I enjoy it. I've reported to CFOs in my career. Sometimes data and analytics reports to CFOs as a separate function. I think it's great.

Our CMOs and others the other C-suite – I'm a C-suite, but the other C-suite – it depends on where you're at – it's always interesting; your view changes based upon your seat sometimes. If you're a CMO, you're just like, "I love this data and analytics stuff. Give me more."

If you're a CFO, you're like, "Ah, I get it, but make sure that we don't over-invest." The bigger question for CFO's is, "Are we going to leverage it." It's generally not over-investing. It's generally, "Are we going to actually take advantage of it?"

Michael Krigsman: How do you make the case to justify innovation with data and analytics when that innovation might be really, really expensive sometimes?

Paul Ballew: It's one of the biggest balancing acts of them all. My way of setting up a data and analytics organization is to think of ourselves somewhat as a business (not exclusively, but as a business).

There always should be a portion of our activities which are R&D. You just have to explain to the organization that the R&D component does matter, that we can't ignore it because the technology is changing too fast.

Where are data and analytics headed in the NFL?

Michael Krigsman: Can you tell us about next-gen data? Where are data and analytics going with the NFL? Where is all of this heading (not in 15 years, but over the next few years)?

Paul Ballew: Personalization at scale are clearly going down that path, which includes where we're going with more unsupervised analytic workstreams. I know we like to throw flippant terms like artificial intelligence or ML out there, but let's just call them unsupervised.

Clearly, player health and safety, you see the work we're having around digital athlete, which is the ability to understand an individual in context. A player in context is very important.

Then for me what's exciting (out into the future) are things like direct to consumer and other capabilities we'll be empowering and enabling.

Then maybe the last piece is we're going global. The NFL is global, and we're ramping up our capabilities to support a global operation, which I've done a few times in my life and I always enjoy that.

Michael Krigsman: What's the composition of the team that you need in place in order to accomplish these things?

Paul Ballew: You need good data scientists. You need good data engineers and the whole data organization, in general. But I also increasingly look for team members that can drive adoption, those translators. Kind of a terminology we've gone away from are decision science support professionals. That's a big part of this journey.

Then individuals with the technology side, software engineers, those that have a background in terms of the tech staff are important as well. Data and analytics is moving into areas more of an end-to-end solution, so we need individuals that can interfaced with the business, and then we need the technical back-end as well.

Michael Krigsman: When you say, "Drive adoption," what does that mean for you?

Paul Ballew: It means being that trusted advisor. If we're building capabilities for the business, making sure the business is leveraging, we're learning from how they're leveraging it, ensuring that we're getting the desired outcomes, and to be jointly accountable, which is a big part of this journey. This is a partnership. We're not just there to throw it over a wall.

Michael Krigsman: Again, at the end of the day, it sounds to me like your underlying mandate is to defuse data and analytics—from a technical standpoint, from a business outcome standpoint, from a cultural standpoint—through the organization in a permanent way, deeply infuse it through the organization. Is that a correct way to look at it?

Paul Ballew: It's a pretty good way of describing it. What I would say is using data and analytics to drive insights that drive workflows and decisions to optimize the business. That's what it's always about.

Can you bring the science to life? If you can bring the science to life in a systematic way, that's what the game is all about.

Michael Krigsman: With that, a huge thank you to Paul Ballew. He is the chief data and analytics officer of the National Football League. Paul, thanks so much for taking the time to be with us today.

Paul Ballew: Thank you, Michael. All the best.

Michael Krigsman: Everybody, thank you for watching. Before you go, please subscribe to our YouTube channel. Hit the subscribe button at the top of our website. Check out CXOTalk.com. We have amazing shows coming up all the time. Thanks for watching and watch the Super Bowl next week. Take care, everybody. Bye-bye.