High-Performance Engineering Management
Given the scarcity of digital business talent, companies must find different ways to attractive top employees and lower turnover.
Watch on YouTube
08:58 
Given the scarcity of digital business talent, companies must find different ways to attractive top employees and lower turnover. In Silicon Valley, typical tools such as bonuses, high salaries, and perquisites might not be enough, because there’s always a company that can outbid you for new talent or entice away your best employees.
To address these challenges, New Relic has changed how it structures and manages teams to gain buy-in and engagement from all team members. Just as increasing the level of engagement between a buyer and brand creates customer loyalty, increasing engagement between an employee and a company can foster employee loyalty.
Mikey Butler, Senior Vice President of Engineering at New Relic, explains how New Relic changed how it defines projects to emphasize product and business goals and form teams around project charters. The engineering group uses New Relic products to measure the results.
Michael Krigsman: I'm Michael Krigsman, industry analyst and host of CXOTalk. And, we are at Future Stack '16, which is New Relic's conference here in San Francisco. And I have the privilege of talking with Mikey Butler, who is the Senior Vice President of Engineering at New Relic. Hey Mikey, how are you?
Mikey Butler: Good, Michael. How are you doing? I'm honored to be here.
Michael Krigsman: Thank you so much! So, you run engineering at New Relic, and very briefly, what does that encompass?
Mikey Butler: Well, it basically encompasses two large responsibilities. One is actually putting product out the door. And approximately two thirds of the 200 people that report to me focus on putting product out the door, and one third focuses on actually keeping the site up. As you know, we're a SaaS offering, so we've got 24/7, 365 responsibility; so one third of the team is focused on that. And that's what we do day-in and day-out.
Michael Krigsman: You have spend a lot of time thinking about what we might call, "high-performing engineering team," ...
Mikey Butler: Yes.
Michael Krigsman: ...and the culture around that. So please, share with us your thoughts.
Mikey Butler: Okay. I would classify my thinking into two buckets, and I use the phrase "reactive and proactive." "Reactive" tends to be the typical blocking and tackling that we're all familiar with: getting engineers to produce code more quickly; having it get through the test cycle more quickly; having it aggregate into a release more quickly, with lower friction; that's the reactive side of things. And that typically, is where it tends to end in most organizations. I was blessed in the case of New Relic to have a very forward-thinking boss. My boss, Jim Gochee, who's the Chief Product Officer. And, he decided to do something very bold, and I was more than happy to sign up for it. And that's something we call "Project Upscale," and it's a complete re-do of how we do engineering. And, it is very state of the art.
And so, what we're trying to do with Upscale is to rethink completely what is the business of an engineering team. It's more than putting product out the door. It's actually properly targeting the efforts of a team, which roughly is six people across all disciplines: so managers, product managers, team leads, and individual engineers; having them focus on the right things; and at the same time having them align with their passions. If I hand out a remit to a team and say, "I want you to work on X," some percent of that team may be interested in that; some percentage won't be interested in that. But, what we created with Upscale is this sequence, or a set of desired deliverables ─ typically twelve to fifteen of them ─ and said, "Alright. Of the forty teams you have, select the ones you want to work on. And why do you want to work on it? And tell us what's in in for the business." We then go back. We look at their proposals. We bless some. We say, "No, try again with others." And when the dust settles, we actually have a good match between what the business needs and what people have as passion. And, therefore, as you well know, if someone's passionate about something, you get a lot more out of it than if they're just in it to turn the crank and do their job.
Michael Krigsman: So it's a very systematic way of creating a culture, where people feel highly invested in the work that they do.
Mikey Butler: Absolutely. For example, one of the nuances there, to give you just how far we took it, we actually gave all 250 or so of our engineers and product managers the opportunity to select which team they were on. It caused, actually, a lot of anxiety inside of the company when we told our CEO, and luckily, he basically signed up for it. But in the beginning we said, "Yeah, we're going to basically let everybody decide what job they want to do." It's like, "Oh my God! How long will that take to settle and what will happen on the other side of that? What will happen to product going out the door? All of that turned out to, let's say, work out perfectly. It couldn't have been more of a textbook perfect outcome. But, if we actually let people decide what team they're going to be on, and every six months or so they get a chance to select another team, the advantage to us [is] that we know the passion is aligned, the advantage to the individual is that: as they follow their passions, they don't have to leave the company ─ that there's a guaranteed opportunity every six months or so to align yourself with your passions. And as you know, in the technology space, that happens pretty often and very fast. And so, people tend to stay much longer. Our attrition rates are far below average as a result of making this move.
Michael Krigsman: So, many companies find that making this kind of change is difficult, because change is hard. So, would you share with us some of the lessons and the things that you learn that make it ...
Mikey Butler: Yeah. Well, part of it is just good old-fashioned gumption. The management team is going to sit in the middle of what will seem like a firestorm once you start this up. You're going to go down the path, and even the engineering team is going to have doubts about what you did. Certainly the teams around you that depend on the engineering team will see this as a moment of chaos, and you kind of have to just gut yourself through that. Find courage and intestinal fortitude when you need it, and wrap your heart around the vision, and keep that… hold to yourself and to the team. And that actually carried us through. The interesting thing is, we hired a consultant to come on board. He's done this with about a dozen other companies, and they all have about the same journey, which is ... It is so easy to innovate the technology and the tools, and far more difficult to innovate how teams do their work. And so that tends to be an exception-case in our industry, and therefore tends to push a lot of the, what I'll call, the "organizational calculus" to its limit. But, if you can go there and get through that, the upside is tremendous.
Michael Krigsman: How did you ensure that trust was part of that organizational calculus, because without trust, this would not have worked.
Mikey Butler: Yes, and so, trust was greatly enhanced by us looking at all of the forty teams, and asking them to elect representatives to actually drive the process. So management did not prescribe the outcome. We actually gave the team a set of business problems: We need the business priorities observed. We want teams to be passionate about what they do. We want them to have specific charters about what they do, and we said, "You go off and you design the system that responds to these constraints in the most effective way." So, not only was it a better outcome than we would have defined had we been sitting in our ivory tower of management, but it helped [with] the buy-in of the team, because their representatives were working with them to fine-tune as they went along. So, that built a lot of trust.
Michael Krigsman: And finally, New Relic is a very data-oriented company.
Mikey Butler: Yes.
Michael Krigsman: And so, what's the data that you look at? How do you know that you're successful at this?
Mikey Butler: Well, that's actually a big talent. So, we tend to drink our own champagne and so, all along the process of building code, deploying code, we actually are using the Insights product and the Analytics for Everyone platform, to continuously gather data. For me, this was actually one of the reasons I came to New Relic. You know, it was one of these things I spent a lot of time studying Information Theory when I was in school. And there's a correlation between how often things change; how fast they change; how frequently they change; or that the size of change, and the sampling rate that you need. And, what we were finding is that if you actually use the old-school method of what I call "metrics-based analyis," that you have to know ahead of time what you're interested in measuring. With event-based and insights-based product, you don't know what you want to measure half the time. You just measure everything, and then you have essentially the full data cube that you can spin however you want, and slice it any way you want after the fact, when you know what the question really is.
So that's what we depend on. ... You heard that we're getting something in the order of 25 trillion events per quarter into the New Relic NRDB backend system. We're probably responsible for a good percentage of that inside of our engineering organization. I don't know the percentage, but we use it a lot And what we do is essentially gather data all about the entire assembly line and how teams are operating. And then, at periodic reviews, we extract that data; rotate the data cube; slice it the way we want, and basically get some very, very interesting insights that we couldn't possibly have gotten if we hadn't used our own tool.
Michael Krigsman: Fantastic! Well, thank you so much!
Mikey Butler: It's my pleasure! Thank you!
Michael Krigsman: We have been talking with Mikey Butler, who is the Senior Vice President of Engineering at New Relic.