Intelligent Orchestration:
Software Delivery for the AI Era, with CEO of GitLab

The AI Paradox

Michael Krigsman: AI tools accelerate coding, but software delivery isn't getting faster. Bill Staples, CEO of GitLab, what's the bottleneck?

Bill Staples: AI and coding agents like Claude Code and Codex are accelerating the coding aspect of the developer's job, but they're creating downstream bottlenecks in overall software delivery. Things like code reviews, bug fixing, and security checks are all becoming overwhelmed with code. Teams and leaders are not seeing an overall increase in innovation velocity. We call that the AI paradox.

Fragmentation in tooling causes a developer productivity impact because those tools were not built by the same company in the same platform, sharing the same data. They're obviously disconnected and assembled by hand by individual developers or by the developer experience teams, and that fragmentation causes them to have to switch contexts all day long as they move the software through the life cycle. It causes them to have to keep the context about that software in their head.

How Developer Tools Create Obstacles

Michael Krigsman: Dive into how typical developer tools create these obstacles to the software development life cycle that interfere with developer productivity.

Bill Staples: They store their own context within their own data systems. In that context, you can open that up with MCP, but then you are limited in terms of identity and the corresponding security, potential security issues there, and then you're also limited based on cloud connectivity. Bandwidth or network throttling may apply, walled gardens in terms of data access, and what you actually are able to extract from that vendor's cloud may apply. And so, that fragmented context leads to issues.

We also have a trust issue in that even with great AI tooling, some individuals and teams are reticent to cede control to AI in some cases. Some organizations aren't ready to let certain aspects of their data into another cloud or into an AI vendor's control, and without that consistent verification and validation of processes, it's not clear which tasks are well suited for AI and which are gonna continue to require human approval.

There's also regulatory aspects with AI. There's a growing need, especially in countries outside the US, where there's heavy data residency requirements, and all kinds of new laws are coming up that are gonna drive governance requirements and standards on especially financially regulated companies to be very careful in how they use AI.

And then there's the budget questions. Where does the budget for AI come from? It obviously often needing to be offset other costs, and the cost of AI can be very high, and it can be unpredictable because often AI vendors are charging a usage-based model, given the underlying cost of service, which then results in more fragmentation and expansion of budgets. So all of those tool challenges come in the way of customers adopting AI across the software life cycle.

How AI Agents Add Complexity

Michael Krigsman: How do AI agents make this software development life cycle more difficult, more complex?

Bill Staples: Agents thrive on context, and so the first thing that they do is look at the context they have. But in order to move software from idea all the way to production, a series of tasks have to be executed, and without that shared context, the agent may end up creating more bottlenecks down the road.

Agents also require a fair amount of steering and coordination, and so if you're orchestrating agents across multiple tools and multiple vendors, you're also increasing the overhead required by an engineer to do that steering.

There's also then increased risk in terms of AI proliferation as teams look at a variety of different tools that are coming onto market, open source and commercial, and bringing those in without IT governance. There's a potential cost explosion associated with those because AI is powered with tokens, and they may or may not be on a common contract or under cost control by the organization.

And last but not least, trust is a real factor here. Do you really trust all of the vendors and the underlying LLM token providers with your organization's data?

Solving the AI Automation Challenge

Michael Krigsman: So how do we solve the issues created by AI automation in software development?

Bill Staples: It's not just a tool solution. It's not about swapping one tool for another or bringing more tools in. It really is a complex people, process, and technology challenge, rethinking the sequential process-based handoff of the software development life cycle.

It's time to shift to a more continuous innovation loop, where the agents are able to do many of those handoffs for us and also learn and validate against the initial business requirements and technical requirements that were laid out at the beginning and continue to iterate on those on their own to drive the right level of quality before humans need to get involved.

It's also important that we address the context challenge. Having fragmented tools, each with their own context, as I said, is not going to lead to the best outcomes. And so this is one way that we're excited here at GitLab because we've always taken a platform approach and given customers many different tools across the lifecycle to do all stages, like the GitLab Orbit service that we're building now.

It's also really important as we think about solving this, that we think about the execution environment. Oftentimes people say, context is king with agents. The more context, the more well-structured context you can give an agent, the better it's able to reason and make decisions and take action. And that context is often difficult to stitch together through MCP across multiple vendors.

And so having an environment where both the compute or the agents, as well as the data, are co-located, we call that an inner loop architecture, where the agents and the data and context are all within one network and one system, we believe are going to lead to a higher quality and faster and lower cost agentic outcomes.

And then finally, super important, especially with AI, that we trust the actions that are going on. So everything has to be secured, has to be logged and traced so that there's an ability to see in retrospect, even the reasoning that was done, the actions that were taken, by whom and when, in order to have an audit trail and understand exactly what was changed and why and by whom, including by the agents and the humans that were steering them. And so, that is also a really key part of the solution.

And again, it is dramatically easier to do that within a common environment with that inner loop architecture, where both the compute and data are co-located, and you've got a process where instead of having handoffs between humans and tools, you've got one continuous innovation platform that lets you do that inside one cloud.

And that's why many teams have found consolidating and going to a strategic vendor relationship. For example, with GitLab, we've shared with many of our customers there's like a 6-month payback period on GitLab Ultimate and a 483% ROI over 3 years when you adopt the very best of our platform of GitLab Ultimate, and that's a human-driven productivity improvement. We call our approach to solving this intelligent orchestration, and that's really what we're focused on now as a company across the platform, including our new Duo Agent platform.

Intelligent Orchestration

Michael Krigsman: Can you elaborate on this concept of intelligent orchestration?

Bill Staples: When we talk about intelligent orchestration, we're really talking about 3 things: workflows, context, and guardrails.

First, when it comes to workflows, I mentioned earlier, the process of software development has got to change, not just the tooling. We've got to move from beyond this stage-based process to a continuous innovation loop. And when it comes to AI, we've got to move beyond just the one-to-one AI chat-based experience that are pretty common now within IDEs like Cursor or with CloudCode. And we're moving towards this multi-agent, agent swarming approach, where those agents are working asynchronously on complex tasks for everything from taking a set of product requirements all the way through coding and testing and reviewing to a merge request, to things like security analysis, ongoing real-time security reviews, code reviews, and static analysis, to pipeline creation and pipeline monitoring and remediation as pipelines fail.

Now, let's talk about context real quick. Think about, for example, an agent making a code change. Historically, in current generation AI tools, has access to that project's source code, but it may not have access, probably doesn't have access to, for example, the issue or the epic that describes the requirements for that code, may not have access to the security scans of that code and yesterday's run that shows there's actually a security issue in the code block I'm looking at, for other reasons. The agent may not know which environment this software is going to be built for or where it's going to be deployed. And all of that context could shape and change and drive a higher quality outcome if the agent could take that context into its context window and reason with that.

With what we're building with the knowledge graph, because we index the code semantically, and we stitch together all of these other pieces of context related to that code, we can hand it over to the LLM in a very efficient way that lets it reason about the related entities of that code and get access to just the information it needs to make good decisions and take action. In some of our early testing here with GitLab Orbit, we've seen agents 40% more accurate and actually 25% faster, thanks to that well-structured, full lifecycle context.

And finally, with guardrails, it's really important that the system take into account the standards the organization has set and the regulatory and compliance requirements that it's under, and that the agents take on the same accountability, the same responsibility to act within those guardrails that the human engineers do. And so the ability to apply those in one place and have both the humans and the agents working under the same set of rules, that are policy-driven, that are declarative, they're well-documented, is a very important part of the platform that we're building.

We also think it's really important that the policy gates apply universally and automatically. Meaning whether the human wrote the code or whether the agent wrote the code, the same rules apply, the same auditing, the same governance is applied anywhere the software is authored or anywhere it's flowing, and that's done as part of our unified platform approach.

Michael Krigsman: Would it be accurate to say that it's this combination of the process, the platform, and the context that enables you to overcome the fragmentation that you described earlier?

Bill Staples: Exactly. Yeah, that's a great way of saying it.

Real-World Examples

Michael Krigsman: Can you give us some real-world examples?

Bill Staples: Even within GitLab, we've been building our platform and using it internally, obviously. And when I think about, for example, the workflow portion, we have been having our engineers use the platform for doing things like accelerated code development. We see engineers who are medium to heavy users actually contributing 2 to 3 times more merge requests than those who are light or no users.

There's teams who go through and build software and create MRs, often run into issues with pipelines failing. And I've heard stories from my own engineering team about, for example, when they use the built-in fixed failed pipelines flow that's part of dual agent platform, the ability to respond to a pipeline failure can go from hours to minutes.

And so again, having the execution and the data in one place, being able to apply agents across the software life cycle, really can yield some pretty dramatic improvements across the board to get to that full innovation velocity increase that we're looking for.

Keys to Customer Success

Michael Krigsman: What are the common threads that drive success among your customers?

Bill Staples: First, I see the full spectrum of customers in terms of their readiness and willingness to experiment with AI and to think about how it might truly accelerate innovation across their organization. And those who are more forward-leading, early adopters, willing to experiment, not just with the tooling, but especially with changing the processes around software engineering to actually be able to take advantage of AI, those are the ones that are seeing the early results.

And then second, and this is probably no surprise, but those who have done more intentional efforts around consolidation and aligning on a platform, they simply have much simpler lives in adopting AI because they get the benefits of shared context and a more intentional process already defined as part of their human engineering approach, which leads then to a simplified and more intentional agentic approach to software engineering as well, that can be in harmony as the humans and AI collaborate on the same platform-based approach to software engineering.

So, I think being intentional and also really being forward-leaning on AI and thinking about not just the tools, but the process changes required, are key factors to success.

How AI Changes the Developer Role

Michael Krigsman: How does all of this change a mid-level developer's job and their role?

Bill Staples: I think it shifts for developers to go from writing code to really helping define the requirements for that code, how the code needs to be structured, and even more broadly, how the software needs to be architected as part of the overall business. They also need to shift from pure implementation and caring about the syntax of the language and the lines of code, to validating that code, and not just validating it technically, but also validating it against the business requirements that were initially set.

I think their execution shifts from really being so tactical about implementing the work items that were assigned to them, to being really more strategic thinkers about the why. Why is the business asking to solve this problem? Why does the software need to behave this way? Not just the how.

It's really a chance for them to move into a much more strategic and creative individual contributor role, where they're an orchestrator using the superpowers of AI agents and coordinating across systems rather than being so narrowly focused on just pieces of a particular project.

I think there will be a lot greater emphasis on correct code. Not necessarily writing the code, but actually the ability to read the code and validate that it's good quality code and meets the business requirement, as well as well-architected services, is going to be an important part of the job. So things that maybe more senior managers and architects would do today become a mid-level person's opportunity. They essentially get this ability to rise in strategic importance and impact in the organization.

The ability to test and drive quality through the agentic outcomes is an important part of the role because, again, they're thinking about business outcomes that they're creating, not just a block of code that gets delivered into a project.

Michael Krigsman: This is a golden opportunity for developers to develop strategic skills, to move closer to understanding business needs, to orchestrate systems, and to guide AI agents.

Bill Staples: Absolutely.

Advice for CIOs and CTOs

Michael Krigsman: What advice do you have for chief information officers and chief technology officers to put these lessons into practice?

Bill Staples: If you're a chief information officer managing the shared infrastructure of the organization, obviously, one of your really important responsibilities is security and compliance of the organization. Oftentimes, that also involves analyzing or assessing AI as part of that responsibility, and it's really important to think about how software engineering is gonna transform, as I've shared, across people, process, and technology boundaries, and to support that dramatic transformation while ensuring the security and compliance of the business is intact.

And so if that means it's part of your purview to provide that to your software engineering teams, then I think looking at a platform-based approach makes a lot of sense. I think being forward-leaning on AI and setting the standards of the organization up front, so you can unblock adoption, is gonna be really important.

And if you don't have that as part of your purview, but you partner with, say, a CTO or with other parts of the organization to provide the developer experience into them, it's really important to understand their challenges that we've talked about here and support them in that transformation. I know my CTO and CIO here at GitLab are working really closely on exactly those set of things.

First Steps for Organizations

Michael Krigsman: Bill, for organizations that want to adopt the lessons you've been describing, what are the first steps?

Bill Staples: Step number one is audit the current level of investment and tools, where there is fragmentation, to consider consolidating. I think it's really important that you're measuring end-to-end flow on software delivery, not just coding velocity, but how long is it taking to go from idea to production?

It's also important, as we talked about, to think about security and governance upfront, even having, buying into a system that has AI governance built in with an audit trail and the ability to identify compliance gaps as part of the solution purchase.

Cost, we identified as a potential challenge, and so obviously, the simplicity is key here. The more you can aggregate and consolidate tooling, the better visibility you're gonna have into the true cost of AI and ensure that you're getting an ROI on that.

A lot of teams like to do pilots. I think that's a great way to try software and get an evaluation quickly on if it's gonna solve your team's needs or not. I think pilots are a great thing, but especially important as you think about consolidation and you think about teams and agents working across the lifecycle, that you consider piloting that, because it will require people, process, and technology changes. And doing that in a safe place outside of production environments is gonna be key.

As you think about autonomy and the ability for organizations to move quickly, I would say defining upfront the criteria that you want people to hold fast to versus the things that are a little bit more free form and they can make decisions on will only accelerate the organization's ability to make decisions and move on the AI opportunity. So those are probably the first things I would focus on.

Metrics to Measure AI Success

Michael Krigsman: And are there a few core metrics that folks can use to evaluate their progress on this journey?

Bill Staples: Yeah, it's one of the number one questions customers ask me is: How do I measure the real value of AI? I put it in a couple of different buckets.

First, end-to-end metrics, as I said, not just lines of code generated from AI or percent of hours saved coding from an AI, but think about overall cycle time. How long does it take to go from idea to production?

When you think about merge requests, this is the ability to take code written by AI agents and actually put it into production, that cycle is really important. So the time from a merge request or a pull request created until it's actually deployed, that ends up being a new bottleneck for a lot of teams. It's important to understand.

Deployment frequency, how often is code actually making it into production? That might be another bottleneck worth paying attention to.

And then change failure rate is also a great one to think about because it doesn't matter if you're cranking out way more code and getting it into the hand of customers in your deployment environment if the bugs are increasing. And so looking at pipeline failures and looking at bug escape rates, also super valuable.

The other last thing, it's been, I know, on lots of teams' minds for a long time, but keep an eye on mean time to recovery. When things do fail in production, how quickly are you able to get up and running again?

And then, of course, governance and quality. Looking at things like how many security vulnerabilities by severity and the trends there, your compliance audit readiness, and so forth, as well as the final category, business outcomes.

Are your developers' satisfaction increasing or decreasing? Because developer satisfaction directly correlates to software innovation and velocity. The time saved across the lifecycle, again, end-to-end, is really important to bake back into the capacity of the organization to innovate.

And then cost efficiency. Is your AI cost scaling with the value you're getting from the investment?

Michael Krigsman: Bill Staples, CEO of GitLab, thank you so much for giving us the inside scoop on using AI effectively in the software development lifecycle.

Bill Staples: Thanks so much, Michael, for having me on.