Writing code was never the bottleneck!

Estimated reading time: 9 minutes

The AI hype train isn’t slowing down, but are we still focusing on the wrong parts of the software development life cycle?

While AI coding assistant vendors and early adopters are both quick to boast of massive productivity gains, recent studies show that AI might actually be slowing developers down.

But debating the effectiveness of this new class of tools misses the point entirely. The top-down push to adopt AI coding tools has largely focused on the wrong things: code generation.

As companies continue to invest in their AI capabilities, even sometimes justifying layoffs, 60% of organizations still struggle to effectively measure their impact on software velocity or stability, according to a recent LeadDev study. Do we need to adjust our focus?

Widely varying promises of productivity

This March, JPMorgan Chase said its engineering efficiency jumped by “as much as 20%” from using AI coding assistants. A recent Atlassian report found that 99% of developers reported time savings with AI tools, with 68% estimating that they save more than 10 hours per week. 

You aren’t alone if it feels like your teams are not quite hitting these heights.

The latest study out of the AI research nonprofit Model Evaluation and Threat Research (METR), found that, despite predicting they would be 24% faster at completing tasks, participating developers were 19% slower when using AI. 

Regardless of their actual effectiveness, developers like using these tools, and they are here to stay. According to a recent McKinsey survey, engineers find that, with generative AI, they are happier, more able to focus on satisfying and meaningful work, and more able to achieve “flow state.”

This improvement to developer experience shouldn’t be ignored. But when software quality, speed, and security are at risk, engineering leadership must weigh AI’s impact on the whole software development lifecycle.

Focus on the boring

“Devs might feel faster with AI, but the hard data could paint a different picture,” Ben Lloyd Pearson, director of developer experience at LinearB, said. “It really all comes down to how much toil AI reduces and how much time it frees up for developers to focus on more strategic work.”

Since engineers have consistently said they wish they had more time to write code, AI adoption should focus on where frustration and bottlenecks are prevalent. 

“For me, AI is most beneficial for boring things – tests, docs, etcetera – or as a guide that can help me generate some basic template code to get me started,” echoed Seth Rosenbauer, co-founder of the Joggr AI documentation tool.

Time and again, you eliminate both developer and cross-organizational toil when you focus on the boring.

“The cool AI use cases that are all shown on keynote stages are not the ones that are going to be used. Be ready to work in the boring,” said Jared Coyle, chief AI officer (CAIO) for the Americas at German software giant SAP.

“Everyone goes into the role [of CAIO] thinking: ‘I’m going to bring magical, agentic-powered robots that are going to change my business entirely.’ What you’re probably going to do is automated approvals for a while so people get comfortable with that. Be ready for the human aspect, because you will become a salesperson for making people’s lives better.”

Coyle says that the top AI use case at SAP, by far, is scanning and processing business expense receipts.

One of the reasons for JPMorgan’s bullishness is because it is running more than 450 back-office proof of concepts. So far, this has led to three impactful AI use case areas:

• Generative AI in the call center – An intelligent Q&A for representatives, integrated across the call center tooling suite to reduce cognitive burden. This is notably not a customer-facing deployment.
• Internal generative AI platform – A proprietary knowledgebase that the bank rolled out across 200,000 Chase employees within eight months.
• AI in software development – The bank uses JetBrains IntelliJ IDE for coding assistance, as well as AI to review code changes to provide feedback and enforce coding standards. The firm also has developed and integrated its own PRBuddy, which automatically generates pull request descriptions and highlights modified code components, with suggestions for changes and documentation.

Developers spend more than half an hour a day looking for things. Whether it’s tasking AI with creating a directory of services and who owns them, or putting a conversational chatbot overlay on top of your internal developer platform, empowering engineers with better documentation and searchability is a quick win.

The aforementioned Atlassian report invites developers to rank points of friction and time wasters. “Finding information” topped the list, but now the second spot goes to “new technology,” which could include various applications of AI. 

“With the adoption of AI being the biggest industry shift in the past 18 months, it would be hard to discount it as a contributor to this result,” said Andrew Boyagi, customer CTO at Atlassian.

“Typically, developers don’t adopt technologies that don’t directly support their workflow – forced adoption of AI tools has likely contributed to the result.”

More than just code generation

“AI came for code generation first because it was the easiest problem to solve,” Honeycomb CTO Charity Majors told LeadDev in a recent interview.

Just because code generation was the first major use case for large language models (LLMs), that doesn’t mean it’s the most important. Because it was never the thing holding developers back. In fact, AI-generated code can become a new hurdle.

Harness’s State of Software Delivery 2025 found that 67% of developers spend more time debugging AI-generated code, while 68% spend more time resolving security vulnerabilities. Respondents from the METR study observed that AI-generated code resulted in “more to fix up, in terms of code conventions, code style” and that they had to accept that they had to “do a lot of simplifying its code.”

“I’ve found it to be an unpleasant experience. When you spend time fixing AI code, it doesn’t feel like you are ‘creating,’ it feels annoying,” Rosenbauer said. “Even if it might be a bit faster to generate and fix, it takes away the dopamine hit of writing and creating great code.”

Stop making them wait

“Devs don’t need AI to write their code, they need it to get out of the damn queue. Most of the frustration I see isn’t about building, it’s about waiting. Waiting for infra, approvals, or some process that exists ‘just because’,” said Tarak Bach Hamba, senior growth manager at Aikido Security. “That’s where AI could help, navigating policy bottlenecks, pre-validating checklists, catching config [and] security issues before review and unblocking safe deploys.”

Just don’t go rushing to have AI as your single code reviewer. As CEO of Quotient, Lizzie Matusov, warned, “That might sound efficient, but research shows it weakens the responsibility developers feel toward code quality and social accountability.”

Peer-to-peer feedback is a powerful intrinsic motivator, research out of the University of Southern Denmark found when it examined the impact of introducing LLMs into code reviews. Participants expressed that peer reviews left developers with a sense of individual responsibility and collective accountability for code quality. 

But, when it shifted to AI code reviews, Matusov wrote in her analysis, “Without a human audience, there was no reciprocity – no social contract to uphold.” 

Don’t just look within the inner and outer loops of software development either. According to Nutanix’s technical marketing director Jose Gomez, developers are also waiting around for code to be deployed to production. These bottlenecks are most likely networking, security, and/or legal’s manual approvals that are ripe for AI automated guardrails.

Train, measure, improve

Another reason for the AI productivity perception gap is that there’s a learning curve with AI, steepened by the fact that it’s hard to study what you really can’t understand.

“Using AI code assistants effectively is a skill that needs to be learned and one with no existing, proven body of work in developer education,” said Kevin Swiber, CEO at Layered Systems.

Adopting AI, just like any other disruptive process or workflow change, is hard. 

Junior engineers often lack the critical eye or the sandboxes to test out AI-generated code first, let alone grasp the complexity that must go into deploying a fleet of AI agents on your behalf. This will only get worse as many organizations freeze hiring junior roles under the false assumption that they can be replaced by AI.

Instead, they should be investing in training. Earlier this month, Canva had every staffer drop everything for a week, as all hands effort dedicated to AI upskilling. 

Remember that it’s only going to get harder with the influx of agentic AI doing things on behalf of employees. “It’s certainly a lot of work to carefully steer agents,” Swiber said, before laying out some of the tasks from their experience working with agentic AI:

• You’re creating very detailed requirements docs with discrete groups of tasks. 
• You’re figuring out a dependency tree for the work breakdown structure. 
• You’re reviewing potentially tens of thousands of lines of code per day. 
• You’re iterating on top of all that, keeping in mind that the LLM capabilities for generating refactored code aren’t equivalent to the LLM capabilities for generating new code. 

“It’s a whole new set of skills. I haven’t yet figured out how a junior developer could use these capabilities most effectively,” they continued. “Working with LLMs requires constant ‘mentoring’ of a machine with behavioral issues.”

The measurement problem

And constant measurement. It’s difficult to close the perception-reality gap when every organization is measuring AI in a different way. 

“If organizations are measuring productivity based on lines of code or number of pull requests, they will undoubtedly see AI as a way to cut costs with layoffs,” Swiber said. “But, as it turns out, those are terrible metrics of productivity. I’ve spent a long time with these AI code assistants. I’ve been coding for 30 years. It’s difficult to get them to produce high quality work, and it takes a whole new set of skills we haven’t even begun to identify as an industry.”

Measure AI at different levels of seniority – both within an organization and the tech industry as a whole. Even at Big Tech companies, senior engineers are the ones seeing the biggest AI productivity gains. 

But also don’t forget to look at AI’s risk to long-term code quality and maintainability. 

Last year’s DORA report found that a 25% increase in AI adoption led to a 7.2% decrease in delivery stability and a 1.5% decrease in delivery throughput.

Organizations should be measuring the impact of AI on the software development lifecycle, as well on the long-term maintainability of code.

“Have we all forgotten about DORA metrics, why they’re relevant, and the fact that they’re the best predictor of organizational performance?” Cameron Brooks, senior director of engineering at Reveleer, said. “Writing code was never the bottleneck!”