Estimated reading time: 3 minutes
Key takeaways:
- Meta’s AI “pods” boosted code changes 220%, but shipped features rose just 36% while incidents jumped 40%.
- Across 500+ orgs, PR throughput rose 37% while developer experience actually fell.
- Right-sized teams are five to eight people, not smaller, and shrinking should come last, not first.
Earlier this year Meta thought it had found a solid model for the future of the AI-native engineering team. Rather than bloated traditional product groups containing 10 to 20 people, it wanted to shrink down to “pods” of three to five workers supported by AI tools and agents.
The results looked positive – at least according to one metric. Code changes to Meta’s internal software platforms and infrastructure reportedly increased 220% year-on-year.
However, as previously reported, metrics don’t tell the full story. The number of changes which actually resulted in new or improved features reaching users rose a more meager 36%. At the same time, major technical and security incidents increased 40%, while the amount of time employees spent firefighting them rose 70%.
Meta’s experiment showed what many engineering managers are realizing: AI can make developers produce more stuff without necessarily making the organization produce more useful things.
Your inbox, upgraded.
Receive weekly engineering insights to level up your leadership approach.
More than numbers
“We are often falling in love with our output numbers and not paying enough attention to our outcome metrics,” says Brian Houck, distinguished scientist at developer intelligence company DX.
The vendor’s latest research across more than 500 organizations found median weekly pull request throughput rose 37% over four quarters. Yet its Developer Experience Index – a composite measure of how effectively and frictionlessly developers can do their work – fell from 67 to 65, while the proportion of engineering time spent building new capabilities remained broadly flat.
Combined the findings show that writing code faster matters rather less if engineers then spend time struggling to review, test, deploy, and operate it.
“The fundamentals of what make for great engineering systems are still true,” says Houck, but AI exacerbates what were previously minor issues into major ones. A senior developer who could once review everything now might face a torrent of AI-generated changes.
That should change how leaders think about the minimum viable engineering team.
Needing a rethink
Peter Bell, founder and CTO of Gather.dev and author of the forthcoming O’Reilly book Scaling AI Adoption in Engineering, reckons two roles become particularly important: product engineers who understand the customer and domain become more vital than ever, as do platform or developer experience engineers who build the machinery that keeps AI-generated work safe.
“Their job is to build the harnesses, build the evaluations, build the pre-commit hooks, and build the merge queues,” Bell says.
Bell reckons that an ideal-sized team is around five to eight people, even if only one or two are working directly on the same repository. Part of that is down to technical pragmatism: several engineers aided by agents can easily generate an unmanageable number of simultaneous changes.
There’s a human element too. “It’s nice to have a group of five to eight humans because that works for humans and it creates a sense of connection,” he says.
More like this
Honey, I shrunk the engineering team
Still, managers love to eke out any efficiencies they can. If they choose to do so, any shrinking of teams should come near the end of an AI transformation, rather than the beginning.
Bell recommends “small, low-risk incremental experiments” where organizations figure out which tasks can safely be handed to agents, then gradually phase out human review where they can.
Meta seemed to try the opposite approach – to radically redesign the organization first, then hope the technology can carry the extra load. It appeared to backfire badly.
“It is a sign that it is very hard to do this right, and more people than not will fail to do so initially,” admits Bell.
However, that doesn’t mean reworking teams isn’t important to adapt to the new AI age.
