Estimated reading time: 7 minutes
Key takeaways:
- Set kill criteria before you start. Define your exit conditions upfront – a timebox, a cost ceiling, a confidence threshold.
- One person owns the final call: assign a single senior “dissenter” with authority to pull the plug.
- Saying no is a skill! Leaders who can’t say no early will drown in review debt and crushed morale.
In Silicon Valley, there’s a timeless mantra: fail fast, fail often. While the debate’s out on whether or not it’s sound business advice, senior software engineering leaders know firsthand how much of today’s tech is built on the failed experiments of the past, hidden in commits gathering cobwebs and yet-to-be emptied Trash bins.
What’s arguably worse than failing often? Stagnation with drawn-out IT projects that lack direction.
“Data warehousing projects are notorious for this,” says Joel Carusone, SVP of data and AI at NinjaOne, an IT automation company. “I don’t know how many projects I’ve seen that have been going on for two years and still haven’t been successful.”
In fact, only 31% of software projects succeed, and 19% of projects are canceled before even reaching production, according to the CHAOS Report, a longstanding report on software project outcomes from The Standish Group.
However, early project abandonment isn’t necessarily negative in the long run. Tempo’s 2026 State of SPM report suggests that teams who frequently cancel more projects actually deliver better outcomes overall.
The thing is, knowing when to pull the plug is tricky. Too often, projects drag on because they begin without clear roadmaps or boundaries, and people get attached to their work and fall into the sunk cost fallacy.
“Engineering resources are scarce and there is a sunk cost versus expected value,” says Carusone. “You may have invested time, but if you don’t have a path to the finish line, it’s often time to call it a day.”
Below, we’ll explore the criteria Carusone uses to guide highly efficient software lifecycles at NinjaOne. Knowing when to pivot – or retire an idea altogether – requires goal-setting and upfront controls, paired with the right judgment and team culture to make informed decisions without second-guessing.
Your inbox, upgraded.
Receive weekly engineering insights to level up your leadership approach.
Set kill criteria upfront and timebox it
“If it’s using engineering resources, nearly everything we do has a predefined timebox on it,” says Carusone.
At NinjaOne, new software projects have predefined kill criteria with time-boxed controls. This is often represented as a certain number of engineering hours dedicated to a project, whether it’s data exploration, traditional planning, or new feature work.
If a team doesn’t have a rudimentary proof of concept after it reaches its set time limit, which is usually a matter of weeks, or can’t see a viable path forward, they step back and decide whether to pivot or scrap the idea entirely.
Sometimes, for this initial time trial, NinjaOne focuses on whether the most unclear component of the project can be realized. For example, when developing a vulnerability tracking feature, the biggest unknown was whether they could successfully match unnormalized data to known software vulnerabilities with enough confidence.
“Our kill switch was whether we could get to a 90% confidence on software comparisons,” explains Carusone. “We got to the end of the timebox, and we weren’t there.”
In this scenario, they ended up pivoting their engineering strategy to a hybrid approach, combining creative coding and large language model (LLM) based approaches to increase confidence from 80% to their goal of 90%. By timeboxing the initial unknown, they avoided substantial wasted efforts.
Review projects often to assess continuation
Kill criteria shouldn’t only exist at the start. As a project moves forward, leaders need regular checkpoints to decide whether it’s worth continuing.
Here, there are some early warning signals to track. For Carusone, cost is a top factor, especially for LLM-based features or machine learning components, which can have more ambiguity or opportunities for failure.
Another factor is financial viability. “If the target price is not competitive with the market, that’s another reason to kill the project,” he says.
To gather a holistic perspective, NinjaOne involves other departments early on to evaluate the required expenses, market fit, and pricing considerations, which can make or break a project. “Everything we do has a dedicated product manager or multiple product managers that are monitoring these things on a daily basis.”
Upfront diligence also includes verifying that an open-source version of what you plan to build doesn’t already exist. “It’s rare that something is going to be completely greenfield.”
It’s especially important to monitor net-new initiatives that have no prior art to work off of and set time-boxes at smaller, more constant intervals, says Carusone. For these projects, he’s often meeting with team members on a daily basis, providing guidance from past experience on what has or hasn’t worked before.
More like this
Designate a leader to make the final call
Once you start operating this way, teams with a good working relationship are typically adaptable to project abandonments and quick pivots, says Carusone. “It becomes inherent and ingrained in the team, and comes pretty naturally in time.”
Still, it’s human nature to be proud of your work after sinking time into it. As Carusone admits, “it can be hugely demoralizing to work on a project for months when it gets canned.” So, how does an engineering team make tough calls without placing blame or hurting feelings?
One method is to designate a “single dissenter kill switch.” This is a single person, usually a principal, staff-level, or otherwise senior technical contributor, with authority to trigger the decision to shut down a project at the end of a timebox. “It’s enabled us to get around indecision,” he says.
At NinjaOne, a big question for dissenters is: ‘would we start this project today knowing what we know now?’ If the answer is yes, the project is on the right track. If not, it’s a sign it needs to be re-evaluated or terminated.
As other engineers see why projects become derailed they become more likely to speak up early on and set criteria upfront. “The broader team sees and learns from senior folks and sees the types of projects that get killed,” he adds.
On the other hand, sometimes you do reach the point of no return, where larger commitments must be made to progress internal experiments toward general releases. If there’s a path forward, a market opportunity to justify it, and all performance, scalability, and telemetry requirements are satisfied, usually the project gets the green light, he says.
London • June 2 & 3, 2026
⏰ Less than a week to go. Last chance to join LDX3.
Be prepared to say no
That said, the majority of projects never see that green light. “For every ten things we explore, maybe we’ll invest in one,” says Carusone. “You have to get really good at saying no.”
Today’s agentic coding tools have the ability to 1000x engineering output, but greater velocity doesn’t mean higher overall quality. Nor does it guarantee all projects will reach production, let alone deliver an ROI to the business.
However, it does mean more code to review. This places more responsibility on leaders to decide which ideas are worth pushing through the last mile. It also means they’ll probably be saying no more frequently.
By pulling the plug early on, leaders not only avoid wasted efforts and cost runoffs, but they avoid worsened morale. “Stopping at the two week mark versus the two month mark is wildly different in terms of morale,” says Carusone.
Of course, knowing when to say no is easier when teams have done the upfront work to define what they are actually trying to build. Many failed software projects stem from a lack of requirements gathering, says Carusone. To avoid this, he encourages teams to find a very specific problem they are trying to solve and narrow the scope early.
“Figure out early on what the requirements are, and find a very specific problem you’re trying to solve,” he says. “Even if it will have hundreds of features, narrow the scope early on to a really small set of things.”
All in all, timeboxing with tight controls, setting single dissenters, and continuous evaluation are helpful guardrails when experimenting and testing new projects. These tactics take emotion out of the decision-making picture, and help engineering leaders avoid sinking valuable resources into an idea that might be spoiled from the get-go due to unforeseen circumstances.
“If 90% of the things you’re taking on you don’t expect to be successful, you’ve got to have those criteria upfront.”
