What does the journey look like when you’ve decided to buy?
About four years ago, I led a project to find the best solution to replace our in-house, buggy, slow, and unscalable reporting tool. And so I did (or at least tried).
As you can imagine, not all the considerations and decisions we took initially made sense over time, and, in hindsight, I would have made changes. In this article, I’ll share my learnings from the decision-making process and hope that it helps you reach the decision that is right for you when looking at build vs. buy.
Everything starts with requirements
I needed to find something better than the existing solution in every aspect that wouldn’t take more than a few months to integrate with our system – a piece of cake.
My initial question was whether to build or buy a new solution, and the answer I got leaned more on the ‘buy’ side. My managers wanted a fast solution and felt that building reporting tools wasn’t our core strength and not where we should spend our energy. I decided to explore two different types of solutions: an open-source-based solution that would allow us to build a solution quickly and without re-inventing the reporting wheel, and a third-party-based solution that only needed integrating with our system.
Up for the challenge, I started with a market analysis.
Third-party solutions
I asked around and quickly found out that there are dozens of different players in this market, from giant companies to small startups, all providing what seemed to me to be almost identical solutions.
I started to take calls with representatives from the different vendors and booked demos to understand the various offerings.
My approach was to look for tools used by people I know and tools I frequently saw in Google searches. In parallel, I figured that I needed to understand our must-have features to narrow down the options and make progress.
After talking some more with my stakeholders, these requirements seemed to be unique:
- Be able to work with our unique DB (Druid)
- Customizable look and feel (‘white label’)
- Integratable authentication/authorization system
I learned that solutions that answer these requirements are mostly labeled as ‘Embeddable Analytics’ tools. Having a name for what I was looking for made it much easier to search for options. I used Gartner and Forrester reports and online forums to look for the key players in this niche.
Open source
Another route I explored was the open source approach. I was looking to see if any project could offer all or most of what we needed. Without going into much detail, I’ll just say that after some looking around, it seemed that the paid solutions dominated this market, and that the open source options were mostly libraries you could use to save some of the development time. Sadly, there wasn’t anything close to being a ‘turn-key’ solution.
At the time, we used open source software, but mostly we used it without significant changes. I felt comfortable diving into the codebase of an open source project to figure out how it worked, but assuming my team could contribute to an open source project to add features felt risky. My strategy was to look for something we could use and extend.
After researching online, I found a few projects that had the chance of working for us. A few were libraries/packages we could use, and a few were fully-fledged reporting solutions. One thing I knew to look for was the community of the project: how active is it? Do people get help? Are there recent releases? Etc.
After reading online and tinkering with the different projects, I ended up with two candidates.
Presenting my results
I shared my deck with my VP R&D, architect, and relevant product manager during this process. I used the input of all three of them, so I felt good with my presentation for our CTO (he managed R&D, product, and operations). I presented what I thought were the two best third-party solutions and two best open source solutions with a technical design of how we would integrate them with our system, including work breakdown and effort estimation. I was proud of myself going into the meeting, and I planned to start with a discussion on buy or build (based on open source), make a decision, and finish with a decision on the best tool to use out of the ones I suggested. This was not how the meeting went. After I showed all the diagrams and numbers and charts and tables, my CTO simply asked me why I didn’t suggest using the business intelligence tool we were currently using for internal cases, and asked me to come back with this option in the deck. So I did just that, and after a couple of weeks, we redid the meeting, and lo and behold, the CTO went with the tool he suggested. As a side note, the open source approach went as soon as I gave an effort estimation of 8–12 months compared to 1–3 months with the paid vendor option.
Working on a solution #1
Since the requirements didn’t go away, I now had to have several sessions with solution engineers from that company to come up with a way to use their system like we wanted to.
After a couple of months of work by R&D teams on both sides, trying to get to a working proof of concept, we all realized it was just not going to work.
Working on a solution #2
After realizing the CTO’s tool wasn’t an option, we went back to the solution I suggested and signed a three-year contract with them. It didn’t take long (about a month) to get a working integration but we ended up spending much more time creating a custom UI that would use our internal widgets and integrate them with the third-party tool. We also did other customization to use that tool like any other microservice in our architecture. From the first decision meeting to a fully working solution, it took approximately ten months (like my estimate for an open-source-based solution).
Epilogue
When it came time for renewal after three years, we decided not to renew the contract and go with an open source option. Some interesting points that led to this decision were:
- It was possible to add new reports without R&D using a ‘Reporting Studio’ tool that came with the solution, but we only added a couple of reports in those three years.
- Visualizations were never used (even though adding them was pretty straightforward).
- The rush to integrate the solution caused tech debts that made the project very fragile after three years.
- This integration was very different from the rest of our stack, which caused confusion and human errors.
- Our confidence in our ability to contribute to open source solutions significantly increased, specifically with a tool that was useful for this use case.
I think the last point was the key to the decision. Without having confidence in our ability to influence the open source project in question, we would have probably renewed the contract. Going through the overhead of switching to another vendor would have been hard to justify.
Conclusion
Ultimately, I think we made the right call of buying a solution vs. building one. We are not in the business of building reporting tools. Implementing all the required capabilities even when leveraging an open source project was not worth the money we would have saved. But, there are still things I would have done differently.
Not all requirements are equal
If I would have challenged some of the requirements (e.g., visualizations, a customizable ‘look and feel’), we could have gone with a much simpler solution. We might have even been able to use the tool my CTO wanted. We would have needed to make many sacrifices to the requirements, but some have proven to be redundant in hindsight.
Think about maintenance
We had to do a lot of custom work to integrate the third-party tool with our system. The extensive customization and the fact that it was unique to this project made the project harder and harder to maintain over time. I don’t think we took this into consideration enough when deciding we must have all of these customizations.
Open source
To truly be able to leverage open source, you need to have a different mindset. I’m not talking here about using packages/libraries in the code you write. I’m talking about building products that depend on an open source solution in a way that would require the ability to make changes and improvements to it.
It may be a topic for a different article, but we have made great progress working with open source in the past couple of years. The best advice I can give here is to start contributing to projects without expecting anything in return – just to know how to ‘talk the talk and walk the walk’. It will allow you to engage with the owners/maintainers of a specific project if needed. Once you do it, you will realize open source is not as scary as it looks, and most projects are happy to get external help.
Either buy or build, not both.
Most of the issues we had came from the customization efforts. We should have stuck with the ‘buy’ approach and used the system the way it was meant to be used.
Final note
There is a phrase I like that goes like this:
‘Good judgment comes from experience; experience comes from bad judgment‘
My hope is that this article will help you make a better judgment based on the experiences I have shared here and to be able to make the best decision of ‘buy vs. build’ you can make. I also wish that you will also have the opportunity of being able to live through your decision and see if you made the right choice.
