Infrastructure is a business decision

Two non-technical metrics to keep operational costs under control

Tech can feel like black magic to founders. When engineers work that magic, it can be awe-inspiring and intimidating at the same time. Leaders without a background in software development have to trust their technical specialists to get the right things done.

While everyone has an opinion on UX and product features, most people stay away from the sorcery of the back-end and its algorithms. It's intimidating, so they leave that up to the experts. And there is no part of the product that non-technical people shy away from more than the infrastructure.

Infra is as far removed from feature and product work as you'll get. It's the machines and services your product runs on. It's boring networks, servers, and databases. The only contact most founders have with infrastructure is that huge AWS bill at the end of the month.

During our technical consultancy assignments, we often see startups with monthly cloud bills of over €10K. While there definitely are valid reasons to get to such a price tag, more often than not, it’s a sign of operational costs that have grown out of control.

While founders shouldn't be hands-on with their software’s infrastructure, it's a bad idea to give the engineers free rein here. Infrastructure is a business decision as much as it is a technical one. It defines the operational cost of your product—and that's more than just the hosting bill.

Keeping an eye on the real price tag of infrastructure

Running software in the cloud is more expensive than it looks. Amazon and Azure have generous free tiers, and Platform-as-a-Service solutions like Vercel allow your teams to get started on a dime. But hidden behind generous credits and limited Starter accounts, looms the real cost. Once your product gets some traction, those costs start to skyrocket.

Cloud platforms allow us to evolve our infrastructure along with our product, and that's a good thing. It eliminates the need for our teams to be first-time-right. But as a result, developers have been taught to neglect operational costs. They get some credits to play with that new Azure database and believe that, since it's currently free, the real thing can't be that much more expensive.

The real thing is always much more expensive.

If we look at the cheapest databases on the Azure cloud for example, we can start with a managed Postgres instance around €15/ month. That’s pocket change for a small database that can scale as we grow! But as soon as we get some actual traffic there and want to set up high availability, that price will rise to around €300 per month. That’s a twentyfold increase for still a very basic database! We see the same effects play out with servers, storage, and every other cloud service out there. We can start on a dime but soon need to upgrade at a premium.

When reviewing the operational cost of your product, always look at the real cost, not just the bill after deducting the credits. Many founders have been caught in expensive vendor lock-in once those credits run out.

Complexity and the cost of change

The second thing a founder should be aware of is the overall cost of change. How expensive would it be to evolve the architecture later down the line? How much effort does it take to just keep the product operational? While it's often difficult for founders to assess the technical complexity of their product, there is a metric you can track: How much of your development capacity is spent on infrastructure?

During technical due diligence audits, we often encounter teams with an overcomplicated architecture. They are building an early SaaS product that runs on an auto-scalable Kubernetes cluster. That should be an obvious red flag. For startups, the complexity of infrastructure should match that of their product. A regular SaaS product with a handful of users should run on a pretty simple setup. Yet these teams have complicated systems designed to support millions of transactions. Such premature scalability comes at a cost. Some teams spend half their time tweaking and tuning the infra. Some of them hire dedicated DevOps engineers or full-blown Platform Engineering teams. That's plenty of development capacity that is not dedicated to solving your user's problems.

In order to gauge whether their infrastructure is at the right level of complexity, founders should keep an eye on the ratio between product development and infrastructure maintenance. A quarter of the capacity is wasted on infra if a four-person development team has a dedicated DevOps engineer. That's a clear sign to grow the team or reduce complexity.

Founders should be able to get a feel for this ratio. How many hours per week does the team spend on maintaining and evolving their infrastructure?

Keeping infrastructure complexity in check can be pretty intimidating for non-technical founders. However, it's vital to prevent the product's total cost of ownership from spiraling out of control. By keeping an eye on the real hosting bill and its impact on development capacity, founders can help their teams make the right decisions for the business.

Keeping costs under control

Now that we know how to spot the two main drivers of operational costs, the next logical step is getting them under control. An experienced CTO has a few tricks up their sleeve to help keep these costs down.

Rightsizing

When developers set up new infrastructure, they provision it for the potential future traffic rather than today’s load. During our technical consulting assignments, it’s common to find overprovisioned hardware, and in a way, that makes sense. Developers try to prevent future performance issues by setting up overly powerful servers. But in a world of on-demand infrastructure, that doesn’t make much sense. Picking the right infrastructure for the state of the product is an art in itself, but it’s surprising how much money an experienced CTO can save by choosing a more modest server setup.

Prepaying

Cloud vendors allow us to scale our infrastructure as we grow. This on-demand infrastructure is a powerful but pricy tool. But not all cloud hosting needs to be on-demand! Most vendors allow you to reserve and prepay resources as well. If you’ve reached a stable, rightsized infrastructure, you can save up to 50% just by prepaying for the year ahead. This is a quick win as it is purely administrative and doesn’t impact your dev team at all. 

Standardize and simplify

Cloud vendors offer a plethora of tools and services that attract developers. Every AWS conference is a showcase of shiny new toys. Who doesn’t want to tap into the power of serverless applications and fancy graph databases? The problem with these services is not just their hidden cost and steep learning curve — it’s their expensive vendor lock-in. Once you’re running on Lambda and DynamoDB, the cost of switching to the competition might become prohibitive.

Most startups don’t need these specialized tools, so we often advise sticking to the tried-and-tested server-with-a-database setup. Not only does that reduce complexity, but it also allows you to move to a different cloud for financial reasons.

The world of infrastructure is as technical as it gets, yet it’s vital for founders to look at the operational cost of things. While keeping those costs under control requires extensive technical experience, most founders can get a long way by keeping track of the real monthly bill and the cost of change. 

So, how does your team score on those two metrics?