Table of Contents

In the last couple of years, I’ve come to work a lot with microservices and Kubernetes (and who hasn’t?) and the more I did so, the more I started realising that today, projects and architectures are defaulting towards microservices, not even giving a thought should something start as simply as possible, which would be a monolith (yes, you read that right, I just suggested a monolith for your project).

Things do need to scale and bigger projects that cost/make more money should be designed for scale from day one, there is no arguing there.

However, it’s almost 2024 and CPUs are much faster, there is more memory, networks are better and a very simple 3-tier architecture can get us so far that I think very few people realise it.

Why this matters and what got me triggered?

Value. If you remove all the complexity of you distributed architecture, peel off the Kubernetes layer, you can deliver pure value so fast that your mind will explode.

It’s very nice to look at Netflix and Shopify and say - wow, these people know the secret sauce, they run all these cool setups and they build for scale, we gotta take notes here.

But the truth is that their architectures didn’t start from day 1 as 100 microservices. Nor 10, for tahat matter. They have naturally evolved into those architectures and nobody talks about the pain points and down sides of those setups.

It took Netflix seven years to complete their cloud migration - they started in 2008 and finished in January 2016 - and God knows how much money. Seven years! Most startups don’t live long enough to finish a migration like that.

Last week I listened to a Podcast “Screaming in the Cloud” and the guest was Kelsey Hightower, talking about his retirement, among many other things.

What caught my attention was a comment that said:

I think a lot of people don’t realize how far you can get with, like, three VMs, a load balancer, and Postgres. My guess is you can probably build pretty much any clone of any service we use today with at least 1 million customers. Most people never reached that level, I don’t even want to say the word scale, but that blueprint is there and most people will probably be better served by that level of simplicity than trying to mimic the behaviors of large customers or large companies with these elaborate use cases.

So that got me wondering for the entire morning - how far you can stretch the 3-tier architecture to scale?

Can you really have 3 VMs, an LB, a Postgres and serve millions of customers?

Can you really build a clone of anything with this setup and make it work?

Hold my tea!

Setting up the infra

As promised, the infra is only:

  • A Load Balancer
  • Virtual Machine x 3
  • Postgres

I will use AWS, because it’s what I work with most but this can be applied to any cloud provider or bare metal.

The code would look something like this:

# 1 x Load Balancer
resource "aws_lb" "app" {
  name               = "boring-architecture-alb"
  load_balancer_type = "application"
  subnets            = aws_subnet.public[*].id
  security_groups    = [aws_security_group.alb.id]
}

# 3 x Virtual Machine, one per AZ
resource "aws_instance" "app" {
  count                  = 3
  ami                    = data.aws_ami.al2023.id
  instance_type          = "t3.medium"
  subnet_id              = aws_subnet.private[count.index].id
  vpc_security_group_ids = [aws_security_group.app.id]

  tags = { Name = "app-${count.index}" }
}

# 1 x Postgres, managed, with a standby so we can sleep at night
resource "aws_db_instance" "postgres" {
  identifier           = "boring-architecture-db"
  engine               = "postgres"
  instance_class       = "db.t3.medium"
  allocated_storage    = 50
  multi_az             = true
  db_subnet_group_name = aws_db_subnet_group.this.name
  skip_final_snapshot  = false
}

Building a clone

Well, I am trying to prove a point here so let’s not get too much in the weeds of building and let’s do a dummy service.

Here is the idea:

  • Backend only, for simplicity
  • Expose some CRUD endpoints
  • Some inserts and reads with the database
  • Simulate business logic processing of 100ms
package main

import (
	"database/sql"
	"encoding/json"
	"log"
	"net/http"
	"os"
	"time"

	_ "github.com/lib/pq"
)

type item struct {
	ID   int    `json:"id"`
	Name string `json:"name"`
}

var db *sql.DB

func main() {
	var err error
	db, err = sql.Open("postgres", os.Getenv("DB_DSN"))
	if err != nil {
		log.Fatal(err)
	}
	// A connection pool per VM. This single block matters more
	// than most architecture decisions people agonise over.
	db.SetMaxOpenConns(25)
	db.SetMaxIdleConns(25)
	db.SetConnMaxLifetime(5 * time.Minute)

	http.HandleFunc("/items", handleItems)
	log.Fatal(http.ListenAndServe(":8080", nil))
}

func handleItems(w http.ResponseWriter, r *http.Request) {
	// our "business logic", the thing the diagrams never show
	time.Sleep(100 * time.Millisecond)

	rows, err := db.Query("SELECT id, name FROM items LIMIT 50")
	if err != nil {
		http.Error(w, err.Error(), http.StatusInternalServerError)
		return
	}
	defer rows.Close()

	items := []item{}
	for rows.Next() {
		var i item
		if err := rows.Scan(&i.ID, &i.Name); err != nil {
			http.Error(w, err.Error(), http.StatusInternalServerError)
			return
		}
		items = append(items, i)
	}
	json.NewEncoder(w).Encode(items)
}

Then we point a load generator at the load balancer and see where it actually falls over:

k6 run --vus 200 --duration 5m load.js

But does it hold up? Let’s look at some examples

Stack Overflow. The site the entire industry copy-pastes from runs as a monolith on 9 on-prem web servers and a single SQL Server with a hot standby. Data says 6000 requests per second, around 2 billion page views a month, pages rendered in about 12ms.

Amazon Prime Video. Their Video Quality Analysis team took a serverless, microservice-y setup built on Step Functions and S3 and collapsed it into a single process on EC2/ECS. Result: 90% cheaper infrastructure. Going from microservices back to a monolith, huh?

Napkin math

Let’s doe some!

  • One modest VM, 4 vCPU, and our endpoint that burns 100ms of processing.
  • One core handles ~10 requests per second at 100ms each.
  • Four cores gets you ~40 req/s per VM.
  • Three VMs gets you ~120 req/s doing nothing clever at all.
  • 120 req/s = 7,200 requests per minute
  • = 432,000 requests per hour
  • = ~10 million requests per day

Now let’s all be honest with ourselves. Does your service do 10 million requests a day? And this is the pessimistic version, without the caching, tuning, and a fake 100ms of business logic you probably can optimise and reduce.

And when it does start dragging, your first move isn’t a rewrite. It’s a bigger instance. Vertical scaling is boring, unfashionable, but it just works. A single Postgres will happily take a beating long before you need to start thinking about sharding it.

So when do you actually need the distributed stuff?

I am not saying microservices are “bad”. I’m saying they are a trade-off, and the trade-off has to be a conscious one.

Reach for the distributed setup when you have a real problem it actually solves:

  • Team scaling: When 3 teams are stepping on each other in one repo and one pipeline, service boundaries buys independent deploys. That’s an org problem that can be solved with architecture.
  • Genuinely different scaling profiles. Your video transcoder needs 64GB of RAM, your API needs 2GB. Splitting those is sensible.
  • Different availability requirements. Payments shouldn’t go down because the recommendations engine fell over.

Notice what is not on that list: “we might need to scale one day” or “it’s best practice”.

Because here is the bill you’re actually signing up for on day 1:

  • A network in the middle of what used to be a function call. And networks fail in creative, deeply annoying ways.
  • Distributed transactions - or the eventual-consistency bugs you get instead of them.
  • N deployment pipelines, N dashboards, N runbooks, N on-call rotations.
  • Debugging that needs distributed tracing to answer “why was this one request slow”.
  • Eventually, a platform team whose whole job is keeping the platform alive.

Conclusion: boring is a feature

Start with the boring thing. A load balancer, a few VMs and a Postgres will take you further than you think - further than Stack Overflow’s traffic, and almost certainly further than yours.

Build the simple thing. Measure it. When it actually breaks, you’ll know exactly which part broke, and you’ll have real numbers telling you what to fix - instead of a guess you made on day 1 based on a conference talk about a company whose problems you do not have.

The architecture that lets you deliver value fastest today, and that you can actually debug at 3AM, beats the architecture that would theoretically scale to a traffic level you will never see.

It doesn’t mean microservices are wrong. It means the ceiling on the boring setup is much higher than the industry acts like it is - high enough that most teams will never reach it, and will spend years paying for distributed complexity they were never going to need.