A URL shortener on Google Cloud, without a database

Every morning the platform emails recruiters the candidates that matched their saved searches overnight. A few days later a second email widens the net : the same search, relaxed a little, with the profiles it brings in. Each link in that second email carries the whole search (the keywords, the filters, the location), base64-encoded in the query string, with the tracking parameters behind it. A link of several hundred characters.

Some recruiters never got them. Not the email : the link. Corporate mail clients and the security gateways in front of them cut long URLs, rewrite them, or flag the message. The recruiter clicked, landed on an empty search, and wrote to customer care. A real need, with a real cost, and the fix is a feature every developer has written once as an exercise : a URL shortener 😅

So we built one, without a database. Here is how.

What it has to do

A shortener is two operations. Write : take a long URL, give back a short code. Read : take a short code, redirect. The read path is the product ; the write path is plumbing. Four constraints shaped the design :

A link must work forever. A recruiter opens an alert three months later. A short code never changes meaning and never expires.
The read path has to be boring. One lookup, one redirect, nothing that can be slow or down on its own schedule.
Only we can write. The shorten endpoint is not on the internet. The platform calls it ; nobody else does.
It must not become an open redirector. A shortener that redirects anywhere is a phishing tool with our domain on it. Only our domains get shortened.

And one constraint of our own : the team is three people running two dozen services. Whatever we add has to need nothing from us once deployed.

The architecture

The rest of this post walks through it from the store outwards.

The store is a bucket

The textbook answer is a key-value store : Redis, Datastore, a small SQL table. All three are something to provision, secure, back up and pay for, so that a few thousand links can be looked up. The question I asked instead was : what do we already run that stores a small value under a key, durably, and serves it over HTTP ?

Cloud Storage. A bucket is a key-value store with eleven nines of durability, where the key is the object name and the value is the object body. The short code is the object name. The long URL is the object body. That is the whole data model.

func (service StorageService) Read(ctx context.Context, shortCode string) (string, error) {
  // an unknown code comes back as storage.ErrObjectNotExist : one round trip, not two
  reader, err := service.client.Bucket(service.bucket).Object(shortCode).NewReader(ctx)
  if err != nil {
    return "", err
  }
  defer reader.Close()

  url, err := ioutil.ReadAll(reader)
  return string(url), err
}

What it buys : nothing to operate, nothing to back up, a regional bucket in the same region as the cluster, and a bill that rounds to zero at our volume. What it costs : a read is an HTTP round trip to Cloud Storage, slower than a Redis GET by an order of magnitude. For a redirect that a human clicks in an email, tens of milliseconds are invisible. It would be the wrong store for a shortener on a hot path ; it is the right one for ours.

Pick the dumbest store that meets the constraints. Then check the constraints again.

The design is not mine. Cloudreach published urly-wurly, a serverless shortener on Cloud Storage, and I took the idea from there. What follows is how it fits our platform.

The read path : one route, one status code

The public service has two routes for health and one catch-all. Whatever path arrives is a short code ; the handler reads the object and redirects.

func Unshort(services *services.Services) gin.HandlerFunc {
  return func(c *gin.Context) {
    shortCode := strings.TrimLeft(c.Request.RequestURI, "/")

    if len(shortCode) > 0 {
      if url, err := services.StorageService.Read(c, shortCode); err == nil {
        c.Redirect(http.StatusMovedPermanently, url)
        return
      }
    }

    c.JSON(http.StatusNotFound, gin.H{"status": 404, "shortCode": shortCode, "error": "Not found"})
  }
}

The status code is a decision, not a default. A 301 tells the browser the mapping is permanent, and browsers cache permanent redirects : the second click on the same link never reaches us. That is only correct because constraint 1 holds. An object, once written, is never rewritten with another URL, so there is nothing a cached redirect can get wrong. A shortener that lets you edit a link’s target has to answer 302 and take every hit. Ours does not, so it does not.

The write path : short codes and collisions

The first version derived the code from the URL : a CRC32 of the string, encoded in base58. Deterministic, so the same URL always gets the same code. It lasted a week. Thirty-two bits is a small space, and with a daily batch of fresh links the birthday math gets uncomfortable in months, not years. A collision here is not a duplicate ; it is a recruiter landing on someone else’s search.

The second version generates a random code with shortid and lets the bucket refuse a code that is taken :

const maxDraws = 5

func (service StorageService) StoreWithShortcode(ctx context.Context, url string) (string, error) {
  for draw := 0; draw < maxDraws; draw++ {
    shortCode, err := service.sid.Generate()
    if err != nil {
      return "", err
    }

    err = service.create(ctx, shortCode, url)
    if err == nil {
      return shortCode, nil
    }

    var apiErr *googleapi.Error
    if !errors.As(err, &apiErr) || apiErr.Code != http.StatusPreconditionFailed {
      return "", err // Cloud Storage down or denying us : that is not a collision
    }

    utils.Logger.Error("short code collision", zap.String("code", shortCode))
  }

  return "", fmt.Errorf("no free short code after %d draws", maxDraws)
}

func (service StorageService) create(ctx context.Context, shortCode, url string) error {
  // the write succeeds only if no object holds this code yet
  w := service.client.Bucket(service.bucket).Object(shortCode).
    If(storage.Conditions{DoesNotExist: true}).
    NewWriter(ctx)

  if _, err := io.WriteString(w, url); err != nil {
    w.Close()
    return err
  }

  return w.Close() // the precondition is checked here : a taken code is a 412
}

The precondition is the honest part. Several pods generate codes, and I have not given them distinct worker ids, so the generator alone does not guarantee uniqueness across the fleet. A read before the write would leave a window : two pods drawing the same code, each reading before the other writes. DoesNotExist closes it. Cloud Storage creates the object only if the name is free, atomically, and answers 412 to the loser, who draws again. No lock service, no read before the write, no race. The loop gives up after five draws, an error from Cloud Storage is returned as an error, never counted as a collision, and the collision log would tell me if the draw space ever got crowded.

Before any of that, the URL is parsed and its registrable domain is checked against a short allow-list from configuration. Anything else is a 400. That is constraint 4, in eleven lines.

Two listeners, one process

The service is one binary that starts two Gin routers in an errgroup : the redirect on port 8080, the admin API on port 8081.

func initializePublicRoutes(r *gin.Engine, services *services.Services) {
  r.GET("/health", handlers.GetHealth)
  r.NoRoute(handlers.Unshort(services)) // everything else is a short code
}

func initializeAdminRoutes(r *gin.Engine, services *services.Services) {
  r.GET("/health", handlers.GetHealth)
  r.POST("/shorten", handlers.PostShorten(services))
  r.POST("/bulk/shorten", handlers.PostBulkShorten(services))
}

Each port gets its own Kubernetes Service of type LoadBalancer, and GKE turns each one into a regional network load balancer, TCP passthrough : TLS ends in the pod. The redirect sits behind an external one, on a reserved static address with a DNS record on our short domain. The admin API sits behind an internal one, with a private address on the cluster’s subnet, reachable from our API gateway and from nothing else. The same pod serves both, and only the redirect is reachable from the internet.

resource "kubernetes_service" "admin-lb" {
  metadata {
    name      = "url-shortener-admin-api"
    namespace = "acme"
    annotations = {
      "cloud.google.com/load-balancer-type" = "Internal"
    }
  }
  spec {
    type                    = "LoadBalancer"
    load_balancer_ip        = google_compute_address.admin.address
    external_traffic_policy = "Local"
    selector                = { app = "url-shortener-api" }
    port {
      port        = 443
      target_port = 8081
    }
  }
}

The alternative was one listener and an authentication middleware on /shorten. I prefer a door that does not exist to a door with a good lock. The admin port has no authentication of its own, because no packet from outside the VPC can reach it ; the API gateway in front of it already authenticates the platform’s callers with a JWT, and it exposes /bulk/shorten to exactly one of them.

The infrastructure, as one module

Everything the service needs outside the cluster is one Terraform module, instantiated once per environment and applied by Cloud Build :

terraform/module/url-shortener/
├── storage.tf    # the bucket, regional, and its IAM bindings
├── account.tf    # a dedicated service account, its key
├── network.tf    # two static addresses (external, internal), two DNS records
├── kube.tf       # the two Services, the key as a Kubernetes secret
└── kms.tf        # a KMS key so Cloud Build can decrypt the app's secrets

The service account can read the objects of that one bucket and create new ones. It cannot overwrite or delete them : constraint 1 is enforced by IAM as well as by the code. Its key lands in the cluster as a secret the pod mounts. That is the part I would change first : a key file is a credential that exists, and Workload Identity would make it not exist. It is on the list.

resource "google_storage_bucket" "links" {
  name          = format("%s-url-shortener-links", var.project)
  project       = var.project
  location      = "europe-west1"
  storage_class = "REGIONAL"
}

resource "google_storage_bucket_iam_binding" "links_read" {
  bucket  = google_storage_bucket.links.name
  role    = "roles/storage.objectViewer"
  members = [format("serviceAccount:%s", google_service_account.shortener.email)]
}

resource "google_storage_bucket_iam_binding" "links_create" {
  bucket  = google_storage_bucket.links.name
  role    = "roles/storage.objectCreator"
  members = [format("serviceAccount:%s", google_service_account.shortener.email)]
}

The application ships like every other service on the platform : a Helm chart, a Jenkins pipeline that hands over to Cloud Build, a smoke test over HTTPS after each deploy, and a horizontal autoscaler between two and six pods. Nothing about the shortener is special on that side, which is the point.

The bug that came with it

Four days after the first deploy the pods were restarting on memory. A profile with pprof under a load generator showed it in a minute : the first version of Read built its own context with a timeout, copied the object into a buffer and closed the reader at the end. On every early return the reader stayed open, and with it a connection. Every bad request leaked one, and a scanner sends plenty of those.

// before : the reader is closed only on the happy path
buffer := new(bytes.Buffer)
buffer.ReadFrom(reader)
err = reader.Close()

// after : closed on every path
defer reader.Close()
url, err := ioutil.ReadAll(reader)

The writer got the same treatment : closed on the error path too. The fix is a handful of lines ; the lesson is to profile a service under load before you call it done, however small it is. It took an afternoon and it is the only incident the service has had.

Shortening in bulk

The alert function builds every link of a run before it sends a single email, so it asks for all of them at once. /bulk/shorten takes a list of URLs and answers with one status per URL, so that a single malformed link does not fail a batch of hundreds :

{
  "datas": [
    { "code": 200, "response": { "url": "https://acme.example/search?encodedSearch=eyJrZXl3b3Jkcy...", "shortCode": "TY8uMmQYp" } },
    { "code": 400, "response": { "url": "https://evil.example/login", "error": "domain 'evil.example' not allowed" } }
  ]
}

The function went live on the shortener on the fifth of May, twelve days after the first commit. Every link in the widened alert is now nine characters after the slash, and the recruiters who wrote to customer care are receiving their candidates again.

The numbers

3 weeks from first commit to the last production tuning, part time, alongside the rest of the platform.
0 databases, caches or queues. One bucket.
1 incident, a memory leak, fixed in an afternoon.
2 to 6 pods, 128 to 512 MiB each, which is more than it needs.

The shortener has been in production for two weeks. It has a bucket, two ports, and no one on call for it 😴

Update, 2026 : the shortener was still in production when I left the mission, more than two years after this post. Neither its code nor its infrastructure changed again : the only later commits renewed a TLS certificate and removed a monitoring agent from staging.