Running a multi-environment client platform on a single k3s box

Most teams reach for a managed Kubernetes control plane the moment they hear the word "Kubernetes," then provision a separate cluster per environment because that's the pattern everyone copies. For a small-to-mid client platform, that's a lot of money and moving parts to solve a problem you don't have yet.

This is a write-up of how we at DigiWorks run a real client platform — Nova Estate, a property/booking application — with fully isolated dev, staging, and production environments on a single lightweight k3s box. No managed control plane, no cluster-per-environment. It's not a hyperscale setup and I'm not going to pretend it is. It's a lean, correct one, and the engineering that makes it correct is worth walking through.

The application

Nova Estate is a fairly conventional three-tier web app:

• a NestJS + Prisma backend API,
• a React / Vite customer frontend,
• an admin dashboard.

Nothing exotic. The interesting part isn't the app — it's how we give it three genuinely isolated environments without triplicating infrastructure cost.

Multi-environment isolation via namespaces

The core decision: instead of a cluster per environment, we use one k3s cluster and lean on Kubernetes namespaces as the isolation boundary. Each environment gets its own namespace, its own deployments, its own ingress hostnames, and — critically — its own database. They share a cluster; they do not share state.

NAMESPACE          WORKLOADS
────────────────   ─────────────────────────────────────────────
nova-estate-dev    backend · frontend · admin   (dev database)
nova-estate-stg    backend · frontend · admin   (staging database)
nova-estate-prod   backend · frontend · admin   (production database)

shared-prod        postgres · mysql · redis · meilisearch
kube-system        traefik (ingress)
cert-manager       cert-manager (Let's Encrypt via DNS-01)

Namespaces are a real boundary in k3s: RBAC, resource quotas, network policy, and secret scoping all key off them. A misconfigured deployment in nova-estate-dev can't read secrets or reach services in nova-estate-prod unless we explicitly wire it to. The environments are named identically per tier, so promoting a change is mechanical: the same Helm chart, a different values file, a different namespace.

I want to be honest about what this doesn't give you. Namespaces are a soft, logical boundary sharing one kernel and one node. They are not the hard, physical isolation of separate clusters. For this class of platform that's an acceptable trade; for a regulated multi-tenant system where prod must be physically walled off from everything else, it wouldn't be.

A shared-services layer, consumed over cluster DNS

Databases and stateful backing services are the expensive part to run three times over. So instead of a Postgres per environment as separate managed instances, we run the stateful services once, in a shared-prod namespace, and give each environment its own logical database inside them:

• PostgreSQL — primary datastore for the app,
• MySQL — for services that expect it,
• Redis — cache, sessions, and queues,
• Meilisearch — search index.

Applications reach these across namespace boundaries using Kubernetes' built-in service DNS. Every service is addressable at <service>.<namespace>.svc.cluster.local, so an app in nova-estate-stg connects to shared Postgres by full DNS name and its own database — never the prod one:

# staging backend env — points at the SHARED services,
# but its own logical database and Redis db-index
DATABASE_URL=postgresql://nova_stg:[email protected]:5432/nova_estate_stg
REDIS_URL=redis://:[email protected]:6379/1
MEILI_HOST=http://meilisearch.shared-prod.svc.cluster.local:7700

The isolation here is a discipline, not an accident: separate database names per environment, separate Redis logical DB indexes, separate credentials. The shared layer saves us from running (and paying memory for) three Postgres instances, three Redis instances, three search engines — while each environment still behaves as if it owns its data.

The trade-off is explicit: shared-prod becomes a blast-radius concentrator. If shared Postgres goes down, every environment loses its database at once. We accept that because the alternative — triplicating stateful services on a single box — costs more RAM than the box has to give, and the failure domain is already the box itself (more on that below).

Secrets that are safe to commit

Everything about this platform lives in git, including the Kubernetes manifests. That immediately raises the question every GitOps setup has to answer: what do you do with secrets? Plain Kubernetes Secret objects are just base64 — worthless to commit.

We use Sealed Secrets. The controller in the cluster holds a private key; we encrypt secrets against its public key with kubeseal, producing a SealedSecret resource that is safe to commit because only the in-cluster controller can decrypt it. The encrypted blob goes in the repo alongside the chart; the plaintext never does.

# create a secret, encrypt it, commit the sealed output
kubectl create secret generic nova-secrets \
  --from-literal=DATABASE_URL="$STG_DB_URL" \
  --dry-run=client -o yaml \
| kubeseal --format yaml \
  --controller-namespace kube-system \
> nova-estate-stg/sealed-secrets.yaml   # safe to git commit

Each environment's namespace gets its own sealed secret. Because a SealedSecret is scoped to its namespace and name by default, the staging blob won't decrypt in the prod namespace even if someone copied it — a nice property that reinforces the namespace boundary rather than fighting it.

Ingress and automatic TLS

Traefik ships with k3s and is our ingress controller. Each environment declares its own hostnames via Ingress objects, and Traefik routes traffic to the right namespace's services.

TLS is handled by cert-manager issuing Let's Encrypt certificates automatically. We use the DNS-01 challenge via Cloudflare rather than HTTP-01, which means we can issue and renew certs — including wildcards — without exposing a challenge endpoint, and it works cleanly for the non-prod hostnames too. Once the Certificate and Ingress are declared, renewals are hands-off; cert-manager watches expiry and re-issues.

CI: build in-cluster, deploy with Helm

The app is packaged as a Helm chart — one chart, three values files, one per environment. That's what makes "same code, different environment" a config change rather than a copy-paste job.

Builds run through Jenkins, which lives on our build host. Images are built with Kaniko — no Docker daemon required, which matters when your builder is itself a pod — and pushed to a private in-cluster Docker registry. The cluster pulls from that registry; images never leave the setup. The end-to-end flow:

git push
   │
   ▼
Jenkins pipeline
   │   build image with Kaniko  (no docker daemon)
   ▼
private in-cluster registry   (push :sha tag)
   │
   ▼
helm upgrade --install nova-estate ./chart \
     -f values-.yaml \
     -n nova-estate-
   │
   ▼
Traefik routes the new pods  →  cert-manager serves valid TLS

A push builds an image, tags it, pushes it, and Helm upgrades the target environment in place. Promotion from staging to production is running the same chart against values-prod.yaml and the prod namespace with a vetted image tag — no bespoke prod deploy path that drifts from what we tested.

The honest trade-offs of going lean

This setup is deliberately cost-efficient, and cost-efficiency is bought with real compromises. If someone pitches you single-box multi-env without naming these, be suspicious.

• Single point of failure. One box runs everything. If the node goes down, all three environments go down with it. There is no cross-node failover because there is no second node. For this tier of platform we treat the box as the failure domain and plan backups and restore-time accordingly — but you should know that going in.
• Discipline is load-bearing. Namespace isolation and the shared-services split only hold if we're rigorous: resource requests and limits on every workload so a runaway dev pod can't starve prod, distinct credentials and database names per environment, and network policy where it matters. The correctness is operational, not automatic.
• Shared services concentrate blast radius. Running Postgres/Redis/Meilisearch once saves a lot of RAM, but a shared-layer outage hits every environment. That's the price of not triplicating stateful services on a box that couldn't hold them anyway.
• It's sized for small-to-mid platforms, not hyperscale. This is the right architecture for a client booking platform, an internal tool, an early-stage product. It is emphatically not the answer for something that needs multi-node HA, physical prod isolation, or elastic scale-out. When a client outgrows it, the Helm-and-namespaces model ports to a bigger cluster cleanly — which is part of why we build it this way from the start.

What you get in exchange is substantial: no managed-control-plane bill, no per-environment cluster multiplying your infra spend, a single small box to reason about, and a deployment path that's identical across environments. For the platforms we run, that's the correct trade — and being candid about the edges is what makes it a defensible one rather than a marketing claim.

Takeaway

You don't need a managed Kubernetes control plane or a cluster per environment to run isolated dev/staging/prod correctly. Namespaces for isolation, a shared-services layer addressed over cluster DNS, sealed secrets in git, automatic TLS from cert-manager, and a Kaniko-plus-Helm pipeline give you a lean platform that's honest about its single-box failure domain and disciplined enough to be safe within it. For small-to-mid client platforms, that combination is hard to beat on cost-to-correctness.