Go GC + Swap on a Tiny VPS: Stop Thrashing Before the OOM Killer

Intro — what this post promises

If you run a Go service on a 512 MB or 1 GB VPS (or a tight container), you have probably seen this movie: CPU looks busy, RSS creeps toward the limit, the process never quite dies — and every request suddenly takes hundreds of milliseconds. Operators blame “GC pauses.” Often the real villain is swap thrashing feeding a GC that keeps trying to reclaim a heap that no longer fits in RAM.

This post is a practitioner playbook for that failure mode. You will get:

1. How Go’s GOGC and soft GOMEMLIMIT interact with Linux memory and swap.
2. Symptoms that distinguish thrashing from a clean OOM kill.
3. What to measure (cgroup, swap, GC, latency) before you touch knobs.
4. A concrete checklist you can run on your own box.

Lab honesty: Critical RSS / cgroup-swap / p99 / GC-cycle tables below are from a ShopperCove lab API (tiny retain/churn HTTP service) under cgroup v2 limits — not a production SaaS — run 29 Sep 2026 (IST) on this Linux box: Go 1.24.4, cgroup 512 MiB (and a 384 MiB thrash pair), hey load, GODEBUG=gctrace=1. Host-level vmstat si/so and Grafana screenshots were not captured; cgroup swap counters stand in for swap pressure. This was not a dedicated 512 MB VPS — same failure mode, different neighbors/disk. Optional follow-ups: Grafana panels, host vmstat, automemlimit compare, and a real-service alloc hotspot.

Related ShopperCove reading (memory theme, different stack): Spring Boot vs Quarkus on a 512 MB VPS. That post is about JVM framework footprint; this one is about Go GC + swap behavior. They complement each other — do not merge them into one “everything memory” mega-post.

Why a “healthy” Go process can still be dying slowly

The soft memory limit is not a hard cgroup ceiling

Since Go 1.19, the runtime accepts a soft memory limit via the GOMEMLIMIT environment variable or runtime/debug.SetMemoryLimit. Official docs define the limit over Go-managed memory roughly as Sys − HeapReleased (or the equivalent runtime/metrics classes), and explicitly exclude things like the mapped binary, memory managed in other languages, and some OS-held memory. See:

• A Guide to the Go Garbage Collector
• runtime package env docs (GOMEMLIMIT / GOGC)

Important consequences on a tiny VPS:

• The limit is soft. Under pressure the GC will try harder, but Go will still allocate past the limit rather than thrash forever at 100% GC CPU (there is a ~50% GC CPU safeguard described in the GC guide).
• Non-Go RSS still counts against your cgroup / VPS RAM. Leaving 5–10% headroom (official rule of thumb in the GC guide for containerised services) is not optional folklore — it is how you avoid “GOMEMLIMIT says fine, kernel says OOM.”

GOGC alone does not know your box size

GOGC sets a ratio of new heap to live heap. Double GOGC ≈ double heap overhead and roughly half GC CPU in steady state (see the GC guide’s math). That is great when RAM is plentiful. On a 512 MB box, a transient live-heap spike + default GOGC=100 can push total footprint into swap — or into the OOM killer — even though “GOGC is fine for throughput.”

GOMEMLIMIT exists precisely because GOGC alone cannot encode “this container only has 512 MiB.”

Swap turns OOM into a latency incident

Without swap, the kernel’s OOM killer is brutal but obvious: process gone, restart, page the on-call.

With swap enabled (common on cheap VPS images), the kernel pages cold anonymous memory to disk. Your Go process stays in ps. Health checks may still pass. Meanwhile:

• Every allocation / GC scan that touches swapped pages pays disk latency.
• GC runs more often as the soft limit or heap target fights reality.
• p99 latency explodes while average CPU looks “utilized.”

That is thrashing: the system is busy moving pages, not serving users.

Symptoms checklist — thrashing vs OOM vs “just busy”

Use this as a triage card. Lab captured cgroup memory.events / swap counters and hey p99. Optional next lab: Grafana screenshots of each host signal (enrichment, not a blocker).

Prefer a clean OOM over silent thrashing when the alternative is serving garbage latency for hours. Many production setups intentionally set vm.swappiness=1 or disable swap on app nodes for exactly this reason — with the trade-off that you must size RAM and GOMEMLIMIT correctly.

What to measure before changing knobs

Host / cgroup layer

# Memory pressure (Linux)
free -h
vmstat 1
# cgroup v2 (path varies; inside container often /sys/fs/cgroup)
cat /sys/fs/cgroup/memory.current
cat /sys/fs/cgroup/memory.max
cat /sys/fs/cgroup/memory.events

Also record whether swap is enabled (swapon --show) and swappiness (sysctl vm.swappiness).

Lab box (measured 29 Sep 2026 IST):

Limitation (honest): host vmstat si/so was not the primary signal on this shared box — used cgroup memory.swap.current / .peak instead. Capturing a clean host vmstat strip is an optional next lab.

Go runtime layer

Useful starting points from the GC guide diagnostics section:

• CPU profile: watch cumulative time in runtime.gcBgMarkWorker, runtime.mallocgc, runtime.gcAssistAlloc.
• GODEBUG=gctrace=1 for per-cycle lines (stderr).
• runtime/metrics or expvar//debug/vars if you expose them — especially total memory classes vs heap released.

# Example only — adjust binary/service unit
GOMEMLIMIT=400MiB GOGC=100 ./my-service
# or: GODEBUG=gctrace=1

Measured headroom sketch for a 512 MiB cgroup (this lab):

Do not set GOMEMLIMIT below a live heap you cannot shrink — phases B/C retained 420 MiB with limit 400 MiB and burned ~1000 GC cycles (~6% GC CPU) with /json p99 11–14 ms.

Application latency layer

Pair memory graphs with percentiles, not only averages. (See the companion draft: Why Your Average Latency Graph Is Lying.) If p50 is fine and p99 is catastrophic while si/so are hot, you are looking at thrash, not “the GC is slow in general.”

Internal links:

• https://www.shoppercove.com/blog/read-server-monitoring-graphs-2
• https://www.shoppercove.com/blog/spring-boot-vs-quarkus-512-mb-vps

The playbook — stop thrashing before the OOM killer

Step 1 — Decide: thrash-prone or fail-fast?

Lab policy (chosen): Prefer fail-fast for the Go app cgroup — set memory.swap.max=0 once GOMEMLIMIT and live set fit. Host may keep swap for the machine; the service slice should not thrash. Evidence: phase E (swap allowed) stayed alive with ~277 MiB cgroup swap and 398 ms churn p99; phase A filled swap then OOM'd — both worse UX than a clean restart after correct sizing (phase D).

Step 2 — Set GOMEMLIMIT with headroom

From the official GC guide suggested uses:

• Do set a memory limit when the Go program owns a reservation (container / dedicated VPS slice).
• Leave ~5–10% headroom for memory Go does not track.
• Don’t set a limit merely to “avoid OOM” when the process is already near the ceiling — that can trade OOM for severe slowdown (thrashing mitigation has limits).
• Don’t bake a low limit into a CLI that shares a laptop with unknown inputs.

Libraries such as KimMachineGun/automemlimit can derive a limit from cgroup; this lab used a manual GOMEMLIMIT=400MiB / 300MiB env. Optional next lab: compare automemlimit vs manual env (enrichment, not a blocker).

Step 3 — Revisit GOGC only after the limit is sane

Pattern that often works for services:

1. Set GOMEMLIMIT to a soft cap under the cgroup.
2. Keep or raise GOGC so the average case is economical; let the limit clip spikes.
3. Only lower GOGC if you are OOMing without thrashing and need a smaller peak heap sooner.

Setting GOGC=off with a memory limit maximizes economy if live heap reliably fits. If live heap routinely exceeds the limit, you get constant GC and a stalled service — worse than OOM. The GC guide calls this out explicitly.

Step 4 — Reduce allocation rate (the real long-term fix)

Knobs buy time. Allocation rate still drives GC frequency. Use heap profiles (inuse_space / alloc_space) and escape analysis (go build -gcflags=-m=3) as described in the GC guide's optimization sections. This lab’s API is a retain/churn toy under cgroup limits — not a production SaaS. Optional next lab: before/after alloc hotspot on a real service (not claimed here; optional).

Step 5 — Alert on the right symptoms

Minimum alert set for tiny VPS Go services:

• Swap-in rate > 0 sustained for N minutes
• cgroup memory.events oom / oom_kill
• GC CPU fraction or assist time above budget
• Latency p99 (not average) SLO burn
• Restart count / OOM kills

Honest advantages and disadvantages

No testimonials. No “we cut latency 70% overnight” claims without a lab.

Worked example (measured lab — 29 Sep 2026 IST)

Scenario: cgroup v2 slice on lab box; ShopperCove lab API (tiny HTTP JSON retain/churn service — not a production SaaS); Go 1.24.4; GOMAXPROCS=2; GOGC=100; load with hey.

Also measured: Phase F (GOMEMLIMIT=300MiB on 384 MiB cgroup, live 320 MiB) avoided swap balloon but burned 1436 GC cycles (~11% GC CPU) with /churn p99 308 ms — wrong fix if live heap does not fit.

Optional next labs (not blockers): Grafana screenshots; host vmstat si/so strip; automemlimit vs manual env; supersede any CMS Swap/Go GC stub carefully at paste time.

External citations

• https://go.dev/doc/gc-guide
• https://pkg.go.dev/runtime
• https://pkg.go.dev/runtime/debug#SetMemoryLimit
• https://github.com/KimMachineGun/automemlimit

FAQ

Q1. Is GOMEMLIMIT a substitute for a container memory limit?
No. Set the cgroup/VPS limit as the hard ceiling; use GOMEMLIMIT so the Go GC cooperates inside that ceiling.

Q2. Should I always disable swap on app VPSes?
Often yes for latency-sensitive services, if you size RAM and limits correctly and alert on OOM. Keep swap only if you consciously accept thrash risk and monitor si/so.

Q3. Can GOMEMLIMIT alone cause thrashing?
If the live heap cannot fit under the limit, GC runs constantly and progress stalls. The runtime softens the limit to avoid infinite thrash, but a mis-set limit still hurts. Prefer raising RAM or cutting live set.

Q4. Where do I put GOMEMLIMIT — systemd, Docker, or code?
Any of the three. Env on the unit/container is simplest to audit. SetMemoryLimit helps when limits change at runtime (e.g. cgo spike). Document one source of truth.

Q5. How is this different from JVM -Xmx on 512 MB?
Different runtime, same operational lesson: the heap knob is not the whole process. See the JVM heap vs RSS draft and the existing Boot vs Quarkus post.

Q6. Will raising GOMAXPROCS fix thrashing?
No. More Ps can increase parallelism and allocator pressure. Cap GOMAXPROCS to your CPU quota separately.

CTAs

• Subscribe via RSS: https://www.shoppercove.com/feed.xml
• About the author / method: https://www.shoppercove.com/about