Nginx Gzip On vs Off: Localhost Wire Size and RPS Lab

Intro — what this post promises

gzip on; looks free until you measure it on a fast path. On a slow WAN, smaller wire bytes win. On localhost (and many LAN/VPC hops), the CPU that builds those bytes can dominate client RPS.

This is a hands-on lab with measured numbers:

1. Three nginx masters: gzip off, gzip_comp_level 1, gzip_comp_level 6.
2. Wire size and Content-Encoding for HTML, JSON, and random binary.
3. Flood RPS / p50 latency with Accept-Encoding: gzip.
4. When level 1 beat level 6 on size for our JSON (yes, really).
5. Why application/octet-stream never got a Content-Encoding.

Related links:

• Nginx limit_req rate limit lab
• HTTP Keep-Alive vs Connection: close lab
• Why your average latency graph is lying (p50 / p95 / p99)
• How to read server monitoring graphs

Lab honesty (30 Sep 2026 IST): Shared Linux lab box (8 cores, kernel 6.12). nginx/1.26.3 host binary. Three masters on 127.0.0.1:18120–18122 only. Payloads: page.html 159 279 B, data.json 443 791 B, noise.bin 524 288 B random. Client: Python 3.13.5 urllib thread floods, Connection: close. No Docker. No public bind. Affiliates: 0.

Verdict up front: repetitive HTML/JSON compressed ~107× / ~38× at level 1, but JSON floods dropped from ~1987 RPS (off) to ~1031 (l1) and ~563 (l6) on loopback. Gzip is a bandwidth vs CPU trade — measure both.

What nginx gzip changes

With gzip on and a matching gzip_types / default HTML behavior, nginx may compress the response body when the client sends Accept-Encoding: gzip. gzip_comp_level picks the zlib effort. gzip_min_length skips tiny bodies. Types you omit stay uncompressed even if the client asks for gzip.

Related links:

• Module ngx_http_gzip_module

Lab topology

:18120  gzip off
:18121  gzip on  level 1
:18122  gzip on  level 6
GET /page.html  /data.json  /noise.bin  (+ tiny /index.html)
Probe headers + 200-request floods (20 workers) with Accept-Encoding: gzip

Arm A — wire size

noise.bin is application/octet-stream — not in our gzip_types, so no Content-Encoding even with Accept-Encoding: gzip. That is the correct MIME lesson, not a bug.

Odd but measured: for this JSON, zlib level 1 produced a smaller body than level 6 (11 584 vs 12 222). We reproduced the same ordering with Python gzip.compress. Higher gzip_comp_level is not a guaranteed size win — and it cost a lot of CPU below.

Arm B — flood RPS on loopback

On loopback there is almost no WAN to save. Compressing a ~434 KiB JSON cut RPS roughly 2× at level 1 and 3.5× at level 6. The tiny HTML payload at level 1 was the one case where RPS ticked up slightly — wire shrank enough that transfer+parse won over compress CPU.

Clients that sent Accept-Encoding: identity against the gzip-on masters still got uncompressed bodies (and we saw Vary: Accept-Encoding on the gzip arms).

When gzip still wins the design

This lab is hostile to gzip on purpose: localhost, Connection: close, CPU on the same box as the client. On real edges — mobile, cross-region, expensive egress — a 38× JSON shrink dominates. Prefer:

• gzip_comp_level 1–3 for dynamic JSON/HTML on fast links.
• Precompressed gzip_static for cold assets when you can.
• Omit incompressible types; do not burn CPU on noise.
• Pair with keep-alive (separate lab) so you are not also paying handshake tax.

Related links:

• HTTP Keep-Alive vs Connection: close lab

Pitfalls we hit (or avoided)

1. Assuming higher gzip_comp_level ⇒ smaller — our JSON said otherwise; always weigh size and RPS.
2. Expecting binary to compress — only listed gzip_types (plus default HTML) participate.
3. Judging gzip on loopback alone — RPS can fall while WAN users still win.
4. Forgetting Vary: Accept-Encoding — caches must key on it.
5. Tiny files — gzip_min_length correctly skipped our 3-byte index.

Practical checklist

• Turn gzip on for text/JSON/HTML; leave random/binary types out.
• Start at gzip_comp_level 1 for dynamic responses; raise only after measuring size and CPU/RPS.
• Confirm Content-Encoding: gzip with a real Accept-Encoding probe — not just a green config test.
• Watch p95/CPU under flood; loopback RPS cliffs predict origin CPU cost.
• Do not confuse this with Brotli/gzip_static — different knobs, not measured here.

Verdict

Gzip crushed wire size for repetitive HTML/JSON (~107× / ~38× at level 1) but halved to thirded JSON client RPS on localhost versus gzip off. Level 6 was worse on both size (for our JSON) and RPS than level 1. Use gzip where bandwidth matters; keep the level modest on dynamic origins; measure MIME coverage so you are not compressing — or failing to compress — by accident.

Evidence path on the lab box: lab-evidence/20-nginx-gzip/results/. Affiliates: 0.