nice / ionice Lab: CPU Nice 19 Slowdown and Idle-Class Disk

Intro — what this post promises

nice and ionice are the knobs people reach for when a batch job should “get out of the way.” Folklore says nice 19 makes you harmless and idle-class IO never steals the disk. This lab measures both.

This is a hands-on lab with measured numbers:

1. Fixed-work CPU victim under 6 busy-loop hogs at different nice values.
2. Proof that nice -5 did nothing here without CAP_SYS_NICE.
3. Why warm page-cache scans hide ionice.
4. O_DIRECT reads/writes under competing best-effort writers: idle vs BE0/BE7.
5. Tails: p95 under nice 19 is where the pain lives.

Related links:

• ProcessPool vs ThreadPool GIL localhost lab
• Why your average latency graph is lying (p50 / p95 / p99)
• mmap vs read (+ O_DIRECT) localhost lab
• epoll vs select FD_SETSIZE localhost lab

Lab honesty (1 Oct 2026 IST): Shared Linux lab box (8 cores, kernel 6.12). Python 3.13.5 + nice / ionice / dd iflag=direct. No Docker. No GPU. No API keys. drop_caches denied (no root). Affiliates: 0.

Verdict up front: victim nice 19 under 6 nice-0 hogs was 5.7× slower p50 (398 ms → 2256 ms) with p95 ~7.4 s. Hogs at nice 19 left the nice-0 victim near baseline (1.07×). Warm cached reads hid ionice; O_DIRECT under 2 BE0 writers made idle-class reads 6.1× slower vs 2.2× for BE0.

What nice and ionice change

Related links:

• nice(1)
• ionice(1)

Lab topology

CPU: 6× busy-loop hog processes at nice N; victim runs fixed 3e6-iter Python loop via nice -n V
IO: 64 MiB file; dd iflag=direct / oflag=direct under ionice; 2 competing BE0 direct writers
Control: warm buffered reads (shows why ionice looked like a no-op on page cache)

Probes: nice -n0/10/19 → reported priority 0/10/19. nice -n-5 → still 0 (no capability). ionice -c3 → idle; -c2 -n0/7 → best-effort: prio 0/7.

Arm A — CPU victim vs nice hogs

Fixed work, n=25 timed runs. Baseline (no hogs, nice 0): p50 ≈ 398 ms, p95 ≈ 627 ms.

Background the batch job (nice 19 on the hogs) and the interactive victim barely noticed. Nice the victim to 19 under aggressive hogs and you pay ~5.7× at p50 — and ~7.4 s p95 on the same fixed work.

Negative nice arms were attempted; without CAP_SYS_NICE the process stayed at priority 0. Do not cite those as “higher priority wins.”

Related links:

• Why your average latency graph is lying (p50 / p95 / p99)

Arm B — warm cache: ionice looks fake

128 MiB sequential read already in page cache, 4 competing BE readers:

Idle ≈ solo. That is not “ionice broken” — it is memory bandwidth, not disk elevator. drop_caches was unavailable without root, so we moved to O_DIRECT.

Related links:

• mmap vs read (+ O_DIRECT) localhost lab

Arm C — O_DIRECT under competing writers

64 MiB dd iflag=direct while 2 BE0 oflag=direct writers hammer separate files.

Reads

Writes (64 MiB oflag=direct + fsync-ish conv=fsync)

Idle-class did what it says on the tin once IO actually hit the device: the victim yielded hard. Best-effort stayed in the fight (~2× read slowdown vs ~6× idle).

How to read these numbers

• nice 19 on the batch job protects latency-sensitive work better than hoping the interactive task “wins” by default under equal nice.
• nice 19 on the latency path under load is a self-inflicted p95 tax.
• ionice idle needs real block IO (direct/cold) to show up; page cache lies.
• BE prio 0 vs 7 was close here under this writer pair; idle vs BE was the cliff.

Pitfalls we hit (or avoided)

1. Judging ionice on warm buffered reads — looked like a no-op until O_DIRECT.
2. Assuming nice -5 works for unprivileged users — probe stayed 0.
3. Looking only at p50 — nice19 victim p95 was multi-second.
4. Nice-ing the wrong process — background the batch hogs, not the request path.
5. Claiming elevator theory for a specific IO scheduler — we report class behavior on this box, not a BFQ paper.

Practical checklist

• Batch / backup / scrape: nice -n19 (and consider ionice -c3 for disk) on the batch side.
• Latency path: keep nice near 0; measure p95 under contention.
• Validate ionice with direct or cold IO, not only cached cat.
• Confirm with nice / ionice probes (no silent privilege fail).
• Keep evidence next to any “we niced it” capacity claim.

Verdict

On this box, a nice-0 victim under 6 nice-0 hogs was fine (1.1×); the same victim at nice 19 paid 5.7× p50 and ~7.4 s p95. Nicing the hogs to 19 protected the victim (1.07×). Warm cache hid ionice; under O_DIRECT writers, idle-class reads were 6.1× slower vs 2.2× for best-effort. Priority knobs work — aim them at the batch job, and measure the device path.

Evidence path on the lab box: lab-evidence/31-nice-ionice/results/. Affiliates: 0.