ThreadPoolExecutor vs Sequential: I/O Lab

Intro — what this post promises

Does concurrent.futures.ThreadPoolExecutor actually shrink wall time? This lab runs I/O-bound time.sleep tasks sequentially vs with a thread pool on Linux localhost, then contrasts a CPU-bound Python math loop — where the GIL keeps threads from helping.

Related links:

• perf_counter vs time localhost lab
• array vs list ints localhost lab
• logging vs print localhost lab
• deque vs list queue localhost lab
• lru_cache hit vs miss localhost lab
• itertools vs python loops localhost lab
• fork COW RSS vs spawn localhost lab
• Why your average latency graph is lying (p50 / p95 / p99)

Lab honesty (1 Oct 2026 IST): Python 3.13.5. Sleep stands in for waiting on sockets/disk (GIL released). Affiliates: 0. Not ProcessPoolExecutor / asyncio.

Verdict up front: 40× sleep(10 ms) sequential ~411 ms; 8 workers ~55 ms (~7.4×); 40 workers ~18 ms (~23×). Same shape of CPU work with 8 threads: ~0.90× (no win — GIL).

Arms

Lab topology

I/O: N in {20,40}, dt in {5ms,10ms}
CPU: 8 tasks of ~200k float ops each
Metric: p50 wall; speedup = seq_p50 / pool_p50

Script: lab-evidence/58-threadpool-vs-sequential/results/run_lab.py.

Lead table — I/O sleep(10 ms) × 40

Ideal sequential ≈ 400 ms; perfect 40-way overlap ≈ 10 ms. We landed ~18 ms — overhead + scheduling, still ~23× faster than one-at-a-time.

Also: 20× 10 ms with 20 workers ~12.8×; 40× 5 ms with 40 workers ~13.5×.

CPU contrast (GIL honesty)

Threads did not beat sequential here (~0.9×). Pure-Python bytecode holds the GIL; use processes (or native extensions that release the GIL) for CPU fan-out.

Why sleep is a fair I/O stand-in

time.sleep releases the GIL the same way many blocking syscalls do. On this box the sequential arm tracked ~N × dt (40 × 10 ms → ~411 ms). With max_workers=N wall time collapsed toward one sleep plus pool overhead (~18 ms). That is the story for fan-out over blocking HTTP clients, DB drivers, or disk reads — not for pure-Python number crunching.

Worker sizing tip from the tables: 8 workers already delivered ~7.4× on 40 tasks; going to 40 pushed ~23×. Past “enough concurrency to cover waiters,” gains flatten and you mostly pay thread stacks.

Pitfalls

1. Benchmarking CPU work with threads and concluding “threading is useless” — I/O tells a different story.
2. max_workers >> runnable waits — diminishing returns and memory for stacks.
3. Ignoring submit/join overhead on tiny sleeps — still large speedups here at 5–10 ms.
4. Assuming asyncio is the only answer — ThreadPool is fine for blocking SDKs.

When to pick what

Reproduce

python3 lab-evidence/58-threadpool-vs-sequential/results/run_lab.py

Evidence: /workspace/lab-evidence/58-threadpool-vs-sequential/results/.

Closing

Threads win when work waits. On this box 40× 10 ms sleeps went from ~411 ms sequential to ~55 ms at 8 workers (~7×) and ~18 ms at 40 (~23×). The same executor on CPU math was ~0.90× — measure whether your tasks release the GIL.