mmap Write vs pwrite Region: Localhost Lab

Intro — what this post promises

Update a fixed-size file region (1 MiB at offset 2 MiB in a 16 MiB file): mmap MAP_SHARED write + msync/flush vs os.pwrite / write+seek with clearly labeled fsync/fdatasync variants. This lab reports MB/s on Linux localhost.

It is not mmap-vs-read-scan (lab 81) and not a whole-file fsync-vs-fdatasync redo (lab 32). Focus: write path — mmap shared mapping vs syscall write to a slice.

Related links:

• mmap vs read scan localhost lab
• fsync vs fdatasync localhost lab
• mmap vs read odirect localhost lab
• stat vs path stat localhost lab
• configparser vs json localhost lab
• html escape vs manual localhost lab
• queue vs deque handoff localhost lab
• stringio vs list join localhost lab

Lab honesty (1 Oct 2026 IST): Python 3.13.5. Affiliates: 0. No Docker. Each one-shot arm opens/closes the fd (harsh but honest).

Verdict up front (1 MiB slice): pwrite only ~7821.2 MB/s (page cache); pwrite+fdatasync ~824.3; mmap+msync(MS_SYNC) ~536.0; mmap+MS_ASYNC ~2005.6; mmap no sync ~1781.2.

Arms

Lab topology

file 16 MiB · slice 1 MiB @ offset 2 MiB · 7 rounds · p50
metric: MB/s = slice_bytes / p50_s / 1024²
Python mmap.flush = msync sync; libc msync for ASYNC/SYNC

Script: lab-evidence/107-mmap-write-vs-write/results/run_lab.py.

Lead table — one-shot update (p50 MB/s)

Cached pwrite looks “impossibly” fast — it has not waited for disk. Compare durable arms to durable arms.

Reuse check (8 updates, aggregate MB/s)

Keeping a mapping hot helps async mmap. Durable pwrite still pays fdatasync each iteration.

Why mmap setup shows up

Each one-shot mmap arm pays open + mmap + munmap + close. In servers that keep a map for the process lifetime, that tax amortizes — see the reuse×8 async arm. Microbenches that only show undurable memcpy into the map overstate steady durable throughput.

Reading it

• Need durability? Compare pwrite+fdatasync vs mmap+MS_SYNC — not vs pwrite alone.
• MS_ASYNC is not a durability guarantee; it only kicks writeback.
• mmap shines for many in-place edits on a long-lived mapping; one-shot open/map/close taxes setup.
• Do not confuse with read/scan mmap labs (81 / 21).

Durability cheat sheet

Never cite undurable MB/s next to durable MB/s without a label — that is how blog charts mislead.

vs read-scan labs

Labs 81/21 measure read/scan bandwidth and page-touch behavior. This post only moves a dirty slice toward storage. Mixing those headlines in one article cannibalizes both topics; keep write-path numbers here.

Pitfalls

• Publishing undurable pwrite_only numbers as “disk throughput.”
• Forgetting page alignment when calling libc msync.
• Rehashing lab 32’s whole-file fsync story without a region focus.
• Assuming MAP_SHARED assign is visible to other processes before sync (visibility vs durability differ).

Reproduce

python3 lab-evidence/107-mmap-write-vs-write/results/run_lab.py

Evidence: summary.json, summary.txt, region.dat.

Limits

One Linux box / local FS. 1 MiB slice only. Open/close per one-shot arm. Not O_DIRECT. Not multi-writer coherency stress.

Takeaway

For a 1 MiB region, undurable pwrite ~7821.2 MB/s led; durable pwrite+fdatasync ~824.3 beat one-shot mmap+MS_SYNC ~536.0 MB/s. Match durability labels before picking a winner.