fsync vs fdatasync vs none: Durable Write Cost Lab

Intro — what this post promises

Calling write() is not the same as durable data. The kernel is happy to park your bytes in the page cache and return. Crash (or power loss) later and those “writes” never happened.

This is a hands-on lab with measured numbers:

1. Wall time for batches of small (256 B) and medium (4 KiB) records with no sync.
2. Same batches with fdatasync per write and fsync per write.
3. Amortization: sync every 10 / every 50 writes.
4. Optional O_SYNC open — durability on every write(2).
5. Honest limits on an overlay/virtio lab box (not a battery-backed WAL appliance).

Related links:

• mmap vs read / O_DIRECT localhost lab
• sendfile vs userspace copy localhost lab
• pipe vs tmpfile IPC localhost lab
• nice / ionice CPU and disk priority lab
• Why your average latency graph is lying (p50 / p95 / p99)

Lab honesty (1 Oct 2026 IST): Shared Linux lab box (8 cores, kernel 6.12). Python 3.13.5 writing under /tmp on overlay atop virtio. No Docker. No public bind. Affiliates: 0. This measures sync cost, not crash recovery or fsck after echo b > /proc/sysrq-trigger.

Verdict up front: none → any per-write sync was the cliff (~100–200×). On this overlay, fsync ≈ fdatasync (within noise). Syncing every 50 medium records bought back hundreds of MB/s.

What fsync / fdatasync / O_SYNC actually change

Textbook claim: fdatasync is cheaper than fsync because it can skip some metadata. On this lab FS the gap was tiny — report that honestly instead of pasting a textbook 2×.

Related links:

• man 2 fsync
• mmap vs read / O_DIRECT localhost lab

Lab topology

Python 3.13 · /tmp on overlay/virtio
Arms: none | fdatasync every N | fsync every N | O_SYNC open
Shapes: 256 B × 2000 records · 4 KiB × 500 records
Metric: batch wall p50/p95 → per-record µs, RPS, MB/s
Warmup batches discarded; repeats 4–8 depending on arm

Script: lab-evidence/32-fsync-vs-fdatasync/results/run_lab.py. Each arm creates/truncates a file, writes N fixed payloads, applies the sync policy, closes, records perf_counter wall time for the whole batch.

RPS / MB/s below are from p50 batch wall (ops and bytes ÷ p50 seconds) so a slow repeat does not silently dilute the headline.

Arm A — small records (256 B × 2000)

none → fdatasync/1 ≈ 198× on batch p50. That is the durability tax in one table.

fsync/1 was not slower than fdatasync/1 here (slightly faster on this run). Treat them as tied on overlay, not as a portable “always use fdatasync” win.

Stability×3 for none p50 (ms): 1.18 / 1.26 / 1.18. For fdatasync/1: 246 / 234 / 233.

Related links:

• Why your average latency graph is lying (p50 / p95 / p99)

Arm B — medium records (4 KiB × 500)

Medium none looks like memory copy (multi‑GB/s class) because it mostly dirties cache. Per-write sync drops you into the ~20–26 MB/s band on this box — still faster than the 256 B sync arms in MB/s (larger payload per barrier) but nowhere near “disk sequential” marketing numbers.

Stability×3 for med fsync/1 p50 (ms): 85.0 / 68.6 / 77.5 (noisy — quote bands, not fake precision).

Arm C — amortize: every 10 / every 50

Same 4 KiB × 500 shape; sync only every N records (plus a final sync if the last group is short).

Group commit is not a metaphor: every-50 recovered into the hundreds of MB/s while still issuing durability barriers. Databases and queues do this for a reason.

Related links:

• pipe vs tmpfile IPC localhost lab
• flock contention exclusive vs shared lab

How to read these numbers

• none ≠ durable. Fast path is page cache. Fine for scratch; wrong for “acked to disk.”
• Per-write sync turns throughput into barrier rate. Small records hurt more in MB/s.
• fsync vs fdatasync: textbook gap did not show on this overlay run — do not ship a fake 3× claim from this post.
• O_SYNC: per-record cost matched sync/1 (~120–180 µs). Convenient API; same economic cliff.
• Every-N is the practical dial between RPO fantasy and RPS reality.

Pitfalls we hit (or avoided)

1. Reporting sum-of-repeats RPS — early harness bug understated RPS by ×repeats; numbers above use p50 batch wall.
2. Assuming fdatasync always wins — not on this FS snapshot.
3. Calling none “disk throughput” — it is not; see mmap / O_DIRECT labs for device-ish paths.
4. One p50 without p95 — sync arms were tighter relative to mean; still show both.
5. Claiming crash safety — we timed syscalls; we did not pull the plug.

Practical checklist

• Decide the durability contract (lost last N records OK? or not?).
• If you need sync, batch (every N / group commit / fdatasync on a WAL segment) before blaming the SSD.
• Benchmark your FS — overlay ≠ XFS on NVMe ≠ cloud block with flaky flush.
• Pair with p95 and a clear record size; do not quote only MB/s from none.
• Keep evidence next to any “we fsync every message” capacity plan.

Related links:

• stdout buffering line vs full localhost lab
• context-switch pipe ping-pong localhost lab
• sendfile vs userspace copy localhost lab

Verdict

On this box, buffered none wrote 256 B records at 373 MB/s~~ (0.65 µs/rec). fdatasync or fsync per write fell to 2 MB/s (~~120–130 µs/rec) — roughly two orders of magnitude. Medium 4 KiB sync/1 sat near ~21–26 MB/s; syncing every 50 climbed back to ~343–373 MB/s. fsync and fdatasync were peers here; the real product decision is how often you pay the barrier, not which synonym you type.

Evidence path on the lab box: lab-evidence/32-fsync-vs-fdatasync/results/. Affiliates: 0.