Unix Domain Socket vs TCP Localhost: Latency and Throughput Lab

Intro — what this post promises

Same-host services often talk over 127.0.0.1:PORT out of habit. A Unix domain socket (AF_UNIX) skips the TCP/IP stack for that hop. This post measures both on one box.

This is a localhost lab with measured numbers:

1. A fair echo comparison: TCP loopback vs Unix domain socket.
2. Two client modes: new connect every request vs one socket, many round-trips.
3. Three payload sizes: 64 B, 4 KiB, 64 KiB.
4. When the gap is huge (3×) and when it shrinks (25–30%).
5. What this does not claim (WAN, TLS, HTTP/2).

Related links:

• HTTP Keep-Alive vs Connection: close lab
• Nginx limit_req rate-limit lab
• Why your average latency graph is lying (p50 / p95 / p99)
• How to read server monitoring graphs

Lab honesty (30 Sep 2026 IST): Shared Linux lab box (8 cores). Python 3.13.5 echo servers — TCP on 127.0.0.1:18210 only, UDS at a workspace path with mode 0600. Client used TCP_NODELAY on TCP. Warmup 100 ops discarded per arm. No Docker. No public bind. Affiliates: 0.

Verdict up front: for same-host IPC, UDS wins. The win is largest when you connect a lot; persistent connections shrink it. Prefer UDS (or a well-tuned keep-alive TCP client) when both ends live on one machine.

What UDS is (and is not)

A Unix domain socket is a filesystem-path (or abstract-name) endpoint for stream or datagram IPC on one host. No IP addresses, no TCP handshake, no TIME_WAIT from TCP.

Related links:

• man 7 unix — Unix domain sockets
• man 7 tcp — TCP protocol

Mental model that matched our lab:

UDS is not a substitute for auth across machines, not HTTP, and not a fix for a slow accept loop (see the backlog lab).

Lab topology

bench_client.py  ──TCP──►  echo_server.py  127.0.0.1:18210
bench_client.py  ──UDS──►  echo_server.py  …/run/echo.sock

Minimal shapes:

# TCP
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(("127.0.0.1", 18210))
s.listen(128)

# UDS
s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
s.bind("/path/to/echo.sock")
s.listen(128)

Client arms: send N fixed-size payloads, read the echo back, record wall time and per-op latency percentiles.

64-byte payload: connect tax dominates

n=5000 sequential echo round-trips, 64-byte payload:

Connect-per: UDS delivered about 3.0× the RPS of TCP (~49374 / ~16531).

Reuse: UDS was about 1.31× TCP (~154621 / ~118215).

That is the headline: if your “localhost microservice” opens a fresh TCP connection per RPC, you are leaving a large same-host win on the table — either switch to UDS or reuse connections (same lesson as the keep-alive lab, different layer).

4 KiB and 64 KiB: gap shrinks, UDS still ahead

4 KiB, n=3000:

64 KiB, n=1000, reuse only (connect cost is noise next to copy):

Larger payloads move the bottleneck toward copying and scheduling. UDS still led; the dramatic 3× story is a connect-heavy story.

How to read this next to keep-alive

Keep-alive is about reusing HTTP/TCP across requests. UDS is about skipping TCP for same-host peers. They stack:

1. Same host → prefer UDS (or a local abstract socket) when the stack allows it (postgres host, redis unixsocket, nginx listen unix:…, gRPC UDS, etc.).
2. Must use TCP → reuse connections; do not shell out to a new curl per call.
3. Watch percentiles, not only mean RPS — our p50/p95/p99 columns are in the evidence JSON.

Related links:

• HTTP Keep-Alive vs Connection: close lab
• Why your average latency graph is lying (p50 / p95 / p99)

Pitfalls we hit (or avoided)

1. Comparing a reused UDS socket to a connect-per TCP client — unfair. We published both modes for both transports.
2. Forgetting TCP_NODELAY on tiny TCP messages — can inflate TCP latency via Nagle; we enabled it for the TCP client.
3. World-writable socket paths — we used mode 0600 on the socket file. Treat the path like a secret capability.
4. Calling this a WAN result — it is not. Cross-host you need TCP/TLS; measure that separately.
5. Ignoring accept backlog — a fast UDS client against a slow accept loop still drops. Pair with the listen-backlog lab.

Practical checklist

• Same-host DB/cache/proxy: check for a native Unix socket option before defaulting to 127.0.0.1.
• If TCP localhost is mandatory: connection pool / keep-alive; measure connect-per vs reuse.
• Lock down socket filesystem permissions (0600 / dedicated directory).
• Report p50/p95, not only average RPS.
• Do not claim WAN or TLS wins from a loopback echo lab.

Verdict

On this box, Unix domain sockets beat TCP loopback for echo IPC. The ~3× RPS win on 64-byte connect-per arms is the clearest signal; reuse still favored UDS by roughly 25–30%. Use UDS for same-host hops when you can; when you cannot, stop paying a TCP handshake per call.

Evidence path on the lab box: lab-evidence/14-uds-vs-tcp/results/. Affiliates: 0.