glob vs rglob vs os.walk: Listing Lab

Intro — what this post promises

Need every file under a tree: is os.walk, pathlib.Path.rglob, or glob.glob(..., recursive=True) fastest? This lab builds a modest fixture tree on disk and reports wall time / files/s on Linux localhost.

Related links:

• pathlib vs os.path localhost lab
• shutil copyfile vs manual localhost lab
• json dumps compact vs indent localhost lab
• str translate vs replace localhost lab
• setdefault vs defaultdict localhost lab
• itertools chain vs flatten localhost lab
• hashlib md5 vs blake2b localhost lab
• perf_counter vs time localhost lab

Lab honesty (1 Oct 2026 IST): Python 3.13.5. Fixture at lab-evidence/76-glob-vs-rglob-vs-walk/fixture_tree/. Affiliates: 0. “Files” arms filter isfile / is_file(); “all” arms include directories.

Verdict up front (modest, 1,210 files): os.walk ~1.30 ms (~929k/s) vs rglob+is_file ~5.95 ms vs glob+isfile ~6.19 ms. Walk led glob by ~4.8× and rglob by ~4.6×. Prefer os.walk for raw listing speed; rglob for Path ergonomics.

Arms

Trees: modest (depth 2) and wider (depth 3).

Lab topology

modest: ~1210 files; wider: ~4170 files
metric: p50 wall; entries/s
fixture persisted under lab-evidence/76-.../fixture_tree/

Script: lab-evidence/76-glob-vs-rglob-vs-walk/results/run_lab.py.

Lead table — modest tree (p50)

Wider tree (4,170 files)

Same ranking at larger scale: walk ~4.7× glob files.

Reading it

• os.walk wins listing — fewer Path object allocations; classic C-backed scandir walk.
• rglob ≈ glob for files after is_file filters (~1.04× modest).
• Filtering is costly — “all entries” glob/rglob is much closer to walk; isfile checks dominate the files arms.
• Use Path when you need Path methods next; don’t pick rglob expecting a speedup over walk.

Fixture note

The tree is generated into evidence so you can re-list the same shape. It is not a kernel FS stress test — enough files to rank APIs, not millions of inodes.

Why isfile filters hurt

glob/rglob return directory entries too. Turning that into a file list means a metadata check per path. os.walk already separates dirnames from filenames, so the files arm mostly pays string joins. If you only need counts or names inside each directory, stay on walk and avoid a second pass of Path.is_file().

Pitfalls

1. Comparing glob-all to walk-files — different entry sets (dirs included).
2. Calling Path.is_file() on every rglob hit without needing stat — expensive.
3. Assuming recursive glob is “free” — pattern matching + filters add up.
4. Network FS / huge trees — re-measure; locality matters.

When to pick what

Reproduce

python3 lab-evidence/76-glob-vs-rglob-vs-walk/results/run_lab.py

Evidence: /workspace/lab-evidence/76-glob-vs-rglob-vs-walk/results/ (+ fixture_tree/).

Closing

Walk to list; Path to ergonomics. On this box modest walk hit ~929k files/s, about ~4.8× glob+isfile and ~4.6× rglob+is_file. Reach for recursive glob when the pattern is the product; reach for walk when the product is “all files under here, fast.”