difflib vs set Ops Similarity: Localhost Lab

Intro — what this post promises

Compare two moderately different token lists: difflib.SequenceMatcher (.ratio() / .get_opcodes()) vs set intersection/union Jaccard. This lab reports ops/s on Linux localhost.

Frame: difflib for edit-aware similarity and opcodes; sets for bag-of-tokens overlap. Affiliates: 0.

Related links:

• zlib vs gzip compress localhost lab
• queue vs deque handoff localhost lab
• stringio vs list join localhost lab
• groupby vs manual localhost lab
• methodcaller vs getattr localhost lab
• hmac compare digest localhost lab
• argparse vs sys argv localhost lab
• shelve vs pickle dict localhost lab

Lab honesty (1 Oct 2026 IST): Python 3.13.5. No Docker. Tokens drawn from a 38-word vocab; mutate ≈15% replace / 5% delete / 5% insert.

Verdict up front (800 tokens): set Jaccard ~104460.5 ops/s; SequenceMatcher.ratio (tokens) ~4517.3; opcodes ~4410.9; ratio on joined strings ~33.8. Use sets for overlap; difflib when you need edit ops.

Arms

Lab topology

n tokens: 200 · 800 · 2000 · 7 rounds · p50
metric: ops/s = 1 / p50_s

Script: lab-evidence/103-difflib-vs-set-ops/results/run_lab.py.

Lead table — 800 tokens (p50)

Sets win by ~23× vs token-level SequenceMatcher here — different questions, different asymptotics.

Scale sketch (ops/s)

Joined-string ratio() collapses as character length grows — tokenize first if you only care about words.

Which tool when

• Jaccard / set ops: duplicate detection, tag overlap, “same bag of words?” — fast and simple.
• SequenceMatcher: unified diffs, change highlighting, edit distance–ish ratios when order matters.
• Opcodes: drive a UI that shows insert/delete/replace spans.

Do not use Jaccard as a drop-in for edit similarity — reordered identical tokens score high on sets and lower on difflib.

Order vs bag

If two lines swap token order, Jaccard may stay high while SequenceMatcher.ratio drops. Pick the metric that matches user-visible “sameness.” For changelog UIs, opcodes beat any scalar overlap score.

Reading it

• Tokenize before SequenceMatcher when inputs are prose-sized.
• Prefer sets when multiplicity/order do not matter.
• Cache SequenceMatcher results if you call both ratio and opcodes (autojunk / internals).
• Report the metric that matches the product question.

Multiplicity caveat

set Jaccard collapses duplicate tokens. If “three copies of error” should weigh more than one, use collections.Counter multiset overlap instead — slower than pure sets, still usually far cheaper than SequenceMatcher on long sequences.

Opcode consumers

get_opcodes() returns 5-tuples (tag, i1, i2, j1, j2). Driving a diff UI from opcodes is the usual win; calling ratio() alone when you already need opcodes wastes a second pass unless you reuse one matcher instance.

Pitfalls

• Running SequenceMatcher on multi‑KB raw strings in a hot loop.
• Treating Jaccard == SequenceMatcher.ratio numerically (they diverge).
• Ignoring duplicates — sets collapse multiplicity; bags need Counter.
• Comparing autojunk on/off without documenting it.

Reproduce

python3 lab-evidence/103-difflib-vs-set-ops/results/run_lab.py

Evidence: summary.json, summary.txt.

Limits

One Linux box. Synthetic vocab + seeded mutations. Not difflib.unified_diff file I/O. Not embedding cosine similarity.

Takeaway

At 800 tokens, set Jaccard ~104460.5 ops/s crushed SequenceMatcher.ratio ~4517.3 ops/s. Use sets for overlap, difflib for edit ops — and avoid char-level ratio() on long joined strings (~33.8 ops/s here).