How to Compare Two Log Files for Differences

Something broke after the last deploy. You’ve got two log files — one from before, one from after — and a hunch that the answer is buried somewhere in the difference between them. Scrolling through thousands of lines looking for “something new” is slow and error-prone, especially when 95% of each file is identical, expected noise.

Why log files resist ordinary diffing

Standard line-by-line diff tools were built for source code, where lines are stable and meaningful. Logs break that assumption in a few specific ways:

Every line has a timestamp. A naive diff sees 10:32:01 request started and 10:32:04 request started as two completely different lines, even though only the timestamp changed and the event is identical.
Line counts rarely match. More traffic, a retried request, or a slightly different startup sequence shifts every subsequent line, and a positional diff (line 500 vs line 500) turns into noise almost immediately.
The signal is sparse. In a 10,000-line log, the actual new error might be three lines. Everything else is routine and expected — the challenge is finding the needle, not confirming there’s a haystack.

Strip what you don’t need before comparing

The single highest-leverage step is removing timestamps and other per-line unique identifiers (request IDs, PIDs) before diffing, so the comparison focuses on the actual message content:

# Strip a leading ISO timestamp from each line
sed -E 's/^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}[^ ]* //' before.log > before_clean.log
sed -E 's/^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}[^ ]* //' after.log > after_clean.log

Adjust the pattern to match your log format (syslog, JSON lines, Apache combined log, etc.) — the goal is the same regardless of format: keep the message, drop the parts that are unique on every single line by design.

Diff the cleaned files

Once timestamps are stripped, a plain text comparison does the real work. Paste both cleaned files into DiffALL’s text comparison tool and read the highlighted differences:

Lines only in the “after” file are new — this is usually where the new error or new behavior lives.
Lines only in the “before” file are missing — a step that used to run and silently stopped.
Changed lines (same position, different content) often mean an error message’s details changed even though the same code path fired — worth checking whether that’s the actual bug.

Filtering by severity first

Before comparing the full files, it’s often faster to filter both down to just warnings and errors:

grep -E "ERROR|WARN|FATAL" before.log > before_errors.log
grep -E "ERROR|WARN|FATAL" after.log > after_errors.log

Compare the filtered files first. If the answer isn’t there, widen to the full logs — but most “what broke” investigations resolve at this level, because routine INFO/DEBUG lines rarely carry the signal you’re looking for.

Structured (JSON) logs need a different first step

If your logs are JSON lines rather than plain text, pretty-print each entry onto multiple lines before diffing — a minified JSON log line is nearly unreadable in a diff view, and small field-order differences look like huge changes even when nothing meaningful moved:

jq -c 'del(.timestamp, .request_id)' before.jsonl | jq . > before_pretty.log
jq -c 'del(.timestamp, .request_id)' after.jsonl | jq . > after_pretty.log

This does two things at once: drops the per-request-unique fields (same principle as the timestamp-stripping above) and reformats each JSON object onto multiple readable lines, so the diff highlights the field that actually changed instead of highlighting an entire minified line because one character moved.

What to look for once you have a clean diff

Pattern in the diff	Likely meaning
New `ERROR` lines with no counterpart in “before”	The direct cause — start here
A `WARN` that appears far more frequently in “after”	A retry loop or degraded fallback path that wasn’t there before
Missing lines from a step that used to log	That step is failing silently or was skipped entirely
Same error, different stack trace	The failure mode changed, even if the symptom looks the same
Everything matches except volume	Likely a load/traffic issue, not a code change

Checklist

[ ] Strip timestamps and per-request unique IDs before comparing.
[ ] Filter to ERROR/WARN first; widen to full logs only if needed.
[ ] Pretty-print structured (JSON) logs onto multiple lines before diffing.
[ ] Check both directions — lines only in “after” (new) and only in “before” (missing).
[ ] Confirm whether a difference is a real regression or just a change in traffic volume.

The two log files already contain the answer — the work is removing the noise that hides it. Strip timestamps, filter by severity, then run a text comparison on what’s left.