Why Archive Sources Sometimes Disagree

Run the same query against two Reddit archives and the results will not match. One holds a post the other missed. Scores differ. Date ranges differ.

The instinct is to decide which one is right. That instinct is wrong, and acting on it discards the most useful information the comparison offers.

Both are samples

Neither archive is Reddit. Each is a record of what one project managed to observe, using its own infrastructure, on its own schedule, subject to its own outages.

Asking which is correct is like asking which of two photographs of a street is the real street. They were taken at different moments from different positions. Both are accurate about what they captured.

Where the differences come from

Capture timing. A score sampled an hour after posting and the same score sampled a week later are different numbers describing the same post at different moments. Neither is an error.

Collection windows. Every archive has periods where collection was degraded or stopped. Those periods rarely align, so one archive's gap is another's coverage.

Edit timing. A post edited between two captures exists in the archives in two versions. Whichever you read is one moment in that content's life.

Retention and scope. Projects differ in what they keep and how they handle deletions, quarantined communities and banned subreddits.

Query semantics. Two services can interpret the same-looking parameters differently. A date filter that is inclusive in one and exclusive in the other produces different counts from identical intent.

That last one is worth checking before concluding anything about coverage. Some apparent disagreements are about the question, not the data.

What disagreement tells you

A record exists in one archive only. That is proof the content existed, and evidence the other archive's coverage has a gap there. Both are useful.

Scores differ substantially. The content was volatile, or one capture happened before the score settled. Either way, the score is a weak number to build on.

Date ranges differ. One archive covers a period the other does not, which matters enormously if that period is central to your work.

Counts differ by a lot. Something structural differs — coverage, query semantics, or retention. Worth understanding before either number is used.

Handling it in practice

Merge by ID, not by preference. Records sharing a Reddit ID are the same record. Combine them and keep the sources attached.

Preserve conflicts rather than resolving them. Where two sources give different values for a field, keeping both and marking the disagreement is more honest than picking one. Silently preferring a source produces a cleaner number that conceals a real uncertainty.

Never pick the source that supports your conclusion. This is the obvious hazard. The subtler one is picking the source with more records, which feels principled and is equally arbitrary.

Report the disagreement when it matters. If two archives differ substantially on something central, that belongs in your write-up. It tells the reader the record is less settled than a single figure implies, which is true.

The reframe

Disagreement between archives is not noise obscuring a signal. It is a direct measurement of how uncertain the historical record is.

A single archive gives you one number and no way to know how much to trust it. Two archives that agree give you some confidence. Two that disagree tell you to be careful — which is more useful than false precision, and is the only situation in which you get that warning at all.