Tag: Research Citations

The link works; the evidence does not

A URL that still loads is no guarantee that the page behind it still supports what you linked. The request succeeds, the browser renders, and the fact you relied on is gone. Researchers call this content drift: a resource keeps its address while its content changes until it no longer represents what someone cited.

Broken links are honest; drift is not

A dead link announces its failure. A 404 or an expired domain tells you something is missing. Content drift never announces itself. The click succeeds, the page looks normal, and nothing warns you that the sentence you came to check was rewritten a day, a month, or a decade ago.

The stockpile page that changed overnight

One documented example comes from the early pandemic. The Strategic National Stockpile page described the stockpile as supporting “state, local, tribal, and territorial responders.” Archived versions from April 3 and April 4, 2020 show the wording changing immediately after public controversy over how the stockpile was supposed to serve states. A Los Alamos web-preservation report uses the two captures as a concrete example of content drift: the live URL survived while the policy language at that URL changed.

How often it happens

Drift is common. In 2016, Shawn Jones and colleagues published Scholarly Context Adrift, examining web references drawn from millions of scholarly articles. Among references for which a representative archived copy could be compared with the live resource, more than three quarters had changed.

Dated records are the workaround

That is why dated captures and version records matter. The Wayback Machine and the Memento protocol are built around the same basic need: reaching a version of a web resource as it existed at a particular time. Recovery has limits. Most references in that 2016 study had no usable archived copy near the publication date, and an archive only preserves what someone captured, whenever they captured it.

When a URL still works, the instinct is to trust it. The fix is a timestamp: save a copy the day you read it, note the date, and link to the capture alongside the address. Drift is silent by default; the way to hear it is to keep a record of what you saw.

Read a paper from a decade ago and you will eventually follow a footnote to the web. Sometimes the page is still there. Often it is not.

How research references rot

When a database, a government report, or a project page vanishes, every paper that cited it points into the void. That is link rot: the URL stops resolving and returns a 404 or an error. Its quieter cousin is content drift, where the address still works but the page now shows something different. A journalism site can keep the URL for an old article while silently replacing the material behind it — the link works, and the citation is still wrong. Researchers Martin Klein and Herbert Van de Sompel, with colleagues, grouped both problems under the single term reference rot.

Their 2014 study in PLOS ONE checked over one million references to web resources pulled from more than 3.5 million science, technology, and medicine articles published between 1997 and 2012. It found reference rot in roughly one in five articles overall; among articles containing web references, the share rose to about seven in ten.

The law decays the same way. Jonathan Zittrain, Kendra Albert, and Lawrence Lessig studied legal citations and found that more than 70 percent of URLs in the law-journal sample and 50 percent of URLs in U.S. Supreme Court opinions no longer pointed to the originally cited information.

Why missing evidence matters

A citation has a job: let a later reader find the material a claim rests on and check whether the claim is fair. Break that path and the paragraph stands on an assertion no one can inspect. Replication becomes guesswork, and a researcher who wants to reuse the dataset linked in a methods section finds it went down with the lab page that hosted it.

The subtler damage comes from pages that appear healthy. Automated checks catch plain 404s, but a page returning a normal status code can still be a custom error page, a redirect to a homepage, or a rewritten article that no longer contains the cited fact. A dead link at least announces its failure; drift does not. It quietly converts a reproducible citation into an unsupported one, in a legal opinion as readily as in a scientific paper.

What persistent identifiers and archives can preserve

Two tools stretch the shelf life of a citation. The first is a persistent identifier such as a DOI or a handle. It keeps a stable name for a resource that can change address. But an identifier is a name, not a copy. If the publisher’s servers disappear, the DOI still resolves — to an error.

The second tool is an archived copy. Web archives such as the Internet Archive’s Wayback Machine keep snapshots later readers can retrieve; the Memento protocol exists specifically to find a snapshot from around the date a paper appeared. The 2014 report behind Perma.cc came out of Harvard’s legal citation study: instead of hoping a cited page stays up, authors archive it up front and link to a preserved copy held across a distributed network of libraries. Journals increasingly archive their own supplementary material, easiest to capture while the author’s browser is still pointing at it.

These measures do not make citations immortal. Archives choose what to capture and can miss heavily dynamic or login-gated pages. A snapshot may catch the page as it briefly looked, not as the author actually saw it. An archive is only as permanent as its funding.

What survives is decided early. Citing a DOI, archiving a copy while the page is still live, and preferring sources with existing archival coverage all cost little at writing time. The alternative is a footnote that spends its life pointing at a hole. The internet forgets on a schedule; citations that remember to archive can leave a record that outlasts the page they cite.