News

The Digital Scholar: Reference Rot and Link Decorations

February 6, 2015 1272

Referencing sources is a fundamental part of the scholarly discourse. This was true 50 years ago and still holds true today, even though, with the emergence of the web, scholarly communication has undergone a dramatic shift from a paper-based to a web-based endeavor. What has changed, however, is the variety of scholarly assets being referenced. Increasingly, we see references to software, ontologies, project websites, presentations, blogs, videos, tweets, etc. Such resources are usually referenced by means of their HTTP “uniform reference identifier,” or URI, as they exist on the web at large. These HTTP URIs allow for immediate access on the web, but also introduce one of the detrimental characteristics of the web to scholarly communication: reference rot.

We introduced the term ‘reference rot’ as a combination of two problems common for URI references: link rot and content drift. Link rot represents the case where a URI ceases to exist and hence the request typically returns a “404 – Page not found” response. Content drift describes the case where the resource identified by its URI changes over time and hence, as time goes by, the request returns content that becomes less and less representative of what was originally referenced.

LSE Impact logo

This article by Martin Klein and Herbert Van de Sompel originally appeared on the LSE Impact of Social Sciences blog as “Reference rot in web-based scholarly communication and link decoration as a path to mitigation” and is reposted under the Creative Commons license (CC BY 3.0).

Our recent study, part of the research track of the Hiberlink project, investigates the extent to which science, technology, and medicine (STM) articles are subject to reference rot. Numerous link rot studies for the general web as well as for scholarly communication have been published over the years but the investigation into the aspect of content drift as well as the sheer scale of our experiments make our study unique to date. We assembled three corpora (arXiv, Elsevier, PubMed Central) consisting of more than 3.5 million scholarly articles published between 1997 and 2012. Out of the 4 million URIs that we found (in about 1.8 million of the considered articles), more than 1 million became subject to our analysis as they referenced web at large resources (blogs, project websites, etc.) as described above. URIs that referenced STM articles were excluded from our study on the assumption that existing persistent identifier and archival infrastructure makes such links immune to reference rot.

Our results show an alarming link rot ratio for all three corpora: 13 percent of arXiv, 22 percent of Elsevier, and 14 percent of PMC articles published in 2012 suffer from link rot. These numbers only increase for older articles, for example, for articles published in 2005 the corresponding numbers are 18 percent, 41 percent, and 36 percent.

Reference Rot in Web

The graphic shows links from the three STM corpora that were studied (right hand side) to the six top level domains they link to (left hand side). The colored portions shows links that are healthy, the grey portions – by far the largest – show links that are infected by reference rot.

We investigated the notion of content drift by checking for archival copies (Mementos) of the referenced web at large resource in web archives. We considered a Memento created within 14 days of the article’s publication date as representative to what was originally intended for the reference. While this time window is chosen somewhat arbitrarily, given the dynamic character of the web, we make the case that chances are fairly high that the referenced web at large resource has changed within 14 days and hence does not necessarily represent the intended content of the reference anymore. Throughout all three corpora, about 75 percent of all URI references lack such representative Mementos.

To elevate these numbers to the article level, we defined an article as being subject to reference rot if it contains at least one URI reference that is either subject to link rot or is lacking a representative Memento. We extrapolated our Elsevier numbers to the scale of the overall STM article landscape and arrived at eye-opening numbers: about 20 percent of STM articles published in 2012 suffer from reference rot. The remaining 80 percent either do not contain any URI references or, when they do, they all still work and a representative Memento is available for each. When considering only those articles that contain one or more URI references, the ratio of infected articles published in 2012 increases to 70 percent.

Reference rot is clearly a significant problem, and, in addition to quantifying its extent, the Hiberlink project also explores ways to mitigate it. The typical strategy to address the problem, followed by those who actually care about it, is as follows:

  • When linking to a web page, a snapshot of the state of the page at linking time is created in a web archive. Several web archives provide on-demand snapshot functionality, including the Internet Archive, archive.today, and perma.cc.
  • With the snapshot created, rather than linking to the original web page, a link to the snapshot is put in place.

For example, we are writing this post on January 21 2015. And we want to link to http://www.w3.org/. That’s the W3C’s page, and it changes rather frequently. In order for future readers of this blog post to see the same W3C content that we saw when linking, we create a snapshot, say in the archive.today web archive. That snapshot is available at https://archive.today/r7cov and, rather than linking to http://www.w3.org/, we link to https://archive.today/r7cov.

While the creation of the snapshot is definitely an essential step in the right direction, there are problems with the linking approach:

  • Linking to the snapshot https://archive.today/r7cov, assumes that the archive.today web archive will exist forever. Unfortunately, there are plenty of indications that web archives do not have eternal life either. If the web archive in which we created the snapshot suffers a temporary glitch, moves its content to another web location, or ceases to exist, visiting the snapshot becomes impossible.
  • When linking to the snapshot https://archive.today/r7cov, the URI of the W3C’s page, http://www.w3.org/, is lost. As a result, future readers of this post cannot visit the W3C page to see its evolved state.

Link decoration is a way to address these problems and to increase the chances that links will lead to meaningful content, even a long time after they were put in place. In order to maximize link robustness, the following information should be available, in a machine-actionable manner, for a link:

The latter two information elements can be used to automatically find snapshots in other web archives in case archive.today‘s service is interrupted, and the snapshot https://archive.today/r7cov becomes inaccessible as a result.

Discussions with interested parties are still underway regarding the best way to convey this information on a link. Until further notice, and for demonstrations purposes, the information is conveyed using HTML5′s attribute extensibility mechanism. Using that approach, this robust link to the W3C home page looks as follows:

Code ccapture

The Memento Time Travel extension for Chrome makes these link decorations accessible when right-clicking on the link. Try it with a version of the reference list of our above mentioned article in which links to web at large resource were decorated. For more information on link decoration, check out the Robust Links site.


Martin Klein is an analyst at the University of California, Los Angeles Research Library. He can be found on Twitter @mart1nkle1n. Herbert Van de Sompel is the team leader of the Prototyping Team at the research library of the Los Alamos National Laboratory. The team does research regarding various aspects of scholarly communication in the digital age. He can be found on Twitter @hvdsomp

View all posts by Martin Klein and Herbert Van de Sompel

Related Articles

Daniel Kahneman, 1934-2024: The Grandfather of Behavioral Economics
News
March 27, 2024

Daniel Kahneman, 1934-2024: The Grandfather of Behavioral Economics

Read Now
2024 Holberg Prize Goes to Political Theorist Achille Mbembe
News
March 14, 2024

2024 Holberg Prize Goes to Political Theorist Achille Mbembe

Read Now
New Feminist Newsletter The Evidence Makes Research on Gender Inequality Widely Accessible
Impact
March 4, 2024

New Feminist Newsletter The Evidence Makes Research on Gender Inequality Widely Accessible

Read Now
Contemporary Politics Focus of March Webinar Series
News
February 21, 2024

Contemporary Politics Focus of March Webinar Series

Read Now
There’s Something In the Air…But Is It a Virus? Part 1

There’s Something In the Air…But Is It a Virus? Part 1

The historic Hippocrates has become an iconic figure in the creation myths of medicine. What can the body of thought attributed to him tell us about modern responses to COVID?

Read Now
The Social Sciences Are Under Attack in Higher Education

The Social Sciences Are Under Attack in Higher Education

The social sciences have been a consistent target for political operatives around the United States in recent years., and recent actions at the state level have opened a new front in the long-running conflict.

Read Now
Jonathan Breckon On Knowledge Brokerage and Influencing Policy

Jonathan Breckon On Knowledge Brokerage and Influencing Policy

Overton spoke with Jonathan Breckon to learn about knowledge brokerage, influencing policy and the potential for technology and data to streamline the research-policy interface.

Read Now
0 0 votes
Article Rating
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

2 Comments
Newest
Oldest Most Voted
Inline Feedbacks
View all comments