Celebrating World Digital Preservation Day: Saving Science’s Digital History through Web Archiving at AIP

Websites might not be the first thing that comes to mind when thinking about historical archives, but they have become an essential part of how science happens and how its stories are told. From sharing new research and reporting on science policy to connecting scientific communities, an enormous amount of the history of 21st-century science is unfolding online. But websites are constantly changing, and when pages are updated or taken down, pieces of that history can disappear.

In celebration of World Digital Preservation Day , the Niels Bohr Library & Archives (NBLA) is excited to share how it is meeting this challenge by expanding into a new collecting area: websites! Using Archive-It , a web-archiving service from the Internet Archive (often known by its Wayback Machine service), NBLA is working to capture and preserve the digital footprints of scientific organizations, institutions, and individual scientists. It’s a natural next step in AIP’s longstanding work to preserve the history of the physical sciences, ensuring that today’s digital records become tomorrow’s documentation and historical resources.

NBLA started piloting its web archiving program just as AIP was wrapping up a major update and migration of its own website. NBLA began piloting crawls of AIP’s websites, including its main site and programs like FYI, AIP’s science policy news service; SPS, the Society of Physics Students; and retired initiatives like the science news service Inside Science. A web crawl essentially takes a snapshot of a website at a point in time, capturing its pages, content, and other features so they can be revisited long after the live website has changed. After plenty of testing, troubleshooting, and fine-tuning to make sure these captures would be useful for future researchers, AIP’s websites are being archived on a regular schedule. The program is now expanding to include websites from AIP Member Societies and selected scientists and scientific institutions, building on the rich collections already preserved at NBLA.

Browse some of the website collections:

• AIP’s websites
• AIP Federation Websites
  
  • AAPM Web Archive
  • SoR Web Archive
• Institutions & Organizations
• Scientist Websites
• Higher Education Physics & Astronomy Departments

Preserving the Websites of Scientists: UMD MLIS Student Margaret Stine on her Field Study Project

As an intern at AIP’s Niels Bohr Library & Archives, I had the privilege of laying the foundation for the Scientist Websites collection . This project was formed to address the concern of digital loss of websites and the resulting prevention of long-term information access. Smaller websites created and managed by individuals can be especially vulnerable if they lack the security resources, technical maintenance, and backup systems that larger organizational websites might have. Given the fact that born digital material has no physical copy to rely upon once a service provider is discontinued, digital archiving has a unique role in preserving non-tangible materials.

I navigated this project by initially scoping AIP’s Oral History Interview (OHI) collection. As this collection of interviews with individuals is one of the most utilized resources from NBLA, preserving corresponding personal scientist websites is incredibly useful for tying together scientists’ lives and their research. Scientists’ personal, public websites generally include a brief biography, relevant work experience (usually in the form of a CV), information about their labs/research, details about lab members, links to presentations, and much more.

AIP now preserves an oral history interview with Dr. Jami Valentine Miller as well as her public website.

In addition to pulling scientists from the OHI source list I also branched out to examining member and affiliate societies of AIP, including researchers and professors with ties to the AIP Federation. My methodology was as follows: I first identified scientists whose websites captured their research and professional representation in some capacity. From there, I compiled the websites’ base URLs into a spreadsheet with corresponding metadata describing websites and their creators. Once I gathered the list of scientists, I selected a group to pilot based on strongest relations to existing NBLA collecting criteria, uploaded the URLs (called “seeds” in Archive-It) to Archive-It alongside their corresponding metadata, and ran a test crawl. After the crawl ran for about a week, I then reviewed the results of the test crawl to ensure I captured all relevant data by examining the URL reports and crawl previews via the Wayback Machine. After confirming there were no major issues with the crawl, I saved the collected data, successfully digitally preserving these websites for future researchers and members of the physical sciences community.

One exciting example from my project includes archiving the most recent Nobel prize winner, Francis Halzen’s, website. After locating his personal, public webpage, I followed my workflow, gathering all relevant metadata including his name, seed URL, affiliate institution, subject headings, and a brief description of the content of his site. Then, I uploaded this data to Archive-It and ran a test crawl. I reviewed the resulting capture and examined the site’s content I collected, including lecture notes from his classes and presentations on his prize-winning work with IceCube going back to 1998! After confirming all the data within his site domain were captured, I saved the crawl.

Why Archive the Web?

Why does all this matter? Future historians studying the 21st century will need more than published research papers to understand the world of science today. They’ll want to know how scientific communities responded to changing policies, shifting research priorities, funding challenges, and new opportunities. Much of that story is playing out on websites right now, during an especially eventful and uncertain period for science. As AIP’s web archives grow, these individual snapshots will build into a fascinating historical record of how science and its institutions evolved over time. World Digital Preservation Day is a great reminder that preserving digital history isn’t just about saving old files—it’s about making sure the stories being created in today’s digital world are still around to be discovered tomorrow. With its new web-archiving program, AIP is helping ensure that the history of 21st-century science doesn’t disappear with the next website update.