Skip to main content
Internet Archive's 25th Anniversary Logo

End of Term Web Crawls

IA

This collection includes web crawls of the Federal Executive, Legislative, and Judicial branches of government performed at the end of US presidential terms of office.



rss RSS

41,726
RESULTS


Show sorted alphabetically

Show sorted alphabetically

SHOW DETAILS
up-solid down-solid
eye
Title
Date Archived
Creator
End of Term 2016 Web Crawls
End of Term 2016 Web Crawls
collection
19,084
ITEMS
35.1M
VIEWS
collection

eye 35.1M

This collection contains web crawls performed as part of the End of Term Web Archive, a collaborative project that aims to preserve the U.S. federal government web presence at each change of administration. Content includes publicly-accessible government websites hosted on .gov, .mil, and relevant non-.gov domains, as well as government social media materials. The web archiving was performed in the Fall and Winter of 2016 and Spring of 2017. For more information, see...
Topics: end of term, federal government, 2016, president, congress, government data
End Of Term 2020 Web Crawls
collection
19,468
ITEMS
6.3M
VIEWS
collection

eye 6.3M

End of Term 2012 Web Crawls
End of Term 2012 Web Crawls
collection
2,383
ITEMS
31.4M
VIEWS
collection

eye 31.4M

This collection contains web crawls performed on the US Federal Executive, Legislative & Judicial branches of government in 2012-2013.
Topics: end of term, US, Federal government, 2012, Obama
End Of Term 2020 Post Inauguration Crawls
collection
19,088
ITEMS
3M
VIEWS
collection

eye 3M

End Of Term 2016 Pre-Inauguration Crawls
End Of Term 2016 Pre-Inauguration Crawls
collection
4,693
ITEMS
13M
VIEWS
collection

eye 13M

This collection contains web crawls performed as the pre-inauguration crawl for part of the End of Term Web Archive, a collaborative project that aims to preserve the U.S. federal government web presence at each change of administration. Content includes publicly-accessible government websites hosted on .gov, .mil, and relevant non-.gov domains, as well as government social media materials. The web archiving was performed in the Fall and Winter of 2016 to capture websites prior to the January...
Topics: end of term, federal government, 2016, president, congress
End Of Term 2016 UNT Crawls
End Of Term 2016 UNT Crawls
collection
1,275
ITEMS
10.9M
VIEWS
collection

eye 10.9M

End of Term 2016 Web Archive government web crawls by project partner the University of North Texas.
Topics: end of term, federal government, 2016, president, congress, university of north texas
Government Web & Data Archive
Government Web & Data Archive
collection
6,269
ITEMS
12.2M
VIEWS
collection

eye 12.2M

This collaborative project is an extension of the 2016  End of Term  project, intended to document the federal government's web presence by archiving government websites and data. As part of this preservation effort, URLs supplied from partner institutions, as well as nominated by the public, will be crawled regularly to provide an on-going view of federal agencies' web and social media presence. Key partners on this effort are the Environmental Data & Governance...
Topics: government, data, federal, congress
collection

eye 15.9M

End of term 2008 crawl data gathered by Internet Archive on behalf of the California Digital Library. This data is currently not publicly accessible.
End of Term 2008 UNT Dot Gov Crawl
End of Term 2008 UNT Dot Gov Crawl
collection
282
ITEMS
9.6M
VIEWS
collection

eye 9.6M

End of term 2008 crawl of .gov domains gathered by University of North Texas . This data is currently not publicly accessible. UNT is a student-focused, public, research university located in Denton, Texas.
End Of Term 2016 Library of Congress Crawls
End Of Term 2016 Library of Congress Crawls
collection
3,892
ITEMS
5.3M
VIEWS
collection

eye 5.3M

End of Term 2016 Web Archive government web crawls by project partner the Library of Congress.
Topics: end of term, federal government, 2016, president, congress, library of congress, web, data, library...
UNT Web
UNT Web
collection
35
ITEMS
5.8M
VIEWS
collection

eye 5.8M

This collection contains all collaborative crawl data contributed by University of North Texas (UNT).
Topics: UNT, web, texas, eot
End of Term 2016 Post-Inauguration Crawls
End of Term 2016 Post-Inauguration Crawls
collection
5,308
ITEMS
5.1M
VIEWS
collection

eye 5.1M

This collection contains web crawls performed as the post-inauguration crawl for part of the End of Term Web Archive, a collaborative project that aims to preserve the U.S. federal government web presence at each change of administration. Content includes publicly-accessible government websites hosted on .gov, .mil, and relevant non-.gov domains, as well as government social media materials. The web archiving was performed in the Winter of 2016  and Spring of 2017 to capture websites...
Topics: end of term, federal government, 2016, president, congress
End of Term 2020 UNT Crawls
End of Term 2020 UNT Crawls
collection
377
ITEMS
82,031
VIEWS
collection

eye 82,031

This collection contains web crawls performed by the UNT Libraries on the US Federal Executive, Legislative & Judicial branches of government in 2020-2021. The seeds for these crawls came from human-nominated URLs that can be viewed in the Nomination Tool  as well as from lists generated by volunteers at UC San Diego and Environmental Data & Governance Initiative (EDGI). Information about this project can be found here: https://end-of-term.github.io/eotarchive/ .
UNT Web
by Internet Archive
web

eye 685,008

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-101.us.archive.org:UNT from Wed Dec 31 05:24:10 PST 2008 to Fri Jan 18 20:34:17 PST 2013.
Topic: crawldata
UNT Web
by Internet Archive
web

eye 828,181

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-101.us.archive.org:UNT from to Fri Jan 18 22:32:21 PST 2013.
Topic: crawldata
End of Term 2008 UNT Dot Mil Crawl
End of Term 2008 UNT Dot Mil Crawl
collection
50
ITEMS
661,973
VIEWS
collection

eye 661,973

End of term 2008 crawl of .mil domains gathered by University of North Texas . This data is currently not publicly accessible. UNT is a student-focused, public, research university located in Denton, Texas.
Data crawled by University of North Texas on behalf of University of North Texas from Fri Jun 04 23:39:52 PDT 2010 to Tue Sep 16 00:56:27 PDT 2008
Topic: crawldata
End of Term 2012 Web Crawls
web

eye 102,808

favorite 0

comment 0

Internet Archive crawldata from End of Term 2012 Collaborative, captured by wbgrp-crawl013.us.archive.org:EOT-2012 from Thu Feb 21 08:47:55 PST 2013 to Thu Feb 21 13:50:20 PST 2013.
Topic: crawldata
UNT Web
by Internet Archive
web

eye 316,989

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-101.us.archive.org:UNT from Fri Oct 24 15:13:03 PDT 2008 to Fri Jan 18 18:24:10 PST 2013.
Topic: crawldata
UNT Web
by Internet Archive
web

eye 255,291

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-101.us.archive.org:UNT from Fri Oct 24 15:21:48 PDT 2008 to Fri Jan 18 18:43:41 PST 2013.
Topic: crawldata
UNT Web
by Internet Archive
web

eye 343,026

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-101.us.archive.org:UNT from Fri Jan 16 00:47:17 PST 2009 to Sat Jan 19 01:21:21 PST 2013.
Topic: crawldata
UNT Web
by Internet Archive
web

eye 301,676

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-101.us.archive.org:UNT from Fri Oct 24 15:04:08 PDT 2008 to Fri Jan 18 19:01:08 PST 2013.
Topic: crawldata
End of Term 2008 California Digital Library Crawl
by Internet Archive
web

eye 182,423

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-102.us.archive.org:CDL from Tue Jan 20 07:19:25 PST 2009 to Mon Nov 5 02:51:51 PST 2012.
Topic: crawldata
End of Term 2008 California Digital Library Crawl
by Internet Archive
web

eye 240,751

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-102.us.archive.org:CDL from Tue Jan 20 10:38:17 PST 2009 to Fri Jan 4 03:31:49 PST 2013.
Topic: crawldata
Data crawled by University of North Texas on behalf of University of North Texas from Sat Aug 22 02:30:45 PDT 2009 to Wed Aug 26 06:17:29 PDT 2009
Topic: crawldata
UNT Web
by Internet Archive
web

eye 212,324

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-101.us.archive.org:UNT from to Fri Jan 18 23:13:57 PST 2013.
Topic: crawldata
UNT Web
by Internet Archive
web

eye 179,395

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-101.us.archive.org:UNT from Tue Dec 2 16:02:08 PST 2008 to Fri Jan 18 22:36:24 PST 2013.
Topic: crawldata
Data crawled by University of North Texas on behalf of University of North Texas from Wed Aug 26 06:26:34 PDT 2009 to Thu Aug 27 15:09:03 PDT 2009
Topic: crawldata
End of Term 2012 Web Crawls
web

eye 65,085

favorite 0

comment 0

Internet Archive crawldata from End of Term 2012 Collaborative, captured by wbgrp-crawl013.us.archive.org:EOT-2012 from Tue Mar 12 19:50:00 PDT 2013 to Tue Mar 12 13:51:20 PDT 2013.
Topic: crawldata
UNT Web
by Internet Archive
web

eye 318,008

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-101.us.archive.org:UNT from to Fri Jan 18 18:45:26 PST 2013.
Topic: crawldata
End of Term 2008 California Digital Library Crawl
by Internet Archive
web

eye 179,899

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-102.us.archive.org:CDL from Wed Jan 14 01:15:08 PST 2009 to Fri Jan 4 03:01:32 PST 2013.
Topic: crawldata
UNT Web
by Internet Archive
web

eye 195,791

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-101.us.archive.org:UNT from Mon Jan 19 20:35:44 PST 2009 to Sat Jan 19 02:12:16 PST 2013.
Topic: crawldata
Data crawled by University of North Texas on behalf of University of North Texas from Thu Aug 27 15:39:00 PDT 2009 to Fri Sep 25 07:40:39 PDT 2009
Topic: crawldata
End Of Term 2016 Pre-Inauguration Crawls
web

eye 39,626

favorite 0

comment 0

Internet Archive crawldata from End of Term 2016, captured by wbgrp-crawl005.us.archive.org:EOT-2016 from Thu Dec 1 16:13:31 PST 2016 to Thu Dec 1 08:32:22 PST 2016.
Topic: crawldata
UNT Web
by Internet Archive
web

eye 288,562

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-101.us.archive.org:UNT from Fri Mar 13 03:40:08 PDT 2009 to Fri Jan 18 19:24:37 PST 2013.
Topic: crawldata
Data crawled by University of North Texas on behalf of University of North Texas from Tue Sep 16 10:16:10 PDT 2008 to Tue Sep 16 11:32:37 PDT 2008
Topic: crawldata
Internet Archive crawldata from End of Term 2020, captured by crawl813.us.archive.org:eot20 from Thu Mar 18 00:01:04 PDT 2021 to Wed Mar 17 17:31:03 PDT 2021.
Topic: crawldata
UNT Web
by Internet Archive
web

eye 197,015

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-101.us.archive.org:UNT from Thu Jan 15 09:35:53 PST 2009 to Fri Jan 18 22:32:24 PST 2013.
Topic: crawldata
Data crawled by University of North Texas on behalf of University of North Texas from Tue Sep 16 16:49:16 PDT 2008 to Tue Sep 16 18:29:23 PDT 2008
Topic: crawldata
UNT Web
by Internet Archive
web

eye 180,102

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-101.us.archive.org:UNT from Thu Mar 5 07:41:11 PST 2009 to Fri Jan 18 20:50:12 PST 2013.
Topic: crawldata
Internet Archive crawldata from End of Term 2020, captured by crawl806.us.archive.org:eot20 from Thu Mar 18 00:01:04 PDT 2021 to Wed Mar 17 17:30:12 PDT 2021.
Topic: crawldata
End of Term 2012 Web Crawls
web

eye 389,010

favorite 0

comment 0

Internet Archive crawldata from End of Term 2012 Collaborative, captured by wbgrp-crawl013.us.archive.org:EOT-2012 from Wed Feb 13 20:24:01 PST 2013 to Wed Feb 13 13:19:31 PST 2013.
Topic: crawldata
UNT Web
by Internet Archive
web

eye 192,820

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-101.us.archive.org:UNT from Thu Jan 1 08:38:13 PST 2009 to Fri Mar 8 10:22:05 PST 2013.
Topic: crawldata
Internet Archive crawldata from End of Term 2020, captured by crawl808.us.archive.org:eot20 from Thu Mar 18 00:01:04 PDT 2021 to Wed Mar 17 17:25:42 PDT 2021.
Topic: crawldata
Data crawled by University of North Texas on behalf of University of North Texas from Thu Sep 18 01:55:35 PDT 2008 to Fri Sep 19 22:43:42 PDT 2008
Topic: crawldata
Internet Archive crawldata from End of Term 2020, captured by crawl814.us.archive.org:eot20 from Thu Mar 18 00:01:04 PDT 2021 to Wed Mar 17 17:28:14 PDT 2021.
Topic: crawldata
Internet Archive crawldata from End of Term 2020, captured by crawl812.us.archive.org:eot20 from Thu Mar 18 00:01:04 PDT 2021 to Wed Mar 17 17:30:49 PDT 2021.
Topic: crawldata
Internet Archive crawldata from End of Term 2020, captured by crawl866.us.archive.org:eot20 from Thu Mar 18 00:01:04 PDT 2021 to Wed Mar 17 17:22:21 PDT 2021.
Topic: crawldata
Internet Archive crawldata from End of Term 2020, captured by crawl809.us.archive.org:eot20 from Thu Mar 18 00:01:04 PDT 2021 to Wed Mar 17 17:27:58 PDT 2021.
Topic: crawldata
Internet Archive crawldata from End of Term 2020, captured by crawl811.us.archive.org:eot20 from Thu Mar 18 00:01:04 PDT 2021 to Wed Mar 17 17:23:14 PDT 2021.
Topic: crawldata
Internet Archive crawldata from End of Term 2020, captured by crawl807.us.archive.org:eot20 from Thu Mar 18 00:01:04 PDT 2021 to Wed Mar 17 17:22:05 PDT 2021.
Topic: crawldata
Data crawled by University of North Texas on behalf of University of North Texas from Wed Mar 09 23:04:17 PDT 2005 to Thu Mar 10 06:47:44 PDT 2005
Topic: crawldata
Internet Archive crawldata from End of Term 2020, captured by crawl865.us.archive.org:eot20 from Thu Mar 18 00:01:05 PDT 2021 to Wed Mar 17 17:24:11 PDT 2021.
Topic: crawldata
Data crawled by University of North Texas on behalf of University of North Texas from Tue Sep 16 07:55:54 PDT 2008 to Tue Sep 16 09:05:30 PDT 2008
Topic: crawldata
UNT Web
by Internet Archive
web

eye 146,535

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-101.us.archive.org:UNT from to Sat Jan 19 01:59:08 PST 2013.
Topic: crawldata
End of Term 2008 California Digital Library Crawl
by Internet Archive
web

eye 104,203

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-102.us.archive.org:CDL from Wed Jan 14 01:54:27 PST 2009 to Fri Jan 4 22:08:15 PST 2013.
Topic: crawldata
End of Term 2008 California Digital Library Crawl
by Internet Archive
web

eye 134,624

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-102.us.archive.org:CDL from Wed Jan 14 01:52:54 PST 2009 to Thu Jan 3 19:07:21 PST 2013.
Topic: crawldata
End of Term 2020 UNT Crawls
by University of North Texas Libraries
web

eye 2,969

favorite 0

comment 0

For End of Term Harvests 2020. WARCs with content from www.cdc.gov/coronavirus just prior to the inauguration, January 20, 2021.
Topics: eot2020, crawldata
End of Term 2020 UNT Crawls
by University of North Texas Libraries
web

eye 2,983

favorite 0

comment 0

For End of Term Harvests 2020. Includes seeds that had a -2 (HTTP connect failed) status code when crawling with Heritrix. Crawled with Brozzler.
Topics: eot2020, crawldata
Data crawled by California Digital Library on behalf of Internet Archive from Tue May 12 06:20:51 PDT 2009 to Tue May 12 22:08:48 PDT 2009
Topic: crawldata
Internet Archive crawldata from End of Term 2020, captured by crawl865.us.archive.org:eot20 from Mon Mar 22 16:03:30 PDT 2021 to Mon Mar 22 10:24:51 PDT 2021.
Topic: crawldata
UNT Web
by Internet Archive
web

eye 135,574

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-101.us.archive.org:UNT from to Sat Jan 19 01:03:44 PST 2013.
Topic: crawldata
UNT Web
by Internet Archive
web

eye 127,916

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-101.us.archive.org:UNT from to Sat Jan 19 00:46:44 PST 2013.
Topic: crawldata
End of Term 2008 California Digital Library Crawl
by Internet Archive
web

eye 96,570

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-102.us.archive.org:CDL from Wed Jan 14 01:52:38 PST 2009 to Sun Jan 6 01:10:58 PST 2013.
Topic: crawldata
End of Term 2008 California Digital Library Crawl
by Internet Archive
web

eye 111,838

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-102.us.archive.org:CDL from Wed Jan 14 01:52:23 PST 2009 to Sun Jan 6 08:53:13 PST 2013.
Topic: crawldata
UNT Web
by Internet Archive
web

eye 90,935

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-101.us.archive.org:UNT from to Fri Jan 18 22:28:56 PST 2013.
Topic: crawldata
Data crawled by University of North Texas on behalf of University of North Texas from Sun Sep 21 23:37:45 PDT 2008 to Mon Sep 22 01:10:14 PDT 2008
Topic: crawldata
Internet Archive crawldata from End of Term 2020, captured by crawl807.us.archive.org:eot20 from Thu Mar 18 00:14:33 PDT 2021 to Wed Mar 17 17:31:44 PDT 2021.
Topic: crawldata
End of Term 2012 Web Crawls
web

eye 206,461

favorite 0

comment 0

Internet Archive crawldata from End of Term 2012 Collaborative, captured by wbgrp-crawl002.us.archive.org:EOT12 from Thu Sep 13 07:37:58 PDT 2012 to Fri Sep 14 17:54:58 PDT 2012.
Topic: crawldata
Internet Archive crawldata from End of Term 2020, captured by crawl866.us.archive.org:eot20 from Thu Mar 18 00:12:20 PDT 2021 to Wed Mar 17 17:30:43 PDT 2021.
Topic: crawldata
End of Term 2008 California Digital Library Crawl
by Internet Archive
web

eye 117,545

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-102.us.archive.org:CDL from Wed Jan 14 01:52:21 PST 2009 to Sat Jan 5 05:24:43 PST 2013.
Topic: crawldata
End of Term 2008 California Digital Library Crawl
by Internet Archive
web

eye 119,173

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-102.us.archive.org:CDL from Wed Jan 14 07:54:57 PST 2009 to Sat Jan 5 04:11:06 PST 2013.
Topic: crawldata
UNT Web
by Internet Archive
web

eye 103,254

favorite 0

comment 0

Internet Archive crawldata uploaded by selenium-101.us.archive.org:UNT from Sat Jan 10 21:45:05 PST 2009 to Sat Jan 19 00:05:06 PST 2013.
Topic: crawldata
Internet Archive crawldata from End of Term 2020, captured by crawl865.us.archive.org:eot20 from Thu Mar 18 00:15:23 PDT 2021 to Wed Mar 17 17:33:00 PDT 2021.
Topic: crawldata
End of Term 2012 Web Crawls
web

eye 16,822

favorite 0

comment 0

Internet Archive crawldata from End of Term 2012 Collaborative, captured by wbgrp-crawl013.us.archive.org:EOT-2012 from Thu Feb 28 13:16:59 PST 2013 to Thu Feb 28 05:56:27 PST 2013.
Topic: crawldata