Ecosyste.ms: Repos

An open API service providing repository metadata for many open source software ecosystems.

GitHub topics: webarchive

Repositories

cipher387/quickcacheandarchivesearch

Quick Cache and Archive search buttons

Language: JavaScript - Size: 85 KB - Last synced: 3 minutes ago - Pushed: about 1 hour ago - Stars: 33 - Forks: 4

toimik/WarcProtocol

Parser for WARC (aka WebArchive) files

Language: C# - Size: 180 KB - Last synced: about 22 hours ago - Pushed: about 23 hours ago - Stars: 8 - Forks: 3

rumca-js/RSS-Link-Database

Bookmarked archived links

Size: 214 MB - Last synced: about 17 hours ago - Pushed: 1 day ago - Stars: 8 - Forks: 0

karust/gogetcrawl

Extract web archive data using Wayback Machine and Common Crawl

Language: Go - Size: 53.7 KB - Last synced: 3 days ago - Pushed: 11 months ago - Stars: 128 - Forks: 15

chatnoir-eu/chatnoir-resiliparse

A robust web archive analytics toolkit

Language: Cython - Size: 1.87 MB - Last synced: 3 days ago - Pushed: 12 days ago - Stars: 43 - Forks: 8

helgeho/ArchiveSpark

An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed at Internet Archive.

Language: Scala - Size: 1.15 MB - Last synced: 3 days ago - Pushed: about 1 month ago - Stars: 141 - Forks: 19

WebarchivCZ/Seeder

Seeder - Czech webarchive curating tool and public site

Language: Python - Size: 11.3 MB - Last synced: 8 days ago - Pushed: 8 days ago - Stars: 15 - Forks: 2

hyponet/webpage-packer

saves webpages as archive files.

Language: Go - Size: 1.75 MB - Last synced: 8 days ago - Pushed: 8 days ago - Stars: 0 - Forks: 0

sul-dlss-deprecated/WASMetadataExtractor 📦

[DEPRECATED] Extract metadata from web archiving ARC and WARC files; used by was_robot_suite

Language: Java - Size: 87.9 MB - Last synced: 25 days ago - Pushed: almost 2 years ago - Stars: 0 - Forks: 3

N0taN3rd/node-warc

Parse And Create Web ARChive (WARC) files with node.js

Language: JavaScript - Size: 7.99 MB - Last synced: 26 days ago - Pushed: over 1 year ago - Stars: 91 - Forks: 23

theorm/webarchive-page-downloader

Get archive history of a page and download pages from web.archive.org

Language: TypeScript - Size: 8.79 KB - Last synced: 27 days ago - Pushed: over 3 years ago - Stars: 1 - Forks: 0

vegetableman/vandal

Navigator for Web Archive

Language: JavaScript - Size: 128 MB - Last synced: 3 months ago - Pushed: 6 months ago - Stars: 149 - Forks: 5

mathis2001/WebHackUrls

Simple python OSINT tool for urls recon thanks to the waybackmachine.

Language: Python - Size: 44.9 KB - Last synced: 2 months ago - Pushed: 11 months ago - Stars: 37 - Forks: 6

ticky/webarchive

📑 Rust utilities for working with Apple's Web Archive file format

Language: Rust - Size: 506 KB - Last synced: 27 days ago - Pushed: about 2 years ago - Stars: 6 - Forks: 0

minch-dev/DownTheMoon Fork of downthemall/downthemall-legacy

A continuation of legacy XUL version of DownThemAll! ✔️preserves web.archive.org timestamps, ✔️advanced filters for remote directory tree mirroring, ✔️UI is tweaked for better UX

Language: JavaScript - Size: 13 MB - Last synced: 4 months ago - Pushed: 4 months ago - Stars: 5 - Forks: 0

rcarmo/python-webarchive

Create WebKit/Safari .webarchive files on any platform

Language: Python - Size: 7.81 KB - Last synced: 10 days ago - Pushed: over 4 years ago - Stars: 44 - Forks: 4

maxmmueller/404-to-Archive-Redirector

Greasemonkey script that redirects from a 404 page to the Wayback Machine.

Language: JavaScript - Size: 15.6 KB - Last synced: 3 months ago - Pushed: 3 months ago - Stars: 0 - Forks: 0

rumca-js/RSS-Link-Database-2023

link archive for year 2023

Language: HTML - Size: 413 MB - Last synced: 4 months ago - Pushed: 4 months ago - Stars: 2 - Forks: 2

jasonmtroos/ccwarcs

R package to provide access to Common Crawl WARC files via Amazon Web Services

Language: R - Size: 566 KB - Last synced: 5 months ago - Pushed: over 4 years ago - Stars: 1 - Forks: 0

Mixnode/mixnode-warcreader-php

Read Web ARChive (WARC) files in PHP.

Language: PHP - Size: 7.81 KB - Last synced: 16 days ago - Pushed: about 7 years ago - Stars: 21 - Forks: 3

gitdev-bash/webArchiver

A archiving utility with an interface for web servers.

Language: Python - Size: 63.5 KB - Last synced: 6 months ago - Pushed: almost 3 years ago - Stars: 1 - Forks: 0

Fooftilly/RSS_archiver

Download and archive RSS feeds to Wayback Machine. Save a list of archived feed in locad db.

Language: Python - Size: 30.3 KB - Last synced: 7 months ago - Pushed: 7 months ago - Stars: 0 - Forks: 0

mijho/crawl-log2xml

Parse a Heritrix crawl.log into an XML sitemap

Language: TypeScript - Size: 104 KB - Last synced: 7 months ago - Pushed: 7 months ago - Stars: 1 - Forks: 0

JanMeritus/WebBEAT

WebBEAT website data extractor

Language: Shell - Size: 43 KB - Last synced: 5 months ago - Pushed: 5 months ago - Stars: 0 - Forks: 2

Aplikace slouží jako automatizované řešení pro identifikaci a popis mrtvých webů. Následně je ukládá do vlastní databáze a zpřístupňuje kurátorům, kteří s informacemi v ní dále nakládají, interpretují je a obsah klasifikují.

Language: PHP - Size: 238 KB - Last synced: about 2 months ago - Pushed: 6 months ago - Stars: 2 - Forks: 0

MozillaCZ/forum-archiv 📦

Přesměrování

Language: HTML - Size: 150 KB - Last synced: 9 months ago - Pushed: almost 2 years ago - Stars: 0 - Forks: 1

gonejack/webarchive-to-singlefile

This command line converts .webarchive file to resources embed .html file

Language: Go - Size: 21.5 KB - Last synced: 10 months ago - Pushed: over 1 year ago - Stars: 7 - Forks: 1

gonejack/html-to-webarchive

This command line converts .html file to Safari's .webarchive file.

Language: Go - Size: 77.1 KB - Last synced: 10 months ago - Pushed: almost 2 years ago - Stars: 0 - Forks: 0

nlnwa/docker-chrome-headless 📦

Language: Shell - Size: 12.7 KB - Last synced: 28 days ago - Pushed: about 6 years ago - Stars: 6 - Forks: 1

DavidCThames/MobileMink-WebService

Web Service to provide memento data to the Mobile Memento app

Language: C - Size: 25.8 MB - Last synced: 12 months ago - Pushed: over 7 years ago - Stars: 3 - Forks: 1

HRN-Projects/common_crawl_with_scrapy

Parsing Huge Web Archive files from Common Crawl data index to fetch any required domain's data concurrently with Python and Scrapy.

Language: Python - Size: 23.9 MB - Last synced: 9 months ago - Pushed: almost 3 years ago - Stars: 4 - Forks: 5

rumca-js/RSS-Link-Database-2022

Link archive for year 2022

Size: 155 MB - Last synced: 11 months ago - Pushed: 11 months ago - Stars: 1 - Forks: 0

ukwa/ukwa-manage

Shepherding our web archives from crawl to access.

Language: Jupyter Notebook - Size: 122 MB - Last synced: 7 months ago - Pushed: 7 months ago - Stars: 10 - Forks: 5

nlnwa/veidemann-harvester 📦

Language: Java - Size: 5.54 MB - Last synced: 28 days ago - Pushed: about 3 years ago - Stars: 6 - Forks: 0

dbmdz/rosetta-openwayback-vpp

Configurable OpenWayback VPP for Rosetta digital preservation system.

Language: Java - Size: 395 KB - Last synced: 9 months ago - Pushed: about 1 year ago - Stars: 0 - Forks: 1

eUgEntOptIc44/TinyWeatherForecastGermanyArchiver Fork of tinyweatherforecastgermanygroup/TinyWeatherForecastGermanyArchiver

submit urls.txt to web archive using GitHub Action

Language: Python - Size: 116 KB - Last synced: 28 days ago - Pushed: 28 days ago - Stars: 0 - Forks: 0

maddsua/harextract 📦

CLI based web archive extractor with lightning-fast base64 decoder (written in C)

Language: C++ - Size: 2.2 MB - Last synced: about 1 year ago - Pushed: over 1 year ago - Stars: 0 - Forks: 0

Sicos1977/WebArchiveExtractor

A .NET Standard 2.0 library to extract a Safari web archive to a folder

Language: C# - Size: 270 KB - Last synced: 10 days ago - Pushed: about 3 years ago - Stars: 5 - Forks: 1

mhucka/devilfish

A utility for simultaneously creating full-page PDF snapshots and web archives of web pages in DEVONthink Pro.

Language: AppleScript - Size: 253 KB - Last synced: 13 days ago - Pushed: almost 4 years ago - Stars: 20 - Forks: 1

raviraa/htmltoebook

Convert html web pages to readable ebook

Language: Go - Size: 219 KB - Last synced: 5 months ago - Pushed: almost 2 years ago - Stars: 1 - Forks: 0

WebarchivCZ/WA-KAT

Catalogization tool for the czech webarchive.

Language: JavaScript - Size: 13.6 MB - Last synced: about 1 month ago - Pushed: 10 months ago - Stars: 2 - Forks: 0

RimeCoOfficial/Arya Fork of Caffeine-Devotee/arya

A snapshot of Arya.Rime.co

Language: HTML - Size: 878 KB - Last synced: about 1 year ago - Pushed: almost 4 years ago - Stars: 0 - Forks: 0

webarchive 55 warc 11 webarchiving 9 archive 8 web-archiving 6 wayback-machine 6 python 4 rss-feed 4 rss-archive 4 rss 4 spark 3 links 3 web-archives 3 warc-files 3 memento 3 safari 3 internet-archive 2 internet-archiving 2 web 2 data-mining 2 hadoop 2 r 2 html 2 golang 2 crawler 2 python3 2 archiving 2 warc-format 2 wayback 2 link-aggregator 2 link-aggregation 2 warc-record 2 webarchives 2 common-crawl-with-python 1 common-crawl-scrapy 1 common-crawl-python 1 common-crawl-data 1 common-crawl-with-scrapy 1 parse-common-crawl 1 scrapy 1 web-crawling 1 web-scraping 1 webarchive-data-scraping 1 cdx 1 common-crawl 1 webservice 1 mobile 1 web-firefox 1 redirect 1 deprecated 1 link-rot 1 monitoring-tool 1 extinct-websites 1 sitemap-xml 1 sitemap-generator 1 sitemap 1 heritrix3 1 heritrix 1 deno 1 crawl 1 timetravel 1 archival-replay 1 replay-tests 1 testing 1 bachelor-thesis 1 content-extraction 1 html-content-extraction 1 machine-learning 1 machine-learning-algorithms 1 article-extractor 1 arweave 1 blockchain 1 bubing 1 crawling 1 scraping 1 cdxj 1 hdfs 1 veidemann 1 openwayback 1 rosetta 1 aiohttp 1 cli 1 har 1 httparchive 1 macos 1 devonthink 1 pdf 1 ebook 1 epub 1 mobi 1 readability 1 analyzer 1 rust-crate 1 recon 1 pentesting 1 osint 1 bugbounty 1 firefox-addon 1 chrome-extension 1 history 1