Ecosyste.ms: Repos
An open API service providing repository metadata for many open source software ecosystems.
GitHub / karust / gogetcrawl
Extract web archive data using Wayback Machine and Common Crawl
JSON API: https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karust%2Fgogetcrawl
Stars: 128
Forks: 15
Open Issues: 1
License: mit
Language: Go
Repo Size: 53.7 KB
Dependencies:
38
Created: almost 5 years ago
Updated: 28 days ago
Last pushed: almost 1 year ago
Last synced: 27 days ago
Topics: commoncrawl, concurrency, crawler, golang, wayback-machine, webarchive
Files
Loading...
Readme
Loading...
Dependencies
Dockerfile
docker
go.mod
go
- github.com/andybalholm/brotli v1.0.5
- github.com/corpix/uarand v0.2.0
- github.com/inconshreveable/mousetrap v1.1.0
- github.com/json-iterator/go v1.1.12
- github.com/klauspost/compress v1.16.5
- github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
- github.com/modern-go/reflect2 v1.0.2
- github.com/slyrz/warc v0.0.0-20150806225202-a50edd19b690
- github.com/spf13/cobra v1.7.0
- github.com/spf13/pflag v1.0.5
- github.com/valyala/bytebufferpool v1.0.0
- github.com/valyala/fasthttp v1.47.0
go.sum
go
- github.com/andybalholm/brotli v1.0.5
- github.com/corpix/uarand v0.2.0
- github.com/cpuguy83/go-md2man/v2 v2.0.2
- github.com/davecgh/go-spew v1.1.0
- github.com/davecgh/go-spew v1.1.1
- github.com/google/gofuzz v1.0.0
- github.com/inconshreveable/mousetrap v1.1.0
- github.com/json-iterator/go v1.1.12
- github.com/klauspost/compress v1.16.5
- github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421
- github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
- github.com/modern-go/reflect2 v1.0.2
- github.com/pmezard/go-difflib v1.0.0
- github.com/russross/blackfriday/v2 v2.1.0
- github.com/slyrz/warc v0.0.0-20150806225202-a50edd19b690
- github.com/spf13/cobra v1.7.0
- github.com/spf13/pflag v1.0.5
- github.com/stretchr/objx v0.1.0
- github.com/stretchr/testify v1.3.0
- github.com/stretchr/testify v1.7.1
- github.com/valyala/bytebufferpool v1.0.0
- github.com/valyala/fasthttp v1.47.0
- gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405
- gopkg.in/yaml.v3 v3.0.1