Ecosyste.ms: Repos
An open API service providing repository metadata for many open source software ecosystems.
GitHub / gosom / scrapemate
Golang Crawling and scraping framework
JSON API: https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gosom%2Fscrapemate
Stars: 58
Forks: 5
Open Issues: 2
License: mit
Language: Go
Repo Size: 107 KB
Dependencies:
475
Created: about 1 year ago
Updated: about 1 month ago
Last pushed: about 1 month ago
Last synced: about 1 month ago
Topics: crawler, go, go-framework, golang, scraper, spider, web-crawler, web-scraping
Files
Loading...
Readme
Loading...
Dependencies
- github.com/PuerkitoBio/goquery v1.8.1
- github.com/andybalholm/cascadia v1.3.1
- github.com/danwakefield/fnmatch v0.0.0-20160403171240-cbb64ac3d964
- github.com/go-playground/locales v0.14.0
- github.com/go-playground/universal-translator v0.18.0
- github.com/go-playground/validator/v10 v10.11.1
- github.com/go-stack/stack v1.8.1
- github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db
- github.com/google/uuid v1.3.0
- github.com/gosom/kit v0.0.0-20230309082109-543b32ac686a
- github.com/gosom/scrapemate v0.4.2
- github.com/leodido/go-urn v1.2.1
- github.com/mattn/go-colorable v0.1.12
- github.com/mattn/go-isatty v0.0.14
- github.com/playwright-community/playwright-go v0.2000.1
- github.com/rs/zerolog v1.28.0
- github.com/syndtr/goleveldb v1.0.0
- golang.org/x/crypto v0.3.0
- golang.org/x/net v0.7.0
- golang.org/x/sync v0.1.0
- golang.org/x/sys v0.5.0
- golang.org/x/text v0.7.0
- gopkg.in/square/go-jose.v2 v2.6.0
- github.com/PuerkitoBio/goquery v1.8.1
- github.com/andybalholm/cascadia v1.3.1
- github.com/danwakefield/fnmatch v0.0.0-20160403171240-cbb64ac3d964
- github.com/go-playground/locales v0.14.1
- github.com/go-playground/universal-translator v0.18.1
- github.com/go-playground/validator/v10 v10.12.0
- github.com/go-stack/stack v1.8.1
- github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db
- github.com/google/uuid v1.3.0
- github.com/gosom/kit v0.0.0-20230309082109-543b32ac686a
- github.com/gosom/scrapemate v0.0.0-00010101000000-000000000000
- github.com/leodido/go-urn v1.2.2
- github.com/mattn/go-colorable v0.1.12
- github.com/mattn/go-isatty v0.0.14
- github.com/playwright-community/playwright-go v0.2000.1
- github.com/rs/zerolog v1.28.0
- github.com/syndtr/goleveldb v1.0.0
- golang.org/x/crypto v0.7.0
- golang.org/x/net v0.9.0
- golang.org/x/sync v0.1.0
- golang.org/x/sys v0.7.0
- golang.org/x/text v0.9.0
- gopkg.in/square/go-jose.v2 v2.6.0
go.mod
go
- github.com/PuerkitoBio/goquery v1.8.1
- github.com/andybalholm/cascadia v1.3.1
- github.com/danwakefield/fnmatch v0.0.0-20160403171240-cbb64ac3d964
- github.com/davecgh/go-spew v1.1.1
- github.com/go-playground/locales v0.14.0
- github.com/go-playground/universal-translator v0.18.0
- github.com/go-playground/validator/v10 v10.11.1
- github.com/go-stack/stack v1.8.1
- github.com/golang/mock v1.6.0
- github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db
- github.com/gosom/kit v0.0.0-20230309082109-543b32ac686a
- github.com/leodido/go-urn v1.2.1
- github.com/mattn/go-colorable v0.1.12
- github.com/mattn/go-isatty v0.0.14
- github.com/playwright-community/playwright-go v0.2000.1
- github.com/pmezard/go-difflib v1.0.0
- github.com/rs/zerolog v1.28.0
- github.com/stretchr/testify v1.8.1
- github.com/syndtr/goleveldb v1.0.0
- golang.org/x/crypto v0.3.0
- golang.org/x/net v0.7.0
- golang.org/x/sync v0.1.0
- golang.org/x/sys v0.5.0
- golang.org/x/text v0.7.0
- gopkg.in/square/go-jose.v2 v2.6.0
- gopkg.in/yaml.v3 v3.0.1
.github/workflows/build.yml
actions
- actions/checkout v3 composite
- actions/setup-go v4 composite
- golangci/golangci-lint-action v3 composite
- github.com/PuerkitoBio/goquery v1.8.1
- github.com/andybalholm/cascadia v1.3.1
- github.com/danwakefield/fnmatch v0.0.0-20160403171240-cbb64ac3d964
- github.com/davecgh/go-spew v1.1.1
- github.com/go-stack/stack v1.8.1
- github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db
- github.com/google/uuid v1.3.0
- github.com/gosom/kit v0.0.0-20230309082109-543b32ac686a
- github.com/gosom/scrapemate v0.0.0-20230309082109-543b32ac686a
- github.com/mattn/go-colorable v0.1.12
- github.com/mattn/go-isatty v0.0.18
- github.com/playwright-community/playwright-go v0.2000.1
- github.com/pmezard/go-difflib v1.0.0
- github.com/rs/zerolog v1.28.0
- github.com/stretchr/testify v1.8.1
- github.com/syndtr/goleveldb v1.0.0
- golang.org/x/net v0.7.0
- golang.org/x/sys v0.6.0
- gopkg.in/square/go-jose.v2 v2.6.0
- gopkg.in/yaml.v3 v3.0.1
- github.com/PuerkitoBio/goquery v1.8.1
- github.com/andybalholm/cascadia v1.3.1
- github.com/coreos/go-systemd/v22 v22.3.3-0.20220203105225-a9a7ef127534
- github.com/danwakefield/fnmatch v0.0.0-20160403171240-cbb64ac3d964
- github.com/davecgh/go-spew v1.1.0
- github.com/davecgh/go-spew v1.1.1
- github.com/fsnotify/fsnotify v1.4.7
- github.com/go-stack/stack v1.8.1
- github.com/godbus/dbus/v5 v5.0.4
- github.com/golang/mock v1.6.0
- github.com/golang/protobuf v1.2.0
- github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db
- github.com/google/uuid v1.3.0
- github.com/gorilla/websocket v1.4.2
- github.com/gosom/kit v0.0.0-20230309082109-543b32ac686a
- github.com/h2non/filetype v1.1.1
- github.com/hpcloud/tail v1.0.0
- github.com/mattn/go-colorable v0.1.12
- github.com/mattn/go-isatty v0.0.14
- github.com/mattn/go-isatty v0.0.18
- github.com/onsi/ginkgo v1.6.0
- github.com/onsi/ginkgo v1.7.0
- github.com/onsi/gomega v1.4.3
- github.com/pkg/errors v0.9.1
- github.com/playwright-community/playwright-go v0.2000.1
- github.com/pmezard/go-difflib v1.0.0
- github.com/rs/xid v1.4.0
- github.com/rs/zerolog v1.28.0
- github.com/stretchr/objx v0.1.0
- github.com/stretchr/objx v0.4.0
- github.com/stretchr/objx v0.5.0
- github.com/stretchr/testify v1.7.0
- github.com/stretchr/testify v1.7.1
- github.com/stretchr/testify v1.8.0
- github.com/stretchr/testify v1.8.1
- github.com/syndtr/goleveldb v1.0.0
- github.com/yuin/goldmark v1.4.13
- golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2
- golang.org/x/crypto v0.0.0-20210921155107-089bfa567519
- golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4
- golang.org/x/net v0.0.0-20180906233101-161cd47e91fd
- golang.org/x/net v0.0.0-20190620200207-3b0461eec859
- golang.org/x/net v0.0.0-20210226172049-e18ecbb05110
- golang.org/x/net v0.0.0-20210916014120-12bc252f5db8
- golang.org/x/net v0.0.0-20220722155237-a158d28d115b
- golang.org/x/net v0.7.0
- golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
- golang.org/x/sync v0.0.0-20190423024810-112230192c58
- golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4
- golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e
- golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a
- golang.org/x/sys v0.0.0-20201119102817-f84b799fce68
- golang.org/x/sys v0.0.0-20210423082822-04245dca01da
- golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1
- golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c
- golang.org/x/sys v0.0.0-20210927094055-39ccf1dd6fa6
- golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a
- golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f
- golang.org/x/sys v0.5.0
- golang.org/x/sys v0.6.0
- golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1
- golang.org/x/term v0.0.0-20210927222741-03fcf44c2211
- golang.org/x/term v0.5.0
- golang.org/x/text v0.3.0
- golang.org/x/text v0.3.3
- golang.org/x/text v0.3.6
- golang.org/x/text v0.3.7
- golang.org/x/text v0.7.0
- golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e
- golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e
- golang.org/x/tools v0.1.12
- golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7
- gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405
- gopkg.in/fsnotify.v1 v1.4.7
- gopkg.in/square/go-jose.v2 v2.6.0
- gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7
- gopkg.in/yaml.v2 v2.2.1
- gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c
- gopkg.in/yaml.v3 v3.0.1
- github.com/PuerkitoBio/goquery v1.8.1
- github.com/andybalholm/cascadia v1.3.1
- github.com/coreos/go-systemd/v22 v22.3.3-0.20220203105225-a9a7ef127534
- github.com/danwakefield/fnmatch v0.0.0-20160403171240-cbb64ac3d964
- github.com/davecgh/go-spew v1.1.0
- github.com/davecgh/go-spew v1.1.1
- github.com/fsnotify/fsnotify v1.4.7
- github.com/go-playground/locales v0.14.1
- github.com/go-playground/universal-translator v0.18.1
- github.com/go-playground/validator/v10 v10.12.0
- github.com/go-stack/stack v1.8.1
- github.com/godbus/dbus/v5 v5.0.4
- github.com/golang/protobuf v1.2.0
- github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db
- github.com/google/uuid v1.3.0
- github.com/gorilla/websocket v1.4.2
- github.com/gosom/kit v0.0.0-20230309082109-543b32ac686a
- github.com/h2non/filetype v1.1.1
- github.com/hpcloud/tail v1.0.0
- github.com/leodido/go-urn v1.2.2
- github.com/mattn/go-colorable v0.1.12
- github.com/mattn/go-isatty v0.0.14
- github.com/onsi/ginkgo v1.6.0
- github.com/onsi/ginkgo v1.7.0
- github.com/onsi/gomega v1.4.3
- github.com/pkg/errors v0.9.1
- github.com/playwright-community/playwright-go v0.2000.1
- github.com/pmezard/go-difflib v1.0.0
- github.com/rs/xid v1.4.0
- github.com/rs/zerolog v1.28.0
- github.com/rwtodd/Go.Sed v0.0.0-20210816025313-55464686f9ef
- github.com/stretchr/objx v0.1.0
- github.com/stretchr/objx v0.4.0
- github.com/stretchr/objx v0.5.0
- github.com/stretchr/testify v1.7.0
- github.com/stretchr/testify v1.7.1
- github.com/stretchr/testify v1.8.0
- github.com/stretchr/testify v1.8.2
- github.com/syndtr/goleveldb v1.0.0
- github.com/yuin/goldmark v1.4.13
- golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2
- golang.org/x/crypto v0.0.0-20210921155107-089bfa567519
- golang.org/x/crypto v0.7.0
- golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4
- golang.org/x/net v0.0.0-20180906233101-161cd47e91fd
- golang.org/x/net v0.0.0-20190620200207-3b0461eec859
- golang.org/x/net v0.0.0-20210226172049-e18ecbb05110
- golang.org/x/net v0.0.0-20210916014120-12bc252f5db8
- golang.org/x/net v0.0.0-20220722155237-a158d28d115b
- golang.org/x/net v0.7.0
- golang.org/x/net v0.8.0
- golang.org/x/net v0.9.0
- golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
- golang.org/x/sync v0.0.0-20190423024810-112230192c58
- golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4
- golang.org/x/sync v0.1.0
- golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e
- golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a
- golang.org/x/sys v0.0.0-20201119102817-f84b799fce68
- golang.org/x/sys v0.0.0-20210423082822-04245dca01da
- golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1
- golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c
- golang.org/x/sys v0.0.0-20210927094055-39ccf1dd6fa6
- golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a
- golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f
- golang.org/x/sys v0.5.0
- golang.org/x/sys v0.6.0
- golang.org/x/sys v0.7.0
- golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1
- golang.org/x/term v0.0.0-20210927222741-03fcf44c2211
- golang.org/x/term v0.5.0
- golang.org/x/text v0.3.0
- golang.org/x/text v0.3.3
- golang.org/x/text v0.3.6
- golang.org/x/text v0.3.7
- golang.org/x/text v0.7.0
- golang.org/x/text v0.8.0
- golang.org/x/text v0.9.0
- golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e
- golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e
- golang.org/x/tools v0.1.12
- golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7
- gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405
- gopkg.in/fsnotify.v1 v1.4.7
- gopkg.in/square/go-jose.v2 v2.6.0
- gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7
- gopkg.in/yaml.v2 v2.2.1
- gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c
- gopkg.in/yaml.v3 v3.0.1
- github.com/PuerkitoBio/goquery v1.8.1
- github.com/andybalholm/cascadia v1.3.1
- github.com/coreos/go-systemd/v22 v22.3.3-0.20220203105225-a9a7ef127534
- github.com/creack/pty v1.1.9
- github.com/danwakefield/fnmatch v0.0.0-20160403171240-cbb64ac3d964
- github.com/davecgh/go-spew v1.1.1
- github.com/davecgh/go-spew v1.1.0
- github.com/fsnotify/fsnotify v1.4.7
- github.com/go-playground/assert/v2 v2.0.1
- github.com/go-playground/locales v0.14.0
- github.com/go-playground/universal-translator v0.18.0
- github.com/go-playground/validator/v10 v10.11.1
- github.com/go-stack/stack v1.8.1
- github.com/godbus/dbus/v5 v5.0.4
- github.com/golang/mock v1.6.0
- github.com/golang/protobuf v1.2.0
- github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db
- github.com/google/uuid v1.3.0
- github.com/gorilla/websocket v1.4.2
- github.com/gosom/kit v0.0.0-20230309082109-543b32ac686a
- github.com/gosom/scrapemate v0.4.2
- github.com/h2non/filetype v1.1.1
- github.com/hpcloud/tail v1.0.0
- github.com/kr/pretty v0.1.0
- github.com/kr/pretty v0.2.1
- github.com/kr/pretty v0.3.0
- github.com/kr/pty v1.1.1
- github.com/kr/text v0.2.0
- github.com/kr/text v0.1.0
- github.com/leodido/go-urn v1.2.1
- github.com/mattn/go-colorable v0.1.12
- github.com/mattn/go-isatty v0.0.14
- github.com/onsi/ginkgo v1.6.0
- github.com/onsi/ginkgo v1.7.0
- github.com/onsi/gomega v1.4.3
- github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e
- github.com/pkg/errors v0.9.1
- github.com/playwright-community/playwright-go v0.2000.1
- github.com/pmezard/go-difflib v1.0.0
- github.com/rogpeppe/go-internal v1.6.1
- github.com/rogpeppe/go-internal v1.8.0
- github.com/rs/xid v1.4.0
- github.com/rs/zerolog v1.28.0
- github.com/stretchr/objx v0.1.0
- github.com/stretchr/testify v1.6.1
- github.com/stretchr/testify v1.7.0
- github.com/stretchr/testify v1.8.1
- github.com/syndtr/goleveldb v1.0.0
- github.com/yuin/goldmark v1.4.13
- golang.org/x/crypto v0.0.0-20211215153901-e495a2d5b3d3
- golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2
- golang.org/x/crypto v0.0.0-20210921155107-089bfa567519
- golang.org/x/crypto v0.3.0
- golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4
- golang.org/x/net v0.0.0-20180906233101-161cd47e91fd
- golang.org/x/net v0.0.0-20220722155237-a158d28d115b
- golang.org/x/net v0.0.0-20190620200207-3b0461eec859
- golang.org/x/net v0.0.0-20210226172049-e18ecbb05110
- golang.org/x/net v0.0.0-20210916014120-12bc252f5db8
- golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2
- golang.org/x/net v0.7.0
- golang.org/x/sync v0.1.0
- golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
- golang.org/x/sync v0.0.0-20190423024810-112230192c58
- golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4
- golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a
- golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e
- golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a
- golang.org/x/sys v0.0.0-20201119102817-f84b799fce68
- golang.org/x/sys v0.0.0-20210423082822-04245dca01da
- golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1
- golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c
- golang.org/x/sys v0.0.0-20210806184541-e5e7981a1069
- golang.org/x/sys v0.0.0-20210927094055-39ccf1dd6fa6
- golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f
- golang.org/x/sys v0.5.0
- golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1
- golang.org/x/term v0.0.0-20210927222741-03fcf44c2211
- golang.org/x/term v0.5.0
- golang.org/x/text v0.3.0
- golang.org/x/text v0.3.3
- golang.org/x/text v0.3.6
- golang.org/x/text v0.3.7
- golang.org/x/text v0.7.0
- golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e
- golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e
- golang.org/x/tools v0.1.12
- golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7
- gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405
- gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127
- gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c
- gopkg.in/errgo.v2 v2.1.0
- gopkg.in/fsnotify.v1 v1.4.7
- gopkg.in/square/go-jose.v2 v2.6.0
- gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7
- gopkg.in/yaml.v2 v2.2.1
- gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c
- gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b
- gopkg.in/yaml.v3 v3.0.1