Data

Tools

Indexes

Applications

Experiments

Repos

An open API service providing repository metadata for many open source software ecosystems.

GitHub / chandrasekharan98 / Multisite-Python-Crawler

An almost generic web crawler built using Scrapy and Python 3.7 to recursively crawl entire websites.

JSON API: http://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chandrasekharan98%2FMultisite-Python-Crawler

Stars: 16
Forks: 5
Open issues: 0

License: None
Language: Python
Size: 15.6 KB
Dependencies parsed at: Pending

Created at: over 6 years ago
Updated at: 9 months ago
Pushed at: about 3 years ago
Last synced at: 6 months ago

Topics: crawling-sites, python, python3, recursive-crawling, scrapy, scrapy-crawler, scrapy-demo, scrapy-spider, website-crawler

    Loading...