Ecosyste.ms: Repos
An open API service providing repository metadata for many open source software ecosystems.
GitHub / my8100 / scrapyd-cluster-on-heroku
Set up free and scalable Scrapyd cluster for distributed web-crawling with just a few clicks. DEMO :point_right:
JSON API: https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/my8100%2Fscrapyd-cluster-on-heroku
Stars: 122
Forks: 94
Open Issues: 5
License: gpl-3.0
Language: Python
Repo Size: 236 KB
Dependencies:
9
Created: about 5 years ago
Updated: 2 months ago
Last pushed: about 4 years ago
Last synced: 18 days ago
Topics: cluster, heroku, logparser, python, scrapy, scrapyd, scrapydweb, web-crawling, web-scraping
Files
Dependencies
- logparser >=0.8.2
- pip >=19.1
- scrapy *
- scrapy-redis *
- scrapyd >=1.2.1
- pip >=19.1
- psycopg2 >=2.7.7
- pymysql >=0.9.3
- scrapydweb >=1.4.0