Ecosyste.ms: Repos
An open API service providing repository metadata for many open source software ecosystems.
GitHub / aolle / romcrawler
A scrapy-based spider for automated search and web data extraction. Able to solve captcha codes through Optical character recognition (OCR) processes, create the required requests, control cookies and sessions.
JSON API: https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aolle%2Fromcrawler
Stars: 6
Forks: 3
Open Issues: 0
License: gpl-3.0
Language: Python
Repo Size: 17.6 KB
Dependencies:
19
Created: almost 9 years ago
Updated: over 1 year ago
Last pushed: almost 3 years ago
Last synced: about 1 year ago
Files
Dependencies
- PIL ==1.1.7
- Scrapy ==0.18.3
- Twisted ==13.1.0
- argparse ==1.2.1
- distribute ==0.6.24
- elementtree ==1.2.7
- lxml ==3.2.3
- pyOpenSSL ==0.13.1
- queuelib ==1.0
- requests ==2.0.1
- w3lib ==1.3
- wsgiref ==0.1.2
- zope.interface ==4.0.5
- OnLinuxsystem *
- libxml2-dev ==2.8.0
- libxslt1-dev ==1.1.26
- pytesserinsite-packagesdirectoryifyouusevirtualenvenvironment. *
- python-dev ==2.7.3
- tesseract-ocr ==3.02.01