my8100/scrapydweb

Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. Docs 文档 :point_right:

Language: Python - Size: 3.05 MB - Last synced at: 7 days ago - Pushed at: 5 months ago - Stars: 3,317 - Forks: 579

my8100/logparser

A tool for parsing Scrapy log files periodically and incrementally, extending the HTTP JSON API of Scrapyd.

Language: Python - Size: 172 KB - Last synced at: about 1 month ago - Pushed at: 7 months ago - Stars: 92 - Forks: 25

my8100/files

Docs and files for ScrapydWeb, Scrapyd, Scrapy, and other projects

Size: 16.7 MB - Last synced at: 2 months ago - Pushed at: 5 months ago - Stars: 421 - Forks: 71

my8100/scrapyd-cluster-on-heroku

Set up free and scalable Scrapyd cluster for distributed web-crawling with just a few clicks. DEMO :point_right:

Language: Python - Size: 236 KB - Last synced at: about 2 months ago - Pushed at: over 5 years ago - Stars: 123 - Forks: 87

my8100/notes

Keep on reading

Language: Python - Size: 216 KB - Last synced at: 22 days ago - Pushed at: over 6 years ago - Stars: 6 - Forks: 2

my8100/scrapyd Fork of scrapy/scrapyd

[PR #326] Native support for basic auth :lock: `pip install -U git+https://github.com/scrapy/scrapyd.git`, then add `username = yourusername` and `password = yourpassword` in the scrapyd.conf file. DOCS :point_right:

Language: Python - Size: 412 KB - Last synced at: over 2 years ago - Pushed at: over 4 years ago - Stars: 10 - Forks: 4

my8100/scrapyd-cluster-on-heroku-scrapyd-app

How to set up Scrapyd cluster on Heroku

Language: Python - Size: 26.4 KB - Last synced at: 4 months ago - Pushed at: over 3 years ago - Stars: 9 - Forks: 29

my8100/awesome-web-scraping Fork of lorien/awesome-web-scraping

List of libraries, tools and APIs for web scraping and data processing.

Language: Makefile - Size: 240 KB - Last synced at: over 2 years ago - Pushed at: over 6 years ago - Stars: 2 - Forks: 1

my8100/awesome-crawler Fork of BruceDone/awesome-crawler

A collection of awesome web crawler,spider in different languages

Size: 45.9 KB - Last synced at: over 2 years ago - Pushed at: over 6 years ago - Stars: 3 - Forks: 1

my8100/scrapyd-CircleCI

Language: Python - Size: 297 KB - Last synced at: 4 months ago - Pushed at: over 4 years ago - Stars: 1 - Forks: 0

my8100/public-test

public-test

Language: Python - Size: 1.49 MB - Last synced at: 4 months ago - Pushed at: over 5 years ago - Stars: 1 - Forks: 1

my8100/scrapyd-cluster-on-heroku-scrapydweb-app-git

How to set up Scrapyd cluster on Heroku

Language: Python - Size: 36.1 KB - Last synced at: 26 days ago - Pushed at: almost 6 years ago - Stars: 1 - Forks: 21

my8100/awesome-python-cn Fork of jobbole/awesome-python-cn

Python资源大全中文版，包括：Web框架、网络爬虫、模板引擎、数据库、数据可视化、图片处理等，由伯乐在线持续更新。

Language: Makefile - Size: 495 KB - Last synced at: over 2 years ago - Pushed at: over 6 years ago - Stars: 5 - Forks: 3

my8100/awesome-scrapy Fork of AccordBox/awesome-scrapy

A curated list of awesome packages, articles, and other cool resources from the Scrapy community.

Size: 2.93 KB - Last synced at: over 2 years ago - Pushed at: over 7 years ago - Stars: 6 - Forks: 2

my8100/awesome-flask Fork of humiaozuzu/awesome-flask

A curated list of awesome Flask resources and plugins

Size: 237 KB - Last synced at: over 2 years ago - Pushed at: over 6 years ago - Stars: 1 - Forks: 0

my8100/scrapy-redis Fork of rmax/scrapy-redis

Redis-based components for Scrapy.

Language: Python - Size: 147 KB - Last synced at: over 2 years ago - Pushed at: over 6 years ago - Stars: 1 - Forks: 1

my8100/scrapyd-cluster-on-heroku-scrapydweb-app

How to set up Scrapyd cluster on Heroku

Language: Python - Size: 43.9 KB - Last synced at: about 2 months ago - Pushed at: almost 6 years ago - Stars: 0 - Forks: 3

my8100/temp

temp

Language: HTML - Size: 136 KB - Last synced at: 4 months ago - Pushed at: about 6 years ago - Stars: 0 - Forks: 1

my8100/scrapyd-cluster-on-heroku-scrapyd-app-basic-auth

scrapyd-cluster-on-heroku-scrapyd-app-basic-auth

Language: Python - Size: 23.4 KB - Last synced at: 30 days ago - Pushed at: about 6 years ago - Stars: 0 - Forks: 0

my8100/scrapyd-cluster-on-heroku-scrapyd-app-git

How to set up Scrapyd cluster on Heroku

Language: Python - Size: 24.4 KB - Last synced at: 4 months ago - Pushed at: about 6 years ago - Stars: 0 - Forks: 0

my8100/Python-Algorithms Fork of TheAlgorithms/Python

All Algorithms implemented in Python

Language: Python - Size: 7.02 MB - Last synced at: over 2 years ago - Pushed at: over 6 years ago - Stars: 0 - Forks: 0

my8100/queuelib Fork of scrapy/queuelib

Collection of persistent (disk-based) queues

Language: Python - Size: 38.1 KB - Last synced at: over 2 years ago - Pushed at: over 7 years ago - Stars: 0 - Forks: 0

my8100/awesome-python-applications Fork of mahmoud/awesome-python-applications

💿 Free software that works great, and also happens to be open-source Python.

Language: Python - Size: 425 KB - Last synced at: over 2 years ago - Pushed at: over 6 years ago - Stars: 1 - Forks: 1

my8100/awesome-python Fork of vinta/awesome-python

A curated list of awesome Python frameworks, libraries, software and resources

Language: Python - Size: 4.47 MB - Last synced at: over 2 years ago - Pushed at: over 6 years ago - Stars: 1 - Forks: 1