Ecosyste.ms: Repos

An open API service providing repository metadata for many open source software ecosystems.

Package Usage: pypi: frontera

A scalable frontier for web crawlers
20 versions
Latest release: about 5 years ago
2 dependent packages
606 downloads last month

View more package details: https://packages.ecosyste.ms/registries/pypi.org/packages/frontera

View more repository details: https://repos.ecosyste.ms/hosts/GitHub/repositories/scrapinghub%2Ffrontera

Dependent Repos 13

plafl/tutorials

==0.3.0.post.dev2 blog/hn_scraper/requirements.txt

Size: 92.8 KB - Last synced: 9 months ago - Pushed: about 9 years ago

singularity014/FlaskSQLalchemy

This repo contains projects related to Fask

==0.8.1 requirements.txt

Size: 46.9 KB - Last synced: about 1 year ago - Pushed: over 1 year ago

scrapinghub/tutorials

==0.3.0.post.dev2 blog/hn_scraper/requirements.txt

Size: 8.79 KB - Last synced: 20 days ago - Pushed: 20 days ago

Alexoner/web-crawlers

Web crawler framework based on scrapy with useful pipelines and middlewares

==0.4.2 requirements.txt

Size: 351 KB - Last synced: 8 months ago - Pushed: almost 8 years ago

loitv1689/frontera_template

>=0.7.1 requirements.txt

Size: 458 KB - Last synced: over 1 year ago

scrapinghub/aduana

Frontera backend to guide a crawl using PageRank, HITS or other ranking algorithms based on the link structure of the web graph, even when making big crawls (one billion pages).

* examples/keywords/requirements.txt
* examples/locations/requirements.txt
* requirements.txt

Size: 11.4 MB - Last synced: 20 days ago - Pushed: 20 days ago

grammy-jiang/frontera-seedloader-mongodb

A seed load from MongoDB for Frontera

* requirements.txt
* setup.py

Size: 43.9 KB - Last synced: 2 months ago - Pushed: over 6 years ago

di3goacosta/tutorials Fork of scrapinghub/tutorials

==0.3.0.post.dev2 blog/hn_scraper/requirements.txt

Size: 12.7 KB - Last synced: about 1 year ago - Pushed: over 1 year ago

nxwi/illacceptanything Fork of illacceptanything/illacceptanything

The project where literally anything* goes.

* data/requirements.txt

Size: 1.33 GB - Last synced: 7 months ago - Pushed: over 1 year ago

scrapinghub/hubstorage-frontera

Hubstorage crawl frontier backend for Frontera

* requirements.txt
* setup.py

Size: 11.7 KB - Last synced: 11 days ago - Pushed: about 7 years ago

TeamHG-Memex/onetera

A single-process Frontera-based crawler managed by Kafka queue

* requirements.txt

Size: 139 KB - Last synced: about 1 year ago - Pushed: over 8 years ago

illacceptanything/illacceptanything

The project where literally anything* goes.

* data/requirements.txt

Size: 1.47 GB - Last synced: 10 days ago - Pushed: 10 days ago

kalessin/martindev

Development Environment Dockerfile with web scraping tools, and machine and deep learning tools

==0.7.1 requirements.txt

Size: 42 KB - Last synced: about 1 year ago - Pushed: over 1 year ago

claudiouzelac/distributed-frontera Fork of marcolin/distributed-frontera

A distributed extension for the Frontera web crawling framework

* requirements.txt
* setup.py

Size: 448 KB - Last synced: 30 days ago - Pushed: almost 9 years ago

middlechild/tutorials Fork of scrapinghub/tutorials

==0.3.0.post.dev2 blog/hn_scraper/requirements.txt

Size: 165 KB - Last synced: about 1 year ago - Pushed: over 8 years ago

pratamayuzar/tutorials Fork of scrapinghub/tutorials

==0.3.0.post.dev2 blog/hn_scraper/requirements.txt

Size: 165 KB - Last synced: 10 months ago - Pushed: over 8 years ago

zanachka/onetera Fork of TeamHG-Memex/onetera

A single-process Frontera-based crawler managed by Kafka queue

* requirements.txt

Size: 139 KB - Last synced: about 1 year ago - Pushed: over 8 years ago

vuchau/onetera Fork of TeamHG-Memex/onetera

A single-process Frontera-based crawler managed by Kafka queue

* requirements.txt

Size: 139 KB - Last synced: about 1 year ago - Pushed: over 8 years ago

ntchambers/illacceptanything

The project where literally anything* goes

* requirements.txt

Last synced: over 1 year ago

optionalg/tutorials Fork of scrapinghub/tutorials

==0.3.0.post.dev2 blog/hn_scraper/requirements.txt

Size: 165 KB - Last synced: about 1 year ago - Pushed: over 8 years ago

starrify/distributed-frontera Fork of marcolin/distributed-frontera

A distributed extension for the Frontera web crawling framework

* requirements.txt
* setup.py

Size: 273 KB - Last synced: about 1 year ago - Pushed: over 8 years ago

abaelhe/ScrapyFronteraDistributed

* requirements.txt
* setup.py

Size: 230 KB - Last synced: about 1 year ago - Pushed: over 7 years ago

007gzs/test

* requirements.txt

Size: 1.6 MB - Last synced: 4 months ago - Pushed: 4 months ago

AshBT/distributed-frontera Fork of marcolin/distributed-frontera

A distributed extension for the Frontera web crawling framework

* requirements.txt
* setup.py

Size: 270 KB - Last synced: about 1 year ago - Pushed: over 8 years ago

rtvt123/distributed-frontera Fork of marcolin/distributed-frontera

A distributed extension for the Frontera web crawling framework

* requirements.txt
* setup.py

Size: 270 KB - Last synced: 10 months ago - Pushed: over 8 years ago

jaisanas/distributed-frontera Fork of marcolin/distributed-frontera

A distributed extension for the Frontera web crawling framework

* requirements.txt
* setup.py

Size: 270 KB - Last synced: 9 months ago - Pushed: over 8 years ago

plafl/aduana Fork of scrapinghub/aduana

Frontera backend to guide a crawl using PageRank, HITS or other ranking algorithms based on the link structure of the web graph, even when making big crawls (one billion pages).