An open API service providing repository metadata for many open source software ecosystems.

GitHub topics: crawling-engine

flulemon/sneakpeek

Sneakpeek is a framework that helps to quickly and conviniently develop scrapers. It’s the best choice for scrapers that have some specific complex scraping logic that needs to be run on a constant basis

Language: Python - Size: 19.7 MB - Last synced at: 6 days ago - Pushed at: over 1 year ago - Stars: 37 - Forks: 0

arthur3486/born2crawl

A highly performant and versatile crawling engine, designed with scalability and extensibility in mind.

Language: Kotlin - Size: 624 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 8 - Forks: 0