Ecosyste.ms: Repos

An open API service providing repository metadata for many open source software ecosystems.

GitHub / biraj21 / web-wanderer

A multi-threaded web crawler written in Python, utilizing ThreadPoolExecutor and Playwright to efficiently crawl dynamically rendered web pages and download them.

JSON API: https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/biraj21%2Fweb-wanderer

Stars: 10
Forks: 1
Open Issues: 0

License: mit
Language: Python
Repo Size: 202 KB
Dependencies: 5

Created: 10 months ago
Updated: 23 days ago
Last pushed: 23 days ago
Last synced: 23 days ago

Topics: data-extraction, multithreading, python, web-crawler, webcrawler

Files
    Loading...
    Readme
    Loading...
    Dependencies
    Pipfile pypi
    Pipfile.lock pypi