GitHub topics: web-scraper | Ecosyste.ms: Repos

pinkpixel-dev/web-scout-mcp

A powerful MCP server extension providing web search and content extraction capabilities. Integrates DuckDuckGo search functionality and URL content extraction into your MCP environment, enabling AI assistants to search the web and extract webpage content programmatically.

Language: JavaScript - Size: 354 KB - Last synced at: about 2 hours ago - Pushed at: about 3 hours ago - Stars: 1 - Forks: 1

juancarlospaco/faster-than-requests

Faster requests on Python 3

Language: Nim - Size: 20.4 MB - Last synced at: about 7 hours ago - Pushed at: about 8 hours ago - Stars: 1,118 - Forks: 91

thaikolja/avherald-scraper

A Python script that extracts the data of avherald.com, which provides commercial flight incident data, into an orderly SQLite DB.

Language: Python - Size: 60.5 KB - Last synced at: about 9 hours ago - Pushed at: about 11 hours ago - Stars: 2 - Forks: 0

sfkbstnc/bbc-rss-scraper

Simple RSS scraper for BBC News feeds using Python.

Language: Python - Size: 22.5 KB - Last synced at: about 14 hours ago - Pushed at: 2 days ago - Stars: 1 - Forks: 0

duyet/awesome-web-scraper

A collection of awesome web scaper, crawler.

Size: 48.8 KB - Last synced at: about 8 hours ago - Pushed at: about 1 year ago - Stars: 273 - Forks: 46

canHazUsername/4chan-Image-Downloader

Automated image downloader for 4chan threads using Playwright and a remote-controlled Chromium browser.

Language: Python - Size: 6.84 KB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 0 - Forks: 0

getmaxun/maxun

🔥 Open Source No Code Web Data Extraction Platform • Turn Websites To APIs & Spreadsheets With No-Code Robots In Minutes 🔥

Language: TypeScript - Size: 4.26 MB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 12,533 - Forks: 973

ryanfobel/ontario-grid-data

Hourly data on the Ontario electricity grid

Language: Jupyter Notebook - Size: 19.6 GB - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 1 - Forks: 1

ew3ezz/Facebook-Checker-APIV3

This Python application is used to verify the status of Facebook accounts, determining whether they are alive, in checkpoint or dead.

Language: Python - Size: 31.3 KB - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 0 - Forks: 0

Scorgep/Facebook-Checker-API

This Python tool is a powerful Facebook account verification tool used to check Facebook profiles and save checked accounts to .txt file.

Size: 1000 Bytes - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 0 - Forks: 0

kr1shnasomani/WebScrub

Python code which extracts the html content, converts it to clean text and pre-processes the text

Language: Python - Size: 563 KB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 2 - Forks: 0

0x676e67/rquest-util

Common utilities for rquest

Language: Rust - Size: 146 KB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 12 - Forks: 6

Cazintosz/GitHub-Account-Creator-Bot

GitHub Account Creator is a Python tool that enables automatic account creation on GitHub. Automatically generating your credentials, the script quickly creates a new account.

Language: Python - Size: 25.4 KB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 0 - Forks: 0

biglocalnews/usc-crime-reports-scraper

A GitHub Action workflow for automating the collection of crime and fire logs posted by the University of Southern California's Department of Public Safety.

Language: Jupyter Notebook - Size: 244 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 1 - Forks: 1

supadata-ai/py

Official Python SDK for the Supadata API.

Language: Python - Size: 52.7 KB - Last synced at: 2 days ago - Pushed at: 4 days ago - Stars: 3 - Forks: 4

kcsoc/society-email-scrape

Scrapes Every Email Address of Every Society in Every University

Language: Python - Size: 920 KB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 24 - Forks: 4

Cazintosz/Facebook-Checker-APIV3

This Python application is used to verify the status of Facebook accounts, determining whether they are alive, in checkpoint or dead.

Language: Python - Size: 9.77 KB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 0 - Forks: 0

Cazintosz/Facebook-Checker-API

This Python tool is a powerful Facebook account verification tool used to check Facebook profiles and save checked accounts to .txt file.

Language: Python - Size: 16.6 KB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 0 - Forks: 0

BruceDone/awesome-crawler

A collection of awesome web crawler,spider in different languages

Size: 74.2 KB - Last synced at: 3 days ago - Pushed at: 11 months ago - Stars: 6,744 - Forks: 716

nipunb123/Facebook-Checker-APIV3

This-Python-application-is-used-to-verify-the-status-of-Facebook-accounts,-determining-whether-they-are-alive,-in-checkpoint-or-dead.

Language: JavaScript - Size: 4.88 KB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 0 - Forks: 0

Bernso/NovelReaderWeb

Website made in python that scrapes lightnovelpub.vip for the novel inputted and will create a page for each of the chapters inside of those novels, also it includes features such as text to speech, text opacity and a font selector. All of these settings will save on your device.

Language: Python - Size: 310 MB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 2 - Forks: 1

thomasm1/python_2018

TensorFlow2 ML; flask, django; IOT; AWS deployments

Language: Rich Text Format - Size: 333 MB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 2 - Forks: 2

scrapfly/python-scrapfly

Scrapfly Python SDK for headless browsers and proxy rotation

Language: Python - Size: 673 KB - Last synced at: 3 days ago - Pushed at: 11 days ago - Stars: 42 - Forks: 11

chadsr/marktplaats-scraper

Marktplaats.nl (Dutch Classifieds) Listing Scraper

Language: Python - Size: 283 KB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 4 - Forks: 1

itsOwen/CyberScraper-2077

A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

Language: Python - Size: 280 KB - Last synced at: 7 days ago - Pushed at: about 1 month ago - Stars: 1,678 - Forks: 148

omkarcloud/botasaurus-starter

🚀 OFFICIAL STARTER TEMPLATE FOR BOTASAURUS SCRAPING FRAMEWORK 🤖

Language: TypeScript - Size: 397 KB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 25 - Forks: 8

D4Vinci/Scrapling

🕷️ An undetectable, powerful, flexible, high-performance Python library to make Web Scraping Easy and Effortless as it should be!

Language: Python - Size: 1.82 MB - Last synced at: 6 days ago - Pushed at: 11 days ago - Stars: 2,969 - Forks: 189

jaebradley/basketball_reference_web_scraper

NBA Stats API via Basketball Reference

Language: HTML - Size: 19.7 MB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 488 - Forks: 115

iZackk26/PuraCompra

Language: JavaScript - Size: 90.2 MB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 0 - Forks: 1

gosom/google-maps-scraper

scrape data data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place

Language: Go - Size: 20.5 MB - Last synced at: 8 days ago - Pushed at: 8 days ago - Stars: 1,311 - Forks: 191

speedycleats/Convert-SiteMap-Pages

Scrapes sitemap URLs and converts webpages into clean, markdown-formatted .txt files - ideal for SEO audits, content archiving, or AI/ChatGPT ingestion workflows.

Language: Python - Size: 6.84 KB - Last synced at: 8 days ago - Pushed at: 8 days ago - Stars: 0 - Forks: 0

dipu-bd/lightnovel-crawler

Generate and download e-books from online sources.

Language: Python - Size: 32.3 MB - Last synced at: 5 days ago - Pushed at: 10 days ago - Stars: 1,684 - Forks: 329

Akalanka47000/scrapher-client

A React frontend which consumes the Scrapher API at - https://github.com/Akalanka47000/scrapher

Language: TypeScript - Size: 0 Bytes - Last synced at: 8 days ago - Pushed at: 8 days ago - Stars: 0 - Forks: 0

dinguschan-owo/Helios

Helios is an COMPLETELY UNBLOCKABLE proxy with tabs that can be static hosted, can be run locally, and is html css js only! This is (as far as i've found) the only true UNBLOCKABLE only HTML proxy that works with any blocking software! Plus its open sauce so you can take this code and build your own proxy! (⭐ PLEASE star if you fork! ⭐)

Language: HTML - Size: 1.43 MB - Last synced at: 9 days ago - Pushed at: 9 days ago - Stars: 132 - Forks: 138

SenZmaKi/Senpwai

A desktop app for tracking and batch downloading anime

Language: Python - Size: 94.7 MB - Last synced at: 6 days ago - Pushed at: about 1 month ago - Stars: 243 - Forks: 25

php-curl-class/php-curl-class

PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs

Language: PHP - Size: 2.71 MB - Last synced at: 4 days ago - Pushed at: about 1 month ago - Stars: 3,287 - Forks: 820

whydrip/Facebook-Checker-APIV3

This Python application is used to verify the status of Facebook accounts, determining whether they are alive, in checkpoint or dead.

Language: Python - Size: 36.1 KB - Last synced at: 10 days ago - Pushed at: 10 days ago - Stars: 0 - Forks: 0

nuni-neomu-areumdawo/Canvas-Scraper

Canvas is the digital learning environment with courses. A Canvas course may consist of announcements, documents, exercises, assessments etc. This program scrapes publicly accessible courses for public and personal enrichment.

Language: Python - Size: 25.4 KB - Last synced at: 10 days ago - Pushed at: 10 days ago - Stars: 1 - Forks: 0

vishnupriyan123/wedding-venues-data-pipeline

AI-powered wedding venue data pipeline — scraping, enrichment, and NLP-driven insights from Hitched UK. 🔄 Actively evolving – multi-sprint project in progress.

Language: Python - Size: 58.4 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 0 - Forks: 0

wikimedia/html-metadata

MetaData html scraper and parser for Node.js (supports Promises and callback style)

Language: JavaScript - Size: 513 KB - Last synced at: 11 days ago - Pushed at: 11 days ago - Stars: 172 - Forks: 43

0x676e67/rquest

An ergonomic Rust HTTP Client with TLS fingerprint

Language: Rust - Size: 4.41 MB - Last synced at: 11 days ago - Pushed at: 11 days ago - Stars: 293 - Forks: 44

snowkluster/TAP

Threat Analysis Platform - A Data collection and normalisation platform to keep with threat actors

Language: Jupyter Notebook - Size: 84.5 MB - Last synced at: 11 days ago - Pushed at: 11 days ago - Stars: 0 - Forks: 0

S4pakXrn/ebay_dailydeals_scraper

A professional Scrapy spider that scrapes eBay daily deals across all categories. Features robust retry logic, exports data to both CSV and Google Sheets, handles AJAX pagination, and supports dynamic proxy rotation for scalable web scraping.

Language: Python - Size: 9.77 KB - Last synced at: 11 days ago - Pushed at: 12 days ago - Stars: 0 - Forks: 0

Heiatmea/GitHub-Account-Creator-Bot

GitHub Account Creator is a Python tool that enables automatic account creation on GitHub. Automatically generating your credentials, the script quickly creates a new account.

Language: Python - Size: 6.84 KB - Last synced at: 12 days ago - Pushed at: 12 days ago - Stars: 0 - Forks: 0

Heiatmea/Facebook-Checker-API

This Python tool is a powerful Facebook account verification tool used to check Facebook profiles and save checked accounts to .txt file.

Language: Python - Size: 17.6 KB - Last synced at: 12 days ago - Pushed at: 12 days ago - Stars: 0 - Forks: 0

Heiatmea/Facebook-Checker-APIV3

This Python application is used to verify the status of Facebook accounts, determining whether they are alive, in checkpoint or dead.

Language: Python - Size: 9.77 KB - Last synced at: 12 days ago - Pushed at: 12 days ago - Stars: 0 - Forks: 0

alanindra/news-enricher

Extracts news article metadata (title, content, date, journalist, entities) from provided URLs.

Language: Python - Size: 31.3 KB - Last synced at: 13 days ago - Pushed at: 13 days ago - Stars: 0 - Forks: 0

lewisdonovan/google-news-scraper

Lightweight scraper for Google News

Language: TypeScript - Size: 895 KB - Last synced at: 11 days ago - Pushed at: about 1 month ago - Stars: 319 - Forks: 66

PhantomInsights/summarizer

A Reddit bot that summarizes news articles written in Spanish or English. It uses a custom built algorithm to rank words and sentences.

Language: Python - Size: 254 KB - Last synced at: 9 days ago - Pushed at: about 3 years ago - Stars: 274 - Forks: 30

PhantomInsights/reddit-bots

A collection of Reddit bots that I use to enhance the subreddits I manage.

Language: Python - Size: 61.5 KB - Last synced at: 12 days ago - Pushed at: over 3 years ago - Stars: 27 - Forks: 11

Jeico908/GitHub-Account-Creator-Bot

GitHub-Account-Creator-is-a-Python-tool-that-enables-automatic-account-creation-on-GitHub.-Automatically-generating-your-credentials,-the-script-quickly-creates-a-new-account.

Size: 0 Bytes - Last synced at: 14 days ago - Pushed at: 14 days ago - Stars: 0 - Forks: 0

GAMERBOY-cdy/Facebook-Checker-API

This-Python-tool-is-a-powerful-Facebook-account-verification-tool-used-to-check-Facebook-profiles-and-save-checked-accounts-to-.txt-file.

Size: 0 Bytes - Last synced at: 14 days ago - Pushed at: 14 days ago - Stars: 0 - Forks: 0

Blissful-collab/Facebook-Checker-APIV3

This-Python-application-is-used-to-verify-the-status-of-Facebook-accounts,-determining-whether-they-are-alive,-in-checkpoint-or-dead.

Size: 0 Bytes - Last synced at: 14 days ago - Pushed at: 14 days ago - Stars: 0 - Forks: 0

zakaria9448z/GitHub-Account-Creator-Bot

GitHub-Account-Creator-is-a-Python-tool-that-enables-automatic-account-creation-on-GitHub.-Automatically-generating-your-credentials,-the-script-quickly-creates-a-new-account.

Size: 0 Bytes - Last synced at: 14 days ago - Pushed at: 14 days ago - Stars: 0 - Forks: 0

adel-alaa-dp/ebay_dailydeals_scraper

A professional Scrapy spider that scrapes eBay daily deals across all categories. Features robust retry logic, exports data to both CSV and Google Sheets, handles AJAX pagination, and supports dynamic proxy rotation for scalable web scraping.

Language: Python - Size: 8.79 KB - Last synced at: 15 days ago - Pushed at: 15 days ago - Stars: 0 - Forks: 0

Decodo/trustpilot_python_scraper

Scrape Trustpilot utilising Decodo's Web Scraping API

Language: Python - Size: 24.4 KB - Last synced at: 11 days ago - Pushed at: 16 days ago - Stars: 8 - Forks: 0

vientorepublic/pal-crawl

국회입법예고(pal.assembly.go.kr)의 진행 중인 입법 예고 크롤러

Language: TypeScript - Size: 214 KB - Last synced at: 12 days ago - Pushed at: 12 days ago - Stars: 0 - Forks: 0

vientorepublic/melona

멜론 음원 서비스 스크래핑 API

Language: TypeScript - Size: 316 KB - Last synced at: 12 days ago - Pushed at: 12 days ago - Stars: 0 - Forks: 0

filesite-io/machete_hero

为Ta荐（TaJian.tv）工作的基于Hero的Node.js实现的去中心化爬虫程序，可抓取B站、抖音、快手、西瓜视频播放页、直播页的标题和封面图

Language: JavaScript - Size: 61.5 KB - Last synced at: 16 days ago - Pushed at: 16 days ago - Stars: 4 - Forks: 2

nuzulul/telegram-scraper

A simple Telegram channel scraper

Language: JavaScript - Size: 36.1 KB - Last synced at: 10 days ago - Pushed at: 7 months ago - Stars: 25 - Forks: 12

PritamSarbajna/dark-web-scraper

A dark web link and image scraper

Language: Python - Size: 52.7 KB - Last synced at: 15 days ago - Pushed at: over 1 year ago - Stars: 12 - Forks: 5

newFaizanAli/RetailRadar

RetailRadar is an AI-powered product research tool that scrapes data from Daraz, Amazon, and OLX to help you analyze market trends, monitor competitor pricing, and generate SEO-friendly product names, descriptions, tags, and ideal pricing — all to boost your sales and profits.

Language: JavaScript - Size: 171 MB - Last synced at: 17 days ago - Pushed at: 17 days ago - Stars: 0 - Forks: 0

luminati-io/LinkedIn-Scraper

Extract LinkedIn data with the #1 LinkedIn Scraper API, including profiles, job postings, company details, connections, and posts. Start your free trial now!

Language: Python - Size: 4.6 MB - Last synced at: 17 days ago - Pushed at: 17 days ago - Stars: 12 - Forks: 5

somashekhar223/Scraping-with-langchain

Scrape any table tagged html webpage

Language: Python - Size: 5.34 MB - Last synced at: 17 days ago - Pushed at: 17 days ago - Stars: 0 - Forks: 0

Metitmato/GitHub-Account-Creator-Bot

GitHub Account Creator is a Python tool that enables automatic account creation on GitHub. Automatically generating your credentials, the script quickly creates a new account.

Language: Python - Size: 14.6 KB - Last synced at: 17 days ago - Pushed at: 17 days ago - Stars: 0 - Forks: 0

Metitmato/Facebook-Checker-API

This Python tool is a powerful Facebook account verification tool used to check Facebook profiles and save checked accounts to .txt file.

Language: Python - Size: 16.6 KB - Last synced at: 17 days ago - Pushed at: 17 days ago - Stars: 0 - Forks: 0

Metitmato/Facebook-Checker-APIV3

This Python application is used to verify the status of Facebook accounts, determining whether they are alive, in checkpoint or dead.

Language: Python - Size: 9.77 KB - Last synced at: 17 days ago - Pushed at: 17 days ago - Stars: 0 - Forks: 0

crwlrsoft/crawler

Library for Rapid (Web) Crawler and Scraper Development

Language: PHP - Size: 1.17 MB - Last synced at: 17 days ago - Pushed at: 17 days ago - Stars: 360 - Forks: 13

milahu/opensubtitles-scraper

scrape subtitles from opensubtitles.org

Language: Python - Size: 1.57 MB - Last synced at: 13 days ago - Pushed at: 18 days ago - Stars: 33 - Forks: 2

TheWebSolver/scraper

A simple and straight-forward PHP package to scrape web content

Language: PHP - Size: 449 KB - Last synced at: 14 days ago - Pushed at: 14 days ago - Stars: 0 - Forks: 0

wearrrrr/HaiKei

HaiKei is an anime streaming website that uses the consumet API

Language: JavaScript - Size: 5.55 MB - Last synced at: 5 days ago - Pushed at: 27 days ago - Stars: 134 - Forks: 43

RoryQo/Canvas-Courses-Complete-Content-Downloader

Automatically download all Canvas course content—including embedded files, linked documents, assignments, modules, pages, and submissions. Runs for every course you have access to, past and present. Saves everything directly to your computer with each course organized into its own folder.

Language: Jupyter Notebook - Size: 19.5 KB - Last synced at: 18 days ago - Pushed at: 18 days ago - Stars: 1 - Forks: 0

networkdynamics/pytok

A web scraper for TikTok using Playwright

Language: Python - Size: 270 KB - Last synced at: 18 days ago - Pushed at: 18 days ago - Stars: 82 - Forks: 12

Decodo/Web-Scraping-API

Web Scraping API code examples for Python, PHP and Node.js

Language: JavaScript - Size: 99.6 KB - Last synced at: 19 days ago - Pushed at: 19 days ago - Stars: 23 - Forks: 8

ibrahimm7004/web-scraping-projects

A collection of web scraping projects I've done in the last 2 years.

Language: Jupyter Notebook - Size: 17.2 MB - Last synced at: 20 days ago - Pushed at: 20 days ago - Stars: 0 - Forks: 0

serpapi/public-roadmap

Public Roadmap for SerpApi, LLC (https://serpapi.com)

Size: 17.6 KB - Last synced at: 7 days ago - Pushed at: 5 months ago - Stars: 68 - Forks: 13

Drlordbasil/AURORA

AURORA (Artificial Unified Responsive Optimized Reasoning Agent) uses lobes and web research for RAG based memory and learning.

Language: Python - Size: 13.7 MB - Last synced at: 6 days ago - Pushed at: 6 months ago - Stars: 17 - Forks: 5

TrashMack/Facebook-Checker-API

This Python tool is a powerful Facebook account verification tool used to check Facebook profiles and save checked accounts to .txt file.

Language: Python - Size: 216 KB - Last synced at: 21 days ago - Pushed at: 21 days ago - Stars: 0 - Forks: 0

TrashMack/Facebook-Checker-APIV3

This Python application is used to verify the status of Facebook accounts, determining whether they are alive, in checkpoint or dead.

Language: Python - Size: 234 KB - Last synced at: 21 days ago - Pushed at: 21 days ago - Stars: 0 - Forks: 1

Bes-js/Pinterest-Scraper

🔴 Web Scraper Application for Pinterest, Made Using TypeScript-Based Selenium Web Driver, Allows You to Extract a Specified or Unlimited Amount of Data via the Pinterest URL You Specify.

Language: TypeScript - Size: 6 MB - Last synced at: 6 days ago - Pushed at: about 1 year ago - Stars: 14 - Forks: 1

MixDevCode/monoschinos-api 📦

Un web-scrapper cuya función es obtener información del sitio conocido como MonosChinos.

Language: TypeScript - Size: 23.4 KB - Last synced at: 17 days ago - Pushed at: 5 months ago - Stars: 3 - Forks: 0

oxylabs/how-to-scrape-amazon-prices

A code for extracting best-selling items, search results, and currently available deals from Amazon using Python and Oxylabs E-Commerce Scraper API.

Language: Python - Size: 71.3 KB - Last synced at: 22 days ago - Pushed at: about 1 month ago - Stars: 480 - Forks: 6

AvatarParzival/GDrive_Link_Finder_Extension

A Chrome extension that scans web pages and extracts all Google Drive file and folder links for quick access and management.

Language: JavaScript - Size: 214 KB - Last synced at: 23 days ago - Pushed at: 24 days ago - Stars: 0 - Forks: 0

TrashMack/GitHub-Account-Creator-Bot

GitHub Account Creator is a Python tool that enables automatic account creation on GitHub. Automatically generating your credentials, the script quickly creates a new account.

Language: Python - Size: 139 KB - Last synced at: 24 days ago - Pushed at: 24 days ago - Stars: 2 - Forks: 1

remcostoeten/whatsapp-online-status-tracker

Monitor a WhatsApp user’s online activity with comprehensive reports that include timestamps, duration online, percentage of time spent online, and visual data through charts and graphs. Track statistics and resume insights between sessions for a complete overview.

Language: HTML - Size: 111 KB - Last synced at: 17 days ago - Pushed at: 6 months ago - Stars: 13 - Forks: 3