Solihatun1/AI-Cursor-Scraping-Assistant

A powerful tool that leverages Cursor AI and MCP (Model Context Protocol) to easily generate web scrapers for various types of websites.

Language: Python - Size: 16.6 KB - Last synced at: about 5 hours ago - Pushed at: about 6 hours ago - Stars: 6 - Forks: 0

youmeat6678/Instagram-Hashtag-Scraper

A Python project for scraping Instagram posts based on specific hashtags. This tool uses Selenium for browser automation and BeautifulSoup for scraping content from Instagram. It's designed to fetch posts by hashtag, with customizable options to automate login, scrape content, and interact with Instagram posts.

Language: Python - Size: 35.2 KB - Last synced at: about 5 hours ago - Pushed at: about 6 hours ago - Stars: 5 - Forks: 0

asokolsky/pycrawl

Web crawling using python

Language: Python - Size: 28.3 KB - Last synced at: about 6 hours ago - Pushed at: about 7 hours ago - Stars: 0 - Forks: 0

sazzadhossainmilon/got-scraping-client

got-scraping-client is a lightweight and efficient tool for web scraping tasks using the popular Got HTTP client. It simplifies data extraction from web pages by providing a straightforward API and built-in support for handling common challenges like pagination and rate limiting.

Language: TypeScript - Size: 24.4 KB - Last synced at: about 8 hours ago - Pushed at: about 9 hours ago - Stars: 0 - Forks: 0

Huolix/web-scraping-beautifulsoup

A Python project that scrapes product data using BeautifulSoup

Language: Jupyter Notebook - Size: 9.77 KB - Last synced at: about 11 hours ago - Pushed at: about 12 hours ago - Stars: 0 - Forks: 0

scrapy/itemadapter

Common interface for data container classes

Language: Python - Size: 272 KB - Last synced at: about 13 hours ago - Pushed at: about 14 hours ago - Stars: 68 - Forks: 12

code-caffeine-shekhawat4u/EMP012-TASK1-26062025

Intern Project 2025: Develop Python automation scripts for various communication platforms and data extraction tasks

Size: 2.93 KB - Last synced at: about 17 hours ago - Pushed at: about 18 hours ago - Stars: 0 - Forks: 0

scrapy/scrapy-bench

A CLI for benchmarking Scrapy.

Language: Python - Size: 8.75 MB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 31 - Forks: 15

hericlibong/worldPressPhotoGalery

web application designed for photojournalism enthusiasts with Scrapy and Django

Language: Python - Size: 50.4 MB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 1 - Forks: 0

wuzhy1ng/BlockchainSpider

A toolkit for blockchain data collection

Language: Python - Size: 13.7 MB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 147 - Forks: 28

amelia05-spec/crowdfunding-real-estate-scrapy

This project is a powerful and extensible scrapy-based crawler designed to extract and aggregate data from multiple real estate crowdfunding platforms. Ideal for investors, analysts and researchers interested in tracking investment opportunities, platform performance and market trends

Language: Python - Size: 31.3 KB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 1 - Forks: 0

ZH1995/ScrapyHub

一个基于Scrapy框架的多网站数据采集系统，目前包含微博热搜榜单爬虫，通过定时爬取存储热搜数据，可用于数据分析和趋势研究。

Language: Python - Size: 89.8 KB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 3 - Forks: 0

murtaja89/public-proxies

🌐 Public Proxy List (Updated Every 2 Hours)

Language: HTML - Size: 1.12 MB - Last synced at: 1 day ago - Pushed at: 2 days ago - Stars: 2 - Forks: 0

TikHub/TikHub-API-Python-SDK

High-performance asynchronous Douyin(抖音) TikTok Xiaohongshu(小红书) Kuaishou(快手) Weibo(微博) Instagram YouTube(油管) Twitter(X) Captcha Solver(验证码解决器) Temp Mail(临时邮箱) API(接口).

Language: Python - Size: 2.05 MB - Last synced at: about 9 hours ago - Pushed at: 7 months ago - Stars: 470 - Forks: 55

xpetz/Netflix-Clone

# Netflix-CloneThis project replicates the Netflix homepage using only HTML and CSS, focusing on responsive design. Future updates will include JavaScript features to enhance interactivity. 🛠️🌐

Language: HTML - Size: 5.42 MB - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 0 - Forks: 0

honzajavorek/czap

Scraping czap.cz data so you can filter available psychotherapists by any criteria you wish

Language: Python - Size: 1.5 MB - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 2 - Forks: 0

humayun-ahmad/web-scraping

A comprehensive collection of beginner to intermediate web scraping scripts using Python, featuring real-world examples and educational resources.

Language: Jupyter Notebook - Size: 357 KB - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 3 - Forks: 1

salimt/Transfermarkt-ETL-and-LIVE-Scores

asyncIO, Github Actions, GCP, dbt, Terraform, Docker

Language: Python - Size: 109 MB - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 2 - Forks: 0

schmwong/APAC-McDelivery-Menu-Logger

Automatically scrapes McDelivery menu data and records it for future visualisation projects

Language: Python - Size: 41.5 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 6 - Forks: 2

Erkmik/best-python-html-parsers

The top Python HTML parsers for web scraping, including Beautiful Soup, lxml, PyQuery, Scrapy, and more.

Size: 6.84 KB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 0 - Forks: 0

vlad1kudelko/2023.11.03-scrapy

Заказ по парсингу

Language: Python - Size: 0 Bytes - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 0 - Forks: 0

NguyenDa18/Portland-Jail-Data-Crawler

Scraper used for recording changes to Portland jail database

Language: Jupyter Notebook - Size: 40.4 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 5 - Forks: 0

sevenjunebaby/Post-Clustering

scrapy & cluster data

Language: Jupyter Notebook - Size: 2.39 MB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 6 - Forks: 0

alltheplaces/alltheplaces

A set of spiders and scrapers to extract location information from places that post their location on the internet.

Language: Python - Size: 32 MB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 698 - Forks: 233

iaoongin/GachaClock

卡池倒计时。支持查看崩坏 星穹铁道，绝区零，鸣潮卡池信息

Language: Python - Size: 54.2 MB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 1 - Forks: 0

irineulucas/SentimenTA

SentimenTA is a sentiment analysis tool designed to analyze and interpret emotions and opinions from textual data. It utilizes natural language processing techniques to provide insights into the sentiment behind the text.

Size: 1000 Bytes - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 0 - Forks: 0

hellock/icrawler

A multi-thread crawler framework with many builtin image crawlers provided.

Language: Python - Size: 282 KB - Last synced at: 1 day ago - Pushed at: 4 months ago - Stars: 889 - Forks: 180

joaopauloaramuni/python

Repo Python

Language: HTML - Size: 151 MB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 44 - Forks: 1

fuunshi/ShareSansarDataScrape

Daily auto scrapping of Share price form Share Sansar

Language: Python - Size: 6.82 MB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 3 - Forks: 0

sachaarbonel/scrapy.dart

Scrapy, a fast high-level web crawling & scraping framework for dart and Flutter

Language: Dart - Size: 557 KB - Last synced at: 5 days ago - Pushed at: over 4 years ago - Stars: 51 - Forks: 7

Erik172/stylos-scrapers

Advanced Fashion Scraper | Stylos Ecosystem. Intelligent extraction of products, prices, and images from fashion retailers like Zara and Mango. Part of the Stylos ecosystem, an AI-powered platform for trend analysis and personalized style recommendations. Built with Scrapy, Selenium, and MongoDB.

Language: Python - Size: 21.4 MB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 1 - Forks: 0

eracle/linkedin

Linkedin Scraper using Selenium Web Driver, Chromium headless, Docker and Scrapy

Language: Python - Size: 686 KB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 912 - Forks: 139

TeamHG-Memex/scrapy-rotating-proxies

use multiple proxies with Scrapy

Language: Python - Size: 44.9 KB - Last synced at: 5 days ago - Pushed at: about 3 years ago - Stars: 762 - Forks: 161

AccordBox/awesome-scrapy

A curated list of awesome packages, articles, and other cool resources from the Scrapy community.

Size: 51.8 KB - Last synced at: 2 days ago - Pushed at: over 2 years ago - Stars: 551 - Forks: 64

open-news-brasil/open-news

📰 Coleta automatizada das últimas notícias das cidades brasileiras

Language: Python - Size: 2.54 MB - Last synced at: 3 days ago - Pushed at: 3 months ago - Stars: 19 - Forks: 2

Boris-code/feapder

🚀🚀🚀feapder is an easy to use, powerful crawler framework | feapder是一款上手简单，功能强大的Python爬虫框架。内置AirSpider、Spider、TaskSpider、BatchSpider四种爬虫解决不同场景的需求。且支持断点续爬、监控报警、浏览器渲染、海量数据去重等功能。更有功能强大的爬虫管理系统feaplat为其提供方便的部署及调度

Language: Python - Size: 1.48 MB - Last synced at: 5 days ago - Pushed at: 3 months ago - Stars: 3,287 - Forks: 511

shengchenyang/AyugeSpiderTools

使 scrapy 开发不用在意 item，pipeline，middleware 等通用场景下模块的编写，解放开发者的双手。

Language: Python - Size: 26 MB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 92 - Forks: 15

zehengl/scrapy-darksky-api

A scrapy app to crawl weather data from Dark Sky Api

Language: Python - Size: 6.85 MB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 7 - Forks: 0

zehengl/scrapy-indeed-company-reviews

A scrapy app to crawl company reviews from Indeed

Language: Python - Size: 63.2 MB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 4 - Forks: 1

GRDRarda/search-a-sorted

Search A Sorted offers a quick way to find elements in a sorted list, whether ascending or descending. With functions like `search_a_sorted_ascending()` and `search_a_sorted_descending()`, it efficiently locates your target in no time! 🐙💻

Language: C - Size: 4.88 KB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 0 - Forks: 0

ConlinH/aio-scrapy

Implement scrapy with asyncio

Language: Python - Size: 960 KB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 65 - Forks: 10

Erik172/nomads-scraper

Language: Python - Size: 18.6 KB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 0 - Forks: 0

my8100/scrapydweb

Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. Docs 文档 :point_right:

Language: Python - Size: 3.05 MB - Last synced at: 6 days ago - Pushed at: 4 months ago - Stars: 3,304 - Forks: 578

chama-45426/hub-api

AI模型接口汇总管理

Language: Go - Size: 31.3 KB - Last synced at: 6 days ago - Pushed at: 7 days ago - Stars: 0 - Forks: 1

whoisjayd/IMDb-TMDb-Movie-Scraper

A powerful and concurrent Scrapy project to scrape comprehensive movie and TV series data from IMDb and TMDb. Features basic and advanced spiders, data enrichment via TMDb API,

Language: Python - Size: 46.9 KB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 1 - Forks: 1

trexarm/Jobsite-Scraper-and-Analyzer

Jobsite Scraper and Analyzer extracts job listings from theprotocol.it for data analysis. Built with Scrapy, Selenium, and Flask. 🚀💻

Language: Python - Size: 13.2 MB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 0 - Forks: 0

chyroc/WechatSogou

基于搜狗微信搜索的微信公众号爬虫接口

Language: Python - Size: 4.12 MB - Last synced at: 7 days ago - Pushed at: over 1 year ago - Stars: 6,085 - Forks: 1,716

Yahlawat/RAG-Financial-News-Agent

A lightweight RAG system that scrapes, embeds, and indexes financial news to generate context-aware stock answers using a vector store and an LLM.

Language: Python - Size: 97.2 MB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 0 - Forks: 0

rl1987/trickster.dev

Language: HTML - Size: 482 MB - Last synced at: 8 days ago - Pushed at: 8 days ago - Stars: 5 - Forks: 5

rheaacharya77/foodmandu-scraper

Foodmandu Scraper extracts restaurant information such as restaurant URLs, images,names,addresses,and cuisines.

Language: Python - Size: 1.1 MB - Last synced at: 8 days ago - Pushed at: 8 days ago - Stars: 0 - Forks: 0

zhangslob/awesome_crawl

腾讯新闻、知乎话题、微博粉丝，Tumblr爬虫、斗鱼弹幕、妹子图爬虫、分布式设计等

Language: Python - Size: 13.8 MB - Last synced at: 5 days ago - Pushed at: 23 days ago - Stars: 294 - Forks: 109

casangi/casadocs

Common Astronomy Software Applications Documentation

Language: Python - Size: 47.6 MB - Last synced at: 8 days ago - Pushed at: 9 days ago - Stars: 12 - Forks: 10

danieladdisonorg/Jobsite-Scraper-and-Analyzer

A comprehensive web scraping and data analysis platform that extracts job listings from theprotocol.it to analyze technology demand trends for developers in the Polish job market.

Language: Python - Size: 13.5 MB - Last synced at: 9 days ago - Pushed at: 9 days ago - Stars: 0 - Forks: 0

Xcrap-Cloud/transformer

Xcrap Transformer is the data transformation package extracted from the Web Scraping framework Xcrap.

Language: TypeScript - Size: 136 KB - Last synced at: 9 days ago - Pushed at: 9 days ago - Stars: 2 - Forks: 0

Gerapy/Gerapy

Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js

Language: Python - Size: 36.6 MB - Last synced at: 10 days ago - Pushed at: 8 months ago - Stars: 3,457 - Forks: 647

rmax/scrapy-redis

Redis-based components for Scrapy.

Language: Python - Size: 228 KB - Last synced at: 5 days ago - Pushed at: 12 months ago - Stars: 5,609 - Forks: 1,584

builker-col/bogota-apartments

"Bogotá Apartments" es un proyecto de código abierto que recopila y analiza datos del mercado inmobiliario de Bogotá, específicamente en el ámbito de los apartamentos. para los data science

Language: Jupyter Notebook - Size: 144 MB - Last synced at: about 11 hours ago - Pushed at: 5 days ago - Stars: 10 - Forks: 6

george-gca/ai_papers_scrapper

Download papers pdfs and other info from main AI conferences

Language: Python - Size: 200 KB - Last synced at: 11 days ago - Pushed at: 11 days ago - Stars: 29 - Forks: 6

gridaco/figma-archives

Figma Files Scraper for Research & Studies

Language: Python - Size: 103 MB - Last synced at: 5 days ago - Pushed at: 11 days ago - Stars: 23 - Forks: 3

pi-2r/devoxxfr2025-tock-studio-IA-Gen

Projet issu du codelab Devoxx France 2025 “À la recherche du RAG perdu” : atelier de 3h pour apprendre à créer un chatbot IA Générative autonome, local et sans Internet, basé uniquement sur des frameworks open source

Language: Python - Size: 57.2 MB - Last synced at: 3 days ago - Pushed at: about 1 month ago - Stars: 8 - Forks: 8

scrapy/flake8-scrapy

A Flake8 plugin to catch common issues on Scrapy spiders

Language: Python - Size: 35.2 KB - Last synced at: 12 days ago - Pushed at: 12 days ago - Stars: 19 - Forks: 4

heitornolla/WebScraping-MercadoLivre

This project performs Web Scraping on Mercado Livre's website, scraping data related to 5 string basses. It then cleans this data and turns it into a SQL database to be loaded on Streamlit. This can be easily adapted for other items.

Language: Python - Size: 213 KB - Last synced at: 12 days ago - Pushed at: 12 days ago - Stars: 1 - Forks: 0

ScrapeEmAll/Telegram-Scraper

A powerful Python script that allows you to scrape messages and media from Telegram channels using the Telethon library. Features include real-time continuous scraping, media downloading, and data export capabilities.

Language: Python - Size: 18.9 MB - Last synced at: 12 days ago - Pushed at: 12 days ago - Stars: 40 - Forks: 0

jxlil/scrapy-impersonate

Scrapy download handler that can impersonate browser' TLS signatures or JA3 fingerprints.

Language: Python - Size: 47.9 KB - Last synced at: 11 days ago - Pushed at: about 2 months ago - Stars: 171 - Forks: 17

rukshar69/scraping-projects

Scraping job lists from careerjet using Scrapy and Cohere LLM AI

Language: Python - Size: 556 KB - Last synced at: 8 days ago - Pushed at: 8 days ago - Stars: 0 - Forks: 0

vifreefly/kimuraframework

Kimurai is a modern web scraping framework written in Ruby which works out of box with Headless Chromium/Firefox, PhantomJS, or simple HTTP requests and allows to scrape and interact with JavaScript rendered websites

Language: Ruby - Size: 193 KB - Last synced at: 6 days ago - Pushed at: about 1 year ago - Stars: 1,012 - Forks: 158

crawlab-team/crawlab

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

Language: Go - Size: 23.9 MB - Last synced at: 13 days ago - Pushed at: 13 days ago - Stars: 11,769 - Forks: 1,837

andros21/pgrank

pgrank - cpp app for computing pagerank

Language: C++ - Size: 662 KB - Last synced at: 14 days ago - Pushed at: 14 days ago - Stars: 0 - Forks: 0

SpiderClub/haipproxy

:sparkling_heart: High available distributed ip proxy pool, powerd by Scrapy and Redis

Language: Python - Size: 1.16 MB - Last synced at: 8 days ago - Pushed at: over 2 years ago - Stars: 5,480 - Forks: 911

AKrekhovetskyi/tech-trend-stat

Analyzer of technology statistics based on job descriptions.

Language: Jupyter Notebook - Size: 372 KB - Last synced at: 16 days ago - Pushed at: 16 days ago - Stars: 1 - Forks: 0

public-law/typed-soup

A type-safe wrapper around BeautifulSoup v4.

Language: Python - Size: 728 KB - Last synced at: 4 days ago - Pushed at: 10 days ago - Stars: 0 - Forks: 0

rogedev/bookscraper

scrapy project

Language: Python - Size: 36.1 KB - Last synced at: 16 days ago - Pushed at: 16 days ago - Stars: 0 - Forks: 0

clemfromspace/scrapy-selenium

Scrapy middleware to handle javascript pages using selenium

Language: Python - Size: 29.3 KB - Last synced at: 6 days ago - Pushed at: 12 months ago - Stars: 947 - Forks: 361

eliasdabbas/advertools

advertools - online marketing productivity and analysis tools

Language: Python - Size: 23.8 MB - Last synced at: 17 days ago - Pushed at: 17 days ago - Stars: 1,239 - Forks: 229

nit-in/nptel

Download NPTEL videos using scrapy

Language: Python - Size: 38.1 KB - Last synced at: 17 days ago - Pushed at: 17 days ago - Stars: 1 - Forks: 0

nit-in/download_ncert_books

download NCERT books using scrapy

Language: Python - Size: 18.1 MB - Last synced at: 17 days ago - Pushed at: 17 days ago - Stars: 7 - Forks: 3

NathanWorkman/seeker

Seeker - another job board aggregator.

Language: Python - Size: 1.83 MB - Last synced at: about 16 hours ago - Pushed at: almost 5 years ago - Stars: 29 - Forks: 7

nit-in/pib

Download articles by the Press Information Bureau, India follow the instructions or download by month from the releases section

Language: Python - Size: 176 MB - Last synced at: 17 days ago - Pushed at: 17 days ago - Stars: 4 - Forks: 2

AlexMathew/scrapple

A framework for creating semi-automatic web content extractors

Language: Python - Size: 1.15 MB - Last synced at: 16 days ago - Pushed at: 16 days ago - Stars: 502 - Forks: 41

honzajavorek/czech-political-parties

Tracking changes in Czech political parties

Language: Python - Size: 1.26 MB - Last synced at: 17 days ago - Pushed at: 17 days ago - Stars: 4 - Forks: 1

my8100/logparser

A tool for parsing Scrapy log files periodically and incrementally, extending the HTTP JSON API of Scrapyd.

Language: Python - Size: 172 KB - Last synced at: 4 days ago - Pushed at: 6 months ago - Stars: 92 - Forks: 25

SoyEdwinCabrera/web_scraping

A Python project for extracting and processing data from websites using web scraping techniques.

Language: Python - Size: 24.4 KB - Last synced at: 17 days ago - Pushed at: 18 days ago - Stars: 0 - Forks: 0

Maestro-111/search-engine

search engine with Django, Scrapy, MongoDB and Elasticsearch

Language: Python - Size: 5.37 MB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 1 - Forks: 0

stefanofusai/scrapy-influxdb-exporter

Export Scrapy spider stats to InfluxDB

Language: Python - Size: 161 KB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 6 - Forks: 0

potlitel/scraping

Demonstrative example of using the Scrapy library for web scraping.

Language: Python - Size: 12.7 KB - Last synced at: 18 days ago - Pushed at: 18 days ago - Stars: 0 - Forks: 0

rydzze/CyberHolmes

Final Year Project | Cyber Threat Intelligence (CTI) Web-based Application

Language: TypeScript - Size: 5.91 MB - Last synced at: 8 days ago - Pushed at: 18 days ago - Stars: 0 - Forks: 0

Insutanto/scrapy-distributed

A series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy.

Language: Python - Size: 58.6 KB - Last synced at: 19 days ago - Pushed at: 19 days ago - Stars: 58 - Forks: 11

ialejandro/crowdfunding-real-estate-scrapy

This project is a powerful and extensible scrapy-based crawler designed to extract and aggregate data from multiple real estate crowdfunding platforms. Ideal for investors, analysts and researchers interested in tracking investment opportunities, platform performance and market trends

Language: Python - Size: 70.3 KB - Last synced at: 19 days ago - Pushed at: 19 days ago - Stars: 1 - Forks: 0

jonbakerfish/TweetScraper

TweetScraper is a simple crawler/spider for Twitter Search without using API

Language: Python - Size: 58.6 KB - Last synced at: 8 days ago - Pushed at: about 4 years ago - Stars: 1,040 - Forks: 314

asandineni/task_ncrb

A Python project to scrape and clean data from the NCRB website.

Language: Python - Size: 4.88 KB - Last synced at: 20 days ago - Pushed at: 20 days ago - Stars: 0 - Forks: 0

tadmonyayafranklin/codealpha_task_1

This is basic Network Sniffer in Python

Language: Python - Size: 6.84 KB - Last synced at: 20 days ago - Pushed at: 20 days ago - Stars: 0 - Forks: 0

juancarlospaco/faster-than-requests

Faster requests on Python 3

Language: Nim - Size: 20.4 MB - Last synced at: 21 days ago - Pushed at: 21 days ago - Stars: 1,120 - Forks: 91

mouday/spider-admin-pro

spider-admin-pro 一个集爬虫Scrapy+Scrapyd爬虫项目查看 和 爬虫任务定时调度的可视化管理工具，SpiderAdmin的升级版

Language: Python - Size: 2.82 MB - Last synced at: 17 days ago - Pushed at: 8 months ago - Stars: 596 - Forks: 85

nicksherron/proxi 📦

Proxy pool. Finds and checks proxies with rest api for querying results. Can find over 25k proxies in under 5 minutes.

Language: Go - Size: 1.09 MB - Last synced at: 5 days ago - Pushed at: about 5 years ago - Stars: 34 - Forks: 4

doroudi/imdb-crawler

imdb.com movies crawler in scrapy

Language: Python - Size: 8.79 KB - Last synced at: 7 days ago - Pushed at: over 5 years ago - Stars: 6 - Forks: 0

reycn/cubox-to-notion

A slight but fast synchronization tool for Notions users to utilize Cubox.

Language: Python - Size: 4.88 MB - Last synced at: 22 days ago - Pushed at: 22 days ago - Stars: 102 - Forks: 10

Randika00/ISM-WEB-Automation-Y23CP-Web

Web scraping refers to the extraction of data from a website. Be it a spreadsheet or an API.

Language: Python - Size: 859 KB - Last synced at: 23 days ago - Pushed at: 23 days ago - Stars: 1 - Forks: 0

adiyaan010205/Price-Pulse

PricePulse is a full-stack web application that automates product price tracking across popular e-commerce platforms like Amazon and eBay. It enables users to receive real-time alerts for price drops, monitor trends via analytics dashboards, and manage tracked items efficiently—all through a modern and responsive interface.

Language: Python - Size: 39.3 MB - Last synced at: 24 days ago - Pushed at: 24 days ago - Stars: 0 - Forks: 0

xuexingdong/databox

A databox setup with scrapy

Language: JavaScript - Size: 11.2 MB - Last synced at: 24 days ago - Pushed at: 24 days ago - Stars: 3 - Forks: 1

gaalcaras/mailingListScraper

A python web scraper for public email lists.

Language: Python - Size: 140 KB - Last synced at: about 22 hours ago - Pushed at: about 7 years ago - Stars: 33 - Forks: 13