GitHub topics: scrapy-framework
ljanyst/scrapy-do
A daemon for scheduling Scrapy spiders
Language: Python - Size: 1.27 MB - Last synced at: 24 days ago - Pushed at: about 4 years ago - Stars: 66 - Forks: 18

NikamRohan/News-Aggregator
News Aggregator Using Python,Django ,Scrapy Frameworks.
Language: Python - Size: 5.31 MB - Last synced at: 2 months ago - Pushed at: over 2 years ago - Stars: 31 - Forks: 11

varunnayal15112/E-Commercial-Sites-Scrappers
Developed various python scripts using Scrapy Framework for scraping various E-Commerce Sites like AliExpress, Flipkart, Amazon and Ebay along with the DataBase Connectivity [MYSQLWorkbench].
Language: Python - Size: 18.6 KB - Last synced at: 2 months ago - Pushed at: over 7 years ago - Stars: 41 - Forks: 14

yashpatel7025/PriceTracker
Amazon Price tracker using Python, Django, Celery Task Queue and Scrapy Frameworks. Front end is developed using HTML, CSS and Bootstrap
Language: Python - Size: 4.93 MB - Last synced at: 2 months ago - Pushed at: over 2 years ago - Stars: 54 - Forks: 18

mahmudahsan/webscraping
Lets learn web scraping and apply them in real application.
Language: HTML - Size: 6.53 MB - Last synced at: about 1 month ago - Pushed at: over 2 years ago - Stars: 32 - Forks: 18

NikamRohan/Price-Tracker
A Price Tracker to track the prices of your willing to buy products at E-Commerce websites once they fall below your mentioned Desired price.
Language: Python - Size: 142 KB - Last synced at: about 2 months ago - Pushed at: over 2 years ago - Stars: 18 - Forks: 8

dojutsu-user/IMDB-Scraper
Scrapy project for scraping data from IMDB with Movie Dataset including 58,623 movies' data.
Language: Python - Size: 15.4 MB - Last synced at: 2 months ago - Pushed at: about 5 years ago - Stars: 57 - Forks: 14

AathifZahir/Py-Scrap
This project demonstrates how to build a web scraper using Scrapy, a powerful Python framework for web scraping, and store the extracted data in MongoDB, a flexible NoSQL database.
Language: Python - Size: 28.3 MB - Last synced at: 3 months ago - Pushed at: 5 months ago - Stars: 0 - Forks: 0

marcellin-d/Shopclue-Spider
Shopclue Spider is a Scrapy-based web scraper designed to extract product data from Shopclues categories. This project effectively bypasses AJAX requests to collect detailed information about products, sellers, and their ratings.
Language: Python - Size: 0 Bytes - Last synced at: 6 months ago - Pushed at: 6 months ago - Stars: 1 - Forks: 0

crawlbase/scrapy-proxycrawl-middleware 📦
Scrapy middleware interface to scrape using ProxyCrawl proxy service
Language: Python - Size: 34.2 KB - Last synced at: 8 days ago - Pushed at: almost 2 years ago - Stars: 11 - Forks: 3

intel00000/web_scraper_Hasmo
A python web scraper designed to collect data from competitor websites, news articles, and market research reports.
Language: Jupyter Notebook - Size: 10.4 MB - Last synced at: 3 months ago - Pushed at: 10 months ago - Stars: 0 - Forks: 0

santhoshse7en/IMDb-Scrapy
A fun projects made using Scrapy. The Spiders included in this are able to extract Movie, TV-Series, TV-Movies based on year and title type. A lot more to come ahead
Language: Python - Size: 97.7 KB - Last synced at: 9 days ago - Pushed at: about 5 years ago - Stars: 3 - Forks: 0

epu-ntua/qualichain-jobcrawler
Language: Python - Size: 99.6 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 3 - Forks: 1

mrt-kousha/scrapy
In this repository, I try to share some of the little tips and tricks and amazing spiders that I used to work with on the scrapy framework.
Language: Python - Size: 32.2 KB - Last synced at: over 1 year ago - Pushed at: over 5 years ago - Stars: 11 - Forks: 3

mtaziz/email-scraper-using-google-x-ray-search-with-linkedin
Language: HTML - Size: 230 KB - Last synced at: over 1 year ago - Pushed at: almost 7 years ago - Stars: 3 - Forks: 0

skatepedia/skatepedia-api
An skate API with scraped data, using Scrapy + Django
Language: Python - Size: 2.79 MB - Last synced at: 6 days ago - Pushed at: over 1 year ago - Stars: 4 - Forks: 0

xtutran/sohoa-crawler
Scrape users's comment from Vnexpress.net using Python Scrapy
Language: Python - Size: 9.77 KB - Last synced at: almost 2 years ago - Pushed at: over 7 years ago - Stars: 0 - Forks: 0

CharesFang/WeiboSpider
微博爬虫,一个基于Scrapy框架的轻量微博爬虫,Sina Weibo Spider
Language: Python - Size: 1.12 MB - Last synced at: over 2 years ago - Pushed at: almost 3 years ago - Stars: 94 - Forks: 22

Jaimin1104/IMDB_Web_Scraping_Project
I have used scrapy framework to scrape data from the IMDB website of Movies like name, directors, actors, votes, duration, genre and imdb rating. i had uploaded this spider on zyte cloud platform to run on it and i scraped about 8 lakhs rows of data from IMDB of different genres.
Language: Python - Size: 33 MB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 1 - Forks: 1

thawdezin/agent
This is my B.E Thesis project.
Language: PHP - Size: 17.8 MB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 1 - Forks: 0

matejbasic/PythonScrapyBasicSetup
Basic setup with random user agents and IP addresses for Python Scrapy Framework.
Language: Python - Size: 29.3 KB - Last synced at: over 2 years ago - Pushed at: over 7 years ago - Stars: 57 - Forks: 14

jayeshmanani/Web-Scraping-Using-Scrapy
This repository consist of the code and some related files for use and scrape the website data. Here in the example it is demonstrated with the craigslist.org website.
Language: Python - Size: 6.84 KB - Last synced at: about 2 years ago - Pushed at: about 6 years ago - Stars: 3 - Forks: 3

AnthonyBloomer/nrscrapy
Monitor Scrapy using the New Relic Python Agent API
Language: Python - Size: 44.9 KB - Last synced at: 25 days ago - Pushed at: over 2 years ago - Stars: 1 - Forks: 2

adamfisher/ScrapyRT.Client
A C# client to make calls to a scrapyrt (Scrapy real-time) HTTP endpoint.
Language: C# - Size: 85.9 KB - Last synced at: 3 months ago - Pushed at: over 2 years ago - Stars: 1 - Forks: 0

chaudharypraveen98/QuotesTutorial
This is a scraper which scrapes the quotes on the website using the scrapy framework
Language: Python - Size: 9.55 MB - Last synced at: 2 months ago - Pushed at: almost 4 years ago - Stars: 0 - Forks: 0

marlesson/scrapy_booking
This is a simple Scrapy Project to scrape Hotels
Language: Python - Size: 15.6 KB - Last synced at: about 2 years ago - Pushed at: over 4 years ago - Stars: 1 - Forks: 0

spandanpal22/Web_Scraping_Amazon
This project uses Scrapy framework to do web scraping on Amazon
Language: Python - Size: 15.6 KB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0

an-anurag/learn-scrapy
A complete scrapy course with code snippets and notes
Language: Python - Size: 8.79 KB - Last synced at: over 2 years ago - Pushed at: about 6 years ago - Stars: 0 - Forks: 1

stivenramireza/quotes-to-scrape-website
Scrapy exercises to extract data from quotes.toscrape.com website.
Language: Python - Size: 6.84 KB - Last synced at: over 2 years ago - Pushed at: over 5 years ago - Stars: 0 - Forks: 0

rafaelhenrique/curse_spider
Curse client for Mac/Linux/Wherever
Language: Python - Size: 10.7 KB - Last synced at: 2 months ago - Pushed at: almost 7 years ago - Stars: 9 - Forks: 0

varunnayal15112/Codeforces-AC-Submissions-Downloader
Downloading AC Submisison files for a given user handle from Codeforces Platform using Scrapy Framework in Python.
Language: Python - Size: 38.1 KB - Last synced at: over 2 years ago - Pushed at: about 7 years ago - Stars: 4 - Forks: 0

JyothishArumugam/web_scraping
A web scraping process
Language: Python - Size: 8.79 KB - Last synced at: about 2 months ago - Pushed at: almost 7 years ago - Stars: 1 - Forks: 0

heman2002/EdgarDataScrapper
Language: Python - Size: 9.77 KB - Last synced at: over 2 years ago - Pushed at: almost 8 years ago - Stars: 1 - Forks: 0
