Topic: "web-crawlers"
ScrapeGraphAI/Scrapegraph-ai
Python scraper based on AI
Language: Python - Size: 14 MB - Last synced at: 1 day ago - Pushed at: 20 days ago - Stars: 20,406 - Forks: 1,731
omrilotan/isbot
🤖/👨🦰 Detect bots/crawlers/spiders using the user agent string
Language: TypeScript - Size: 4.26 MB - Last synced at: 15 days ago - Pushed at: 16 days ago - Stars: 1,048 - Forks: 84
bambooww/iot-tree
A IOT-Server,The tree root supports devices and data management, and the upper tree supports to combine various business functions by message flow
Language: Java - Size: 55.4 MB - Last synced at: 11 days ago - Pushed at: 11 days ago - Stars: 119 - Forks: 42
ScrapingAnt/amazon_scraper
Amazon products scraper with using of rotating proxies and headless Chrome from ScrapingAnt
Language: JavaScript - Size: 52.7 KB - Last synced at: 15 days ago - Pushed at: over 1 year ago - Stars: 83 - Forks: 19
crawler-commons/url-frontier
API definition, resources and reference implementation of URL Frontiers
Language: Java - Size: 1.14 MB - Last synced at: 11 days ago - Pushed at: 12 days ago - Stars: 50 - Forks: 12
officialpm/scrape-amazon
🤩 Python Package for Scraping Amazon Product Reviews ✨
Language: Python - Size: 134 KB - Last synced at: 12 months ago - Pushed at: over 2 years ago - Stars: 34 - Forks: 11
ancatmara/learnpython2018
Python course for 2nd year NLP students at NRU HSE, 2018-2019
Language: Jupyter Notebook - Size: 2.65 MB - Last synced at: about 1 month ago - Pushed at: over 1 year ago - Stars: 23 - Forks: 25
OwenOrcan/YiraBot-Crawler
YiraBot: Simplifying Web Scraping for All. A user-friendly tool for developers and enthusiasts, offering command-line ease and Python integration. Ideal for research, SEO, and data collection.
Language: Python - Size: 221 KB - Last synced at: 19 days ago - Pushed at: 8 months ago - Stars: 19 - Forks: 0
dod-advana/gamechanger-crawlers
GAMECHANGER Policy Analytics Site Crawlers
Language: Python - Size: 204 MB - Last synced at: 4 days ago - Pushed at: 12 months ago - Stars: 17 - Forks: 13
fluoos/crawl2ai
一款强大的大模型微调数据集生成和管理工具。
Language: Python - Size: 864 KB - Last synced at: 10 days ago - Pushed at: 11 days ago - Stars: 16 - Forks: 2
ancatmara/learnpython2017
Python course for 2nd year NLP students at NRU HSE, 2017-2018
Language: Jupyter Notebook - Size: 878 KB - Last synced at: 4 months ago - Pushed at: over 1 year ago - Stars: 16 - Forks: 8
romis2012/is-bot
Detect bots/crawlers/spiders via user-agent string
Language: Python - Size: 38.1 KB - Last synced at: 13 days ago - Pushed at: 4 months ago - Stars: 13 - Forks: 1
michaelradu/web-crawler
A Web Crawler developed in Python.
Language: Python - Size: 6.84 KB - Last synced at: 4 months ago - Pushed at: almost 3 years ago - Stars: 7 - Forks: 2
AAlkiyumi/Senior-Design-Project
Web scraper for collecting product and review data from e-commerce sites using Scraping Bee, AWS, Selenium, and Pandas. Focuses on cost-effective solutions, user-friendly interfaces, and efficient data extraction and analysis.
Language: Python - Size: 208 MB - Last synced at: 3 months ago - Pushed at: 4 months ago - Stars: 2 - Forks: 0
ancatmara/crawlers
Web crawlers for Early Irish & Welsh data
Language: Jupyter Notebook - Size: 248 MB - Last synced at: 5 months ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
Indianajaune/Indianajaune.github.io 📦
CV Hosted on Github
Language: HTML - Size: 12.4 MB - Last synced at: over 2 years ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0
mem-dixy/web-crawler 📦
Give it a URL, and it will try to find all URLs it finds.
Language: PHP - Size: 123 KB - Last synced at: about 2 years ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0
codassassin/web-crawler-v2.0
This is an advanced version of the previously released version of web-crawler
Language: Python - Size: 41 KB - Last synced at: over 2 years ago - Pushed at: almost 4 years ago - Stars: 0 - Forks: 0
codassassin/web_crawler
This is a simple web crawler created using python
Language: Python - Size: 11.7 KB - Last synced at: over 2 years ago - Pushed at: almost 4 years ago - Stars: 0 - Forks: 0
abhigyan2001/WebCrawlin101
A tutorial on Crawling the Web and Scraping off the Useful bits - without actually doing anything!
Language: Python - Size: 11.7 KB - Last synced at: over 2 years ago - Pushed at: over 4 years ago - Stars: 0 - Forks: 0
LYingSiMon/examples-of-web-crawlers Fork of shengqiangzhang/examples-of-web-crawlers
一些非常有趣的python爬虫例子,对新手比较友好,主要爬取淘宝、天猫、微信、豆瓣、QQ等网站。(Some interesting examples of python crawlers that are friendly to beginners. )
Size: 225 MB - Last synced at: over 2 years ago - Pushed at: about 5 years ago - Stars: 0 - Forks: 0
