Topic: "web-scraper"
getmaxun/maxun
No Code Web Data Extraction Platform β’ Turn Websites To APIs & Spreadsheets In Minutes
Language: TypeScript - Size: 4.04 MB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 13,061 - Forks: 1,035
BruceDone/awesome-crawler
A collection of awesome web crawler,spider in different languages
Size: 74.2 KB - Last synced at: about 2 months ago - Pushed at: about 1 year ago - Stars: 6,744 - Forks: 716
D4Vinci/Scrapling
π·οΈ An undetectable, powerful, flexible, high-performance Python library to make Web Scraping Easy and Effortless as it should be!
Language: Python - Size: 1.9 MB - Last synced at: 6 days ago - Pushed at: 10 days ago - Stars: 5,424 - Forks: 302
jaypyles/Scraperr
Self-hosted webscraper.
Language: TypeScript - Size: 2.4 MB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 3,360 - Forks: 148
arpit-omprakash/100ProjectsOfCode
A list of practical knowledge-building projects.
Size: 26.4 KB - Last synced at: 9 days ago - Pushed at: about 1 year ago - Stars: 3,345 - Forks: 301
php-curl-class/php-curl-class
PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs
Language: PHP - Size: 2.73 MB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 3,287 - Forks: 821
anaskhan96/soup
Web Scraper in Go, similar to BeautifulSoup
Language: Go - Size: 99.6 KB - Last synced at: 28 days ago - Pushed at: over 1 year ago - Stars: 2,200 - Forks: 168
gosom/google-maps-scraper
scrape data data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
Language: Go - Size: 20.6 MB - Last synced at: 6 days ago - Pushed at: 11 days ago - Stars: 2,101 - Forks: 246
dipu-bd/lightnovel-crawler
Generate and download e-books from online sources.
Language: Python - Size: 33.4 MB - Last synced at: 5 days ago - Pushed at: 9 days ago - Stars: 1,743 - Forks: 339
itsOwen/CyberScraper-2077
A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama
Language: Python - Size: 355 KB - Last synced at: 6 days ago - Pushed at: 9 days ago - Stars: 1,715 - Forks: 154
juancarlospaco/faster-than-requests
Faster requests on Python 3
Language: Nim - Size: 20.4 MB - Last synced at: 17 days ago - Pushed at: 17 days ago - Stars: 1,120 - Forks: 91
tholian-network/stealth
:rocket: Stealth - Secure, Peer-to-Peer, Private and Automateable Web Browser/Scraper/Proxy
Language: JavaScript - Size: 19.9 MB - Last synced at: about 1 month ago - Pushed at: over 1 year ago - Stars: 1,084 - Forks: 321
platonai/PulsarRPA
PulsarRPA: An AI-Enabled, Super-Fast, Thread-Safe Browser Automation Solution! π
Language: Kotlin - Size: 30.4 MB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 883 - Forks: 128
Oshan96/monkey-dl
Bulk download your favourite anime episodes from your favourite anime websites
Language: Python - Size: 807 KB - Last synced at: about 1 month ago - Pushed at: about 2 years ago - Stars: 840 - Forks: 71
gildas-lormeau/single-file-cli
CLI tool for saving a faithful copy of a complete web page in a single HTML file (based on SingleFile)
Language: JavaScript - Size: 5.16 MB - Last synced at: 27 days ago - Pushed at: 3 months ago - Stars: 830 - Forks: 83
postmodern/spidr
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
Language: Ruby - Size: 685 KB - Last synced at: about 1 month ago - Pushed at: 5 months ago - Stars: 818 - Forks: 107
je-suis-tm/web-scraping
Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist
Language: Python - Size: 1.88 MB - Last synced at: about 1 month ago - Pushed at: over 3 years ago - Stars: 787 - Forks: 177
k0rnh0li0/onlyfans-dl π¦
OnlyFans content downloader
Language: Python - Size: 2.43 MB - Last synced at: 3 months ago - Pushed at: over 1 year ago - Stars: 782 - Forks: 223
cassidoo/scrapers
A list of scrapers from around the web.
Size: 58.6 KB - Last synced at: 3 months ago - Pushed at: 5 months ago - Stars: 660 - Forks: 104
oxylabs/how-to-scrape-google-scholar
A guide for extracting titles, authors, and citations from Google Scholar using Python and Oxylabs SERP Scraper API.
Language: Python - Size: 290 KB - Last synced at: about 1 month ago - Pushed at: 3 months ago - Stars: 581 - Forks: 6
spekulatius/PHPScraper
A universal web-util for PHP.
Language: PHP - Size: 6.53 MB - Last synced at: 27 days ago - Pushed at: about 1 year ago - Stars: 561 - Forks: 76
oxylabs/quick-start-guide
Python quick start guides to get the most out of Oxylabs' Web Scraper API free trial.
Size: 46.9 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 523 - Forks: 3
AlexMathew/scrapple
A framework for creating semi-automatic web content extractors
Language: Python - Size: 1.15 MB - Last synced at: 12 days ago - Pushed at: 12 days ago - Stars: 502 - Forks: 41
oxylabs/how-to-scrape-amazon-prices
A code for extracting best-selling items, search results, and currently available deals from Amazon using Python and Oxylabs E-Commerce Scraper API.
Language: Python - Size: 71.3 KB - Last synced at: about 1 month ago - Pushed at: 3 months ago - Stars: 497 - Forks: 6
jaebradley/basketball_reference_web_scraper
NBA Stats API via Basketball Reference
Language: HTML - Size: 19.3 MB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 495 - Forks: 120
austinoboyle/scrape-linkedin-selenium
`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.
Language: HTML - Size: 269 KB - Last synced at: about 1 month ago - Pushed at: over 2 years ago - Stars: 490 - Forks: 166
shaikhsajid1111/social-media-profile-scrapers
Fetch user's data across social media
Language: Python - Size: 4.5 MB - Last synced at: about 1 month ago - Pushed at: 12 months ago - Stars: 478 - Forks: 77
paulpierre/markdown-crawler
A multithreaded πΈοΈ web crawler that recursively crawls a website and creates a π½ markdown file for each page, designed for LLM RAG
Language: Python - Size: 1.14 MB - Last synced at: about 11 hours ago - Pushed at: 11 months ago - Stars: 391 - Forks: 47
crwlrsoft/crawler
Library for Rapid (Web) Crawler and Scraper Development
Language: PHP - Size: 1.02 MB - Last synced at: 15 days ago - Pushed at: 15 days ago - Stars: 364 - Forks: 13
0x676e67/wreq
An ergonomic Rust HTTP Client with TLS fingerprint
Language: Rust - Size: 4.87 MB - Last synced at: about 3 hours ago - Pushed at: about 3 hours ago - Stars: 362 - Forks: 53
lewisdonovan/google-news-scraper
Lightweight scraper for Google News
Language: TypeScript - Size: 895 KB - Last synced at: 24 days ago - Pushed at: 3 months ago - Stars: 330 - Forks: 66
oxylabs/web-unblocker
Free trial Web Unblocker - an AI-powered proxy solution that can bypass even the most sophisticated anti-bot systems.
Language: Python - Size: 1.06 MB - Last synced at: about 9 hours ago - Pushed at: 3 months ago - Stars: 327 - Forks: 48
passivebot/facebook-marketplace-scraper π¦
This repository contains a script to scrape Facebook Marketplace data using Playwright, BeautifulSoup and Streamlit.
Language: Python - Size: 664 KB - Last synced at: about 1 month ago - Pushed at: about 1 year ago - Stars: 302 - Forks: 86
PhantomInsights/summarizer
A Reddit bot that summarizes news articles written in Spanish or English. It uses a custom built algorithm to rank words and sentences.
Language: Python - Size: 254 KB - Last synced at: about 2 months ago - Pushed at: over 3 years ago - Stars: 274 - Forks: 30
duyet/awesome-web-scraper
A collection of awesome web scaper, crawler.
Size: 48.8 KB - Last synced at: about 2 months ago - Pushed at: about 1 year ago - Stars: 273 - Forks: 46
shaikhsajid1111/facebook_page_scraper
Scrapes facebook's pages front end with no limitations & provides a feature to turn data into structured JSON or CSV
Language: Python - Size: 98.6 KB - Last synced at: about 1 month ago - Pushed at: 12 months ago - Stars: 257 - Forks: 75
SenZmaKi/Senpwai
A desktop app for tracking and batch downloading anime
Language: Python - Size: 94.7 MB - Last synced at: 6 days ago - Pushed at: about 1 month ago - Stars: 251 - Forks: 27
epiqueras/getsy
A simple browser/client-side web scraper.
Language: TypeScript - Size: 127 KB - Last synced at: about 1 month ago - Pushed at: about 8 years ago - Stars: 241 - Forks: 15
wikimedia/html-metadata
MetaData html scraper and parser for Node.js (supports Promises and callback style)
Language: JavaScript - Size: 512 KB - Last synced at: about 6 hours ago - Pushed at: 29 days ago - Stars: 175 - Forks: 43
oxylabs/how-to-scrape-indeed
A tutorial for collecting job postings from Indeed using Python and Oxylabs Web Scraper API.
Language: Python - Size: 52.7 KB - Last synced at: about 1 month ago - Pushed at: 3 months ago - Stars: 152 - Forks: 1
suntong/cascadia
Go cascadia package command line CSS selector
Language: Go - Size: 122 KB - Last synced at: 25 days ago - Pushed at: almost 2 years ago - Stars: 142 - Forks: 11
wearrrrr/HaiKei
HaiKei is an anime streaming website that uses the consumet API
Language: JavaScript - Size: 5.83 MB - Last synced at: 5 days ago - Pushed at: 9 days ago - Stars: 133 - Forks: 43
dinguschan-owo/Helios
Helios is an COMPLETELY UNBLOCKABLE proxy with tabs that can be static hosted, can be run locally, and is html css js only! This is (as far as i've found) the only true UNBLOCKABLE only HTML proxy that works with any blocking software! Plus its open sauce so you can take this code and build your own proxy! (β PLEASE star if you fork! β)
Language: HTML - Size: 1.43 MB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 132 - Forks: 138
gan-of-culture/get-sauce
A command line program to download Hentai videos and images from multiple websites
Language: Go - Size: 4.21 MB - Last synced at: 6 months ago - Pushed at: 6 months ago - Stars: 132 - Forks: 11
areed1192/python-sec
A simple python library that allows for easy access of the SEC website so that someone can parse filings, collect data, and query documents.
Language: Python - Size: 26.2 MB - Last synced at: about 1 month ago - Pushed at: 5 months ago - Stars: 125 - Forks: 50
oxylabs/playwright-web-scraping
A tutorial for web scraping using Playwright headless browser
Language: Python - Size: 46.9 KB - Last synced at: about 1 month ago - Pushed at: 4 months ago - Stars: 124 - Forks: 13
Fytex/Instagram-Giveaways-Winner
Instagram Bot which when given a post url will spam mentions to increase the chances of winning. Win Instagram Giveaways!
Language: Python - Size: 12.1 MB - Last synced at: 3 months ago - Pushed at: over 3 years ago - Stars: 117 - Forks: 23
oxylabs/chatgpt-web-scraping
Learn to create ChatGPT prompts that generate a web scraping code with proper CSS selectors.
Size: 1.54 MB - Last synced at: 30 days ago - Pushed at: 4 months ago - Stars: 111 - Forks: 0
dhvitish/AnimeEZ π¦
AnimeEZ - An Anime Streaming website without any ads for free (Demo - https://animeez.live) BTW ITS MADE IN HTML
Language: CSS - Size: 12 MB - Last synced at: 2 months ago - Pushed at: about 2 years ago - Stars: 111 - Forks: 59
k0r0pt/Project-Tauro
A Router WiFi key recovery/cracking tool with a twist.
Language: Java - Size: 104 KB - Last synced at: 6 days ago - Pushed at: over 6 years ago - Stars: 92 - Forks: 16
Krisseck/Detect-CMS
PHP Library for detecting CMS
Language: PHP - Size: 66.4 KB - Last synced at: 15 days ago - Pushed at: over 2 years ago - Stars: 90 - Forks: 51
khuyentran1401/top-github-scraper
Scape top GitHub repositories and users based on keywords
Language: HTML - Size: 452 KB - Last synced at: about 1 month ago - Pushed at: almost 2 years ago - Stars: 85 - Forks: 25
networkdynamics/pytok
A web scraper for TikTok using Playwright
Language: Python - Size: 270 KB - Last synced at: 2 months ago - Pushed at: 2 months ago - Stars: 82 - Forks: 12
watzon/arachnid π¦
Powerful web scraping framework for Crystal
Language: Crystal - Size: 230 KB - Last synced at: about 1 year ago - Pushed at: almost 3 years ago - Stars: 77 - Forks: 12
scrapehero/yellowpages-scraper
Yellowpages.com Web Scraper written in Python and LXML to extract business details available based on a particular category and location.
Language: Python - Size: 16.6 KB - Last synced at: 3 months ago - Pushed at: over 4 years ago - Stars: 77 - Forks: 65
serpapi/public-roadmap
Public Roadmap for SerpApi, LLC (https://serpapi.com)
Size: 17.6 KB - Last synced at: 3 days ago - Pushed at: 7 months ago - Stars: 75 - Forks: 15
ardauzunoglu/TRScraper
TRScraper, doΔal dil iΕleme uygulamalarΔ±nda kullanΔ±lmak amacΔ±yla geliΕtirilmiΕ, TΓΌrkΓ§e iΓ§erik girilen bΓΌyΓΌk platformlarda metin madenciliΔi yapma imkanΔ± sunan bir uygulamadΔ±r.
Language: Python - Size: 970 KB - Last synced at: about 2 years ago - Pushed at: over 4 years ago - Stars: 66 - Forks: 3
GoTrained/Scrapy-Craigslist
Web Scraping Craigslist's Engineering Jobs in NY with Scrapy
Language: Python - Size: 195 KB - Last synced at: 5 months ago - Pushed at: almost 8 years ago - Stars: 66 - Forks: 37
janchaloupka/web-scraper-nabidek-pronajmu
NΓ‘stroj pro hlΓdΓ‘nΓ novΓ½ch nabΓdek nemovitostΓ na populΓ‘rnΓch realitnΓch serverech. NabΓdky jsou vypisovΓ‘ny do Discord roomky.
Language: Python - Size: 94.7 KB - Last synced at: 3 months ago - Pushed at: over 1 year ago - Stars: 60 - Forks: 19
ankitmathur3193/song-cli
A command line interface for downloading Bollywood and punjabi songs
Language: Python - Size: 39.1 KB - Last synced at: 8 days ago - Pushed at: about 2 years ago - Stars: 59 - Forks: 13
milahu/aiohttp_chromium
aiohttp-like interface to chromium. based on selenium_driverless to bypass cloudflare
Language: Python - Size: 999 KB - Last synced at: 3 months ago - Pushed at: 7 months ago - Stars: 53 - Forks: 8
Cooya/Linkedin-Client
Web scraper for grabing data from Linkedin profiles or company pages (personal project)
Language: JavaScript - Size: 1.84 MB - Last synced at: over 1 year ago - Pushed at: almost 3 years ago - Stars: 53 - Forks: 17
cobalt-uoft/uoft-scrapers
Public web scraping scripts for the University of Toronto.
Language: Python - Size: 619 KB - Last synced at: 3 months ago - Pushed at: about 5 years ago - Stars: 51 - Forks: 14
SanjaySunil/email-scraper
Generate thousands of temporary emails within seconds!
Language: Python - Size: 341 KB - Last synced at: 2 months ago - Pushed at: 8 months ago - Stars: 50 - Forks: 8
ryanirl/CraigslistScraper
Simple webscraper for Craigslist.
Language: Python - Size: 2.48 MB - Last synced at: 7 days ago - Pushed at: about 1 year ago - Stars: 48 - Forks: 22
Nasdin/VideoRecognition-realtime-autotrainer-alerts
State of the art object detection in real-time using YOLOV3 algorithm. Augmented with a process that allows easy training of the classifier as a plug & play solution . Provides alert if an item in an alert list is detected.
Language: Python - Size: 60.5 MB - Last synced at: 2 months ago - Pushed at: over 6 years ago - Stars: 48 - Forks: 24
rzkfyn/otakudesu-scraper π¦
unofficial otakudesu.cam rest api
Language: TypeScript - Size: 329 KB - Last synced at: 6 days ago - Pushed at: 9 months ago - Stars: 47 - Forks: 23
mawrkus/jason-the-miner
β A versatile Web scraper for Node.js
Language: JavaScript - Size: 2.47 MB - Last synced at: about 2 months ago - Pushed at: 3 months ago - Stars: 45 - Forks: 11
codegratia/react-node-web-scraper
Final Year project, scraping data of e-commerce stores and display in ReactJS app.
Language: JavaScript - Size: 14 MB - Last synced at: 9 months ago - Pushed at: 9 months ago - Stars: 45 - Forks: 18
JLospinoso/abrade
A fast Web API scraper written in C++ and built on Boost ASIO
Language: C++ - Size: 1.89 MB - Last synced at: 3 months ago - Pushed at: about 5 years ago - Stars: 45 - Forks: 4
scrapfly/python-scrapfly
Scrapfly Python SDK for headless browsers and proxy rotation
Language: Python - Size: 673 KB - Last synced at: 3 days ago - Pushed at: about 2 months ago - Stars: 43 - Forks: 11
GoncaloMark/CobWeb-lnx
CobWeb is a Python library for web scraping. The library consists of two classes: Spider and Scraper.
Language: Python - Size: 7.75 MB - Last synced at: 6 days ago - Pushed at: over 1 year ago - Stars: 38 - Forks: 2
CNuge/email-report
A modular template for scraping data from the web to send yourself scheduled email reports
Language: Python - Size: 34.2 KB - Last synced at: almost 2 years ago - Pushed at: almost 5 years ago - Stars: 38 - Forks: 9
worldwidemisery/pycamp
a command-line tool to fetch a random bandcamp album from a chosen genre - instantly.
Language: Python - Size: 113 KB - Last synced at: 17 days ago - Pushed at: 20 days ago - Stars: 35 - Forks: 0
FudgeRK/MyfansDownloader
myfans.jp content downloader
Language: Python - Size: 160 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 35 - Forks: 15
milahu/opensubtitles-scraper
scrape subtitles from opensubtitles.org
Language: Python - Size: 1.57 MB - Last synced at: about 2 months ago - Pushed at: 2 months ago - Stars: 33 - Forks: 2
ssimunic/jsonscraper
JSON configurable concurrent scraper
Language: Go - Size: 20.5 KB - Last synced at: about 1 year ago - Pushed at: about 8 years ago - Stars: 33 - Forks: 1
FoamoftheSea/mod5project
Developing a long/short equity investment portfolio with Machine Learning predictions using data acquired from web-scraping. Flatiron Module 5 Project.
Language: Jupyter Notebook - Size: 36.4 MB - Last synced at: about 1 year ago - Pushed at: over 4 years ago - Stars: 32 - Forks: 19
qascade/yast
Yet Another Streaming Tool
Language: Go - Size: 4.28 MB - Last synced at: about 1 year ago - Pushed at: about 2 years ago - Stars: 31 - Forks: 10
Encryptor-Sec/Web-Scraper
Web Scraper is a melange of Web tools for web hacking, reconnaissance, bug bounty so on. This tool consists of 20 most used web tools for security assessment
Language: Python - Size: 3.5 MB - Last synced at: 9 days ago - Pushed at: 9 days ago - Stars: 30 - Forks: 11
oxylabs/how-to-scrape-amazon-product-data
The process of extracting product data from Amazon using Python, including titles, ratings, prices, images, and descriptions.
Size: 2.42 MB - Last synced at: 20 days ago - Pushed at: 20 days ago - Stars: 29 - Forks: 1
nuzulul/telegram-scraper
A simple Telegram channel scraper
Language: JavaScript - Size: 36.1 KB - Last synced at: 21 days ago - Pushed at: 9 months ago - Stars: 29 - Forks: 12
gmastergreatee/Fanfiction-Manager
Provides extreme flexibility with the help of Rule system to power-users* in downloading, tracking-status & reading novels from ANY site they want.
Language: JavaScript - Size: 1.7 MB - Last synced at: 10 months ago - Pushed at: 10 months ago - Stars: 29 - Forks: 3
Shivanshu-Gupta/web-scrapers
A repository of my web-scraping projects
Language: Python - Size: 267 KB - Last synced at: 8 months ago - Pushed at: 8 months ago - Stars: 28 - Forks: 20
jetkai/proxy-scraper
This is an application that scrapes various Proxy API Endpoints, then compiles the proxies into files within the "/proxies/" directory.
Language: Kotlin - Size: 104 KB - Last synced at: 3 months ago - Pushed at: over 2 years ago - Stars: 27 - Forks: 5
PhantomInsights/reddit-bots
A collection of Reddit bots that I use to enhance the subreddits I manage.
Language: Python - Size: 61.5 KB - Last synced at: about 2 months ago - Pushed at: over 3 years ago - Stars: 27 - Forks: 11
raprocks/sanfoundry-scraper
A Small Scraping Script written in Python that helps you collect and merge all questions for a subject on sanfoundry.com into a HTML document with additional data.
Language: Python - Size: 31.3 KB - Last synced at: over 2 years ago - Pushed at: over 3 years ago - Stars: 27 - Forks: 9
NobilityDeviant/Wcofun.com_Downloader
A java & Kotlin cartoon and anime downloader for https://www.wcofun.com/
Language: Kotlin - Size: 32.2 MB - Last synced at: 10 months ago - Pushed at: 10 months ago - Stars: 26 - Forks: 2
0x01h/hepsiburada-review-scraper π¦
Hepsiburada review/comment and rating scraper. Turkish text dataset creator for data science and NLP projects. π
Language: Python - Size: 69.3 KB - Last synced at: over 2 years ago - Pushed at: over 5 years ago - Stars: 26 - Forks: 4
omkarcloud/botasaurus-starter
π OFFICIAL STARTER TEMPLATE FOR BOTASAURUS SCRAPING FRAMEWORK π€
Language: TypeScript - Size: 397 KB - Last synced at: 6 days ago - Pushed at: about 2 months ago - Stars: 25 - Forks: 9
luminati-io/LinkedIn-Scraper
Extract LinkedIn data with the #1 LinkedIn Scraper API, including profiles, job postings, company details, connections, and posts. Start your free trial now!
Language: Python - Size: 4.6 MB - Last synced at: about 2 hours ago - Pushed at: 2 months ago - Stars: 25 - Forks: 8
codefornola/assessor-scraper
A project to scrape the assessor's website and make the data accessible for advanced queries
Language: Python - Size: 2.77 MB - Last synced at: about 1 year ago - Pushed at: over 2 years ago - Stars: 25 - Forks: 16
kcsoc/society-email-scrape
Scrapes Every Email Address of Every Society in Every University
Language: Python - Size: 975 KB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 24 - Forks: 4
Decodo/Web-Scraping-API
Web Scraping API code examples for Python, PHP and Node.js
Language: JavaScript - Size: 77.1 KB - Last synced at: about 1 month ago - Pushed at: 2 months ago - Stars: 24 - Forks: 9
dhvitish/AnimeEZ-api
AnimeEZ API which scrapes from gogoanime to get details and streaming link of anime(s) without ads.
Language: JavaScript - Size: 20.5 KB - Last synced at: about 1 month ago - Pushed at: about 1 year ago - Stars: 24 - Forks: 21
NoxelS/openai-scraper
This is a template repository for building a web scraper with OpenAI support. The repository provides a basic project structure with TypeScript and Puppeteer pre-configured, as well as OpenAI's GPT-3 API integration. With this template, you can easily build a scraper that uses machine learning to analyze and extract insights from the scraped data.
Language: TypeScript - Size: 26.4 KB - Last synced at: about 1 year ago - Pushed at: over 1 year ago - Stars: 24 - Forks: 4
raymelon/tagalog-dictionary-scraper
Builds a Tagalog dictionary by collecting Tagalog words from tagalog.pinoydictionary.com
Language: Python - Size: 997 KB - Last synced at: over 1 year ago - Pushed at: over 2 years ago - Stars: 23 - Forks: 14
faraui/cloudflare-bypass-headless-web-scraper
Headless web-scraper template that bypasses the Cloudflare IUAM protection. Working on X virtual frame buffer (Xvfb) and Perl modified WWW::Mechanize::Chrome module.
Language: Shell - Size: 1.2 MB - Last synced at: 10 months ago - Pushed at: 10 months ago - Stars: 21 - Forks: 0
KnlnKS/uber_eats_scraper
An Uber Eats scraper written in python.
Language: Python - Size: 10.2 MB - Last synced at: about 2 months ago - Pushed at: about 3 years ago - Stars: 21 - Forks: 4
michaeluno/php-simple-web-scraper
A PHP application which runs on Heroku and dumps web site outputs including JavaScript generated contents.
Language: PHP - Size: 1.4 MB - Last synced at: 2 months ago - Pushed at: about 4 years ago - Stars: 20 - Forks: 19
