An open API service providing repository metadata for many open source software ecosystems.

GitHub topics: webscraper

mghmm/allecijfers-scraper

This script automatically downloads the “250 onderwerpen” tables for every Dutch municipality from AlleCijfers.nl and saves each set of tables into a separate Excel workbook.

Language: Python - Size: 1.95 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 0 - Forks: 0

dimitryzub/ecommerce-scraper-py

Scrape ecommerce websites such as Amazon, eBay, Walmart, Home Depot, Google Shopping from a single module in Python🐍

Language: Python - Size: 147 KB - Last synced at: 12 days ago - Pushed at: almost 2 years ago - Stars: 12 - Forks: 1

Abhishekn1947/SamGov-Scraper

Automated Python scraper for sam.gov contracts

Language: Python - Size: 2.47 MB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 0 - Forks: 0

AnthonyBloomer/rcp

Python client for RealClearPolitics.

Language: Python - Size: 9.77 KB - Last synced at: 7 days ago - Pushed at: over 2 years ago - Stars: 29 - Forks: 9

CuriousLearner/GeeksForGeeksScrapper

Scrapes g4g and creates PDF

Language: Python - Size: 147 KB - Last synced at: 6 days ago - Pushed at: about 5 years ago - Stars: 147 - Forks: 65

Decodo/trustpilot_python_scraper

Scrape Trustpilot utilising Decodo's Web Scraping API

Language: Python - Size: 24.4 KB - Last synced at: about 2 months ago - Pushed at: 2 months ago - Stars: 8 - Forks: 0

lhfmartin/lihkg-scraper

Download LIHKG posts with images

Language: Python - Size: 85 KB - Last synced at: 2 months ago - Pushed at: 2 months ago - Stars: 1 - Forks: 1

yawnbo/movers

Watch movies from the terminal :)

Language: Rust - Size: 225 KB - Last synced at: about 1 month ago - Pushed at: 2 months ago - Stars: 1 - Forks: 0

sakan811/Find-Osaka-And-Japan-Average-Hotel-Price

Showcase visualizations about Osaka and Japan Average Hotel Price. The data was collected from Booking.com

Language: Python - Size: 27.5 MB - Last synced at: 2 months ago - Pushed at: 2 months ago - Stars: 1 - Forks: 0

jfernando-work/GCScraper

A web scraper that collects product listings from a popular musical instrument retailer's website.

Language: Python - Size: 60.5 KB - Last synced at: 2 months ago - Pushed at: 2 months ago - Stars: 0 - Forks: 0

HelloThereMatey/tedata

Scraper for Trading Economics

Language: Jupyter Notebook - Size: 5.94 MB - Last synced at: 25 days ago - Pushed at: 2 months ago - Stars: 3 - Forks: 1

ejones18/gig_scaper 📦

A flask UI / web scraper to find gigs/concerts for an input artists across multiple sites! Now includes visualisation of gig venue locations!

Language: Python - Size: 59.6 KB - Last synced at: 2 months ago - Pushed at: 2 months ago - Stars: 0 - Forks: 0

Pawel-Parma/dademoiselle

Web scraper for Phantom of the Night Genshin Impact web event

Language: Rust - Size: 8.1 MB - Last synced at: 2 months ago - Pushed at: 2 months ago - Stars: 0 - Forks: 0

DevEnterpriseSoftware/scrapi-mcp

MCP server for using ScrAPI to scrape web pages.

Language: JavaScript - Size: 310 KB - Last synced at: 19 days ago - Pushed at: 2 months ago - Stars: 3 - Forks: 1

funbeedev/TrelloBoardsAutoExport

A web scraper script to automate JSON export of Trello boards. Useful for backing up all your Trello boards automatically.

Language: Python - Size: 7.22 MB - Last synced at: 5 days ago - Pushed at: over 2 years ago - Stars: 5 - Forks: 0

jishnukoliyadan/NAV_Scrapper

NAV Scraper is a Python tool that fetches real-time stock and mutual fund NAVs, merges them with holdings data, and stores results in a database or exports to JSON. It supports automated daily updates via cron jobs and uses rate-limited API calls for efficient scraping.

Language: Python - Size: 24.4 KB - Last synced at: 2 months ago - Pushed at: 2 months ago - Stars: 0 - Forks: 0

serpapi/lego-ai-parser 📦

Lego AI Parser is an open-source application that uses OpenAI to parse visible text of HTML elements.

Language: Python - Size: 199 KB - Last synced at: 4 days ago - Pushed at: about 1 year ago - Stars: 233 - Forks: 15

intergalacticalvariable/reader

📚 This is an adapted version of Jina AI's Reader for local deployment using Docker. Convert any URL to an LLM-friendly input with a simple prefix http://127.0.0.1:3000/https://website-to-scrape.com/

Language: TypeScript - Size: 2.57 MB - Last synced at: 2 months ago - Pushed at: 9 months ago - Stars: 185 - Forks: 31

GeminidSystems/GoogleNewsScraper

A Python package that scrapes Google News article data while remaining undetected by Google. Our scraper can scrape page data up until the last page and never trigger a CAPTCHA (download stats: https://pepy.tech/project/GoogleNewsScraper)

Language: Python - Size: 15.3 MB - Last synced at: 7 days ago - Pushed at: over 3 years ago - Stars: 12 - Forks: 5

mghmm/funda-scraper

A Python scraper that retrieves property data from Funda and saves it as CSV files.

Language: Python - Size: 3.91 KB - Last synced at: about 2 months ago - Pushed at: 3 months ago - Stars: 1 - Forks: 0

haydenheroux/rpifood

Command-line interface for viewing food on RPI's campus. Scrapes data from the Sodexo Everyday API endpoints.

Language: Go - Size: 38.1 KB - Last synced at: about 1 month ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

gamemann/How-To-Use-Selenium-And-BeautifulSoup

A full lab and how-to guide on how to use Selenium paired with Beautiful Soup to parse and extract data from a website using Python.

Language: Python - Size: 228 KB - Last synced at: 7 days ago - Pushed at: over 1 year ago - Stars: 16 - Forks: 1

TsekNet/gomls

Scrape zillow.com listings from your terminal, supports multiple output types (json, csv, html, etc.)

Language: Go - Size: 2.92 MB - Last synced at: 6 days ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

rootVIII/proxy_requests 📦

a class that uses scraped proxies to make http GET/POST requests (Python requests)

Language: Python - Size: 750 KB - Last synced at: 7 days ago - Pushed at: over 4 years ago - Stars: 390 - Forks: 43

vypivshiy/scrape-schema 📦

Structuring parsed data into python objects

Language: Python - Size: 1.6 MB - Last synced at: 7 days ago - Pushed at: 3 months ago - Stars: 4 - Forks: 1

anouarbensaad/wsvuls

wsvuls - website vulnerability scanner detect issues [ outdated server software and insecure HTTP headers.]

Language: Python - Size: 108 KB - Last synced at: 9 days ago - Pushed at: over 3 years ago - Stars: 54 - Forks: 12

3xploitGuy/webscrape

A web scraper to scrape email's and phone numbers from Websites.

Language: Shell - Size: 6.84 KB - Last synced at: 2 months ago - Pushed at: about 3 years ago - Stars: 83 - Forks: 15

R34P3R9URU/Web-Scraper

Basic Pure GO Web Scraper

Language: Go - Size: 2.93 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 1 - Forks: 0

naghim/AS-utils

A collection of small, useful scripts for automating common tasks. These tools were designed to help with file management, subtitle processing, and more.

Language: Python - Size: 9.77 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

aalapd/wormpy

A simple Python-based tool designed to scrape websites for content.

Language: Python - Size: 286 KB - Last synced at: 3 months ago - Pushed at: 9 months ago - Stars: 54 - Forks: 8

spencerhobert/valorantPredictor

A web scraper and Machine Learning project. The web scraper will gather the information, feed it to a Machine Learning Model, and then the MLM will predict who will win a future match.

Language: Python - Size: 14.1 MB - Last synced at: 27 days ago - Pushed at: 9 months ago - Stars: 1 - Forks: 0

kevanwee/elitiscraper

elitigation scraper using beautifulsoup4

Language: Python - Size: 0 Bytes - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 1 - Forks: 0

xmedinavei/python-web-scrapper

A Python web scraper for BBC News using Selenium and BeautifulSoup4, powered by AI prompt engineering with ChatGPT. It extracts article data like URLs, headlines, and images, handles dynamic content, and outputs clean JSON. Includes Docker support for easy deployment.

Language: HTML - Size: 38.1 KB - Last synced at: 27 days ago - Pushed at: 5 months ago - Stars: 0 - Forks: 1

stoozebot/bot-inspect

An automated bot that scraps update from the university website and sends it to the users via webhook.

Language: TypeScript - Size: 221 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

dwallach1/Stocker

Financial Web Scraper & Sentiment Classifier

Language: Python - Size: 8.94 MB - Last synced at: 16 days ago - Pushed at: over 4 years ago - Stars: 152 - Forks: 41

Kush238/go-web-scraper

Web Scraper made using GO

Language: Go - Size: 10.7 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

ItsAmaryllis/goprox

🚀 Goprox: Simplify Google searches with automatic proxy handling and user-agent selection.

Language: Python - Size: 14.6 KB - Last synced at: 19 days ago - Pushed at: about 1 year ago - Stars: 10 - Forks: 0

Ceguim13/py-scraper

Py-Scraper is a powerful web scraping application built using Python, Flask, and BeautifulSoup. This project allows users to search for information on the web and store the results in a PostgreSQL database. It is designed to be user-friendly and efficient, making it easy for developers and researchers to gather data from various sources.

Language: Python - Size: 5.86 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

AdritPal08/Smart-Web-Scraper-2.0-using-Gen-AI

Effortless Data Extraction, Powered by : Generative AI

Language: Python - Size: 8.52 MB - Last synced at: 3 months ago - Pushed at: 5 months ago - Stars: 2 - Forks: 0

wwachira/NextJS-WebScraper

WebScraping Oversight

Language: TypeScript - Size: 53.7 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

albegonzalezp/webScraper

Basic Golang web scraper template using Go Colly and PostgreSQL

Language: Go - Size: 29.3 KB - Last synced at: 3 months ago - Pushed at: 10 months ago - Stars: 0 - Forks: 0

xpotify/scraper

Scraper designed for Xpotify's client to gather information from websites🌟

Language: JavaScript - Size: 36.1 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 1 - Forks: 0

mehmetozkaya/DotnetCrawler

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

Language: C# - Size: 70.3 KB - Last synced at: about 2 months ago - Pushed at: over 2 years ago - Stars: 176 - Forks: 66

lntechnical2/webscrap-bot

Simple Telegram webscrap bot

Language: Python - Size: 13.7 KB - Last synced at: 3 months ago - Pushed at: over 2 years ago - Stars: 19 - Forks: 16

swp-berlin/webassistant

Monitor websites for new publications

Language: Python - Size: 1.59 MB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 3 - Forks: 0

darryldejong/Vacaturejager

VacatureJager is een webscraper die vacatures van werkzoeken.nl verzamelt en je helpt bij het zoeken naar een baan

Language: Python - Size: 215 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 1 - Forks: 0

build-on-aws/bedrock-agents-webscraper

This repo provides guidance on setting up a bedrock agent to webscrape and internet search via action groups

Language: Python - Size: 37.8 MB - Last synced at: 3 months ago - Pushed at: 5 months ago - Stars: 45 - Forks: 6

Open-Technology-Foundation/html_deltags

html_deltags - Python module and Shell script to detag and normalize a HTML file/stream. Removes specified HTML tags -- including those containing certain keywords -- and comments.

Language: Python - Size: 39.1 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

99monisha/Smart-Web-Scraper-2.0-using-Gen-AI

Effortless Data Extraction, Powered by : Generative AI

Size: 1000 Bytes - Last synced at: 25 days ago - Pushed at: 3 months ago - Stars: 1 - Forks: 0

Architrixs/Wattpad2Epub

Python Script to Scrape Wattpad Story and convert to Epub and html file. Easiest to use.

Language: Python - Size: 217 KB - Last synced at: 3 months ago - Pushed at: almost 2 years ago - Stars: 31 - Forks: 5

mominurr/Web-Scraping-Projects

Explore a variety of web scraping projects showcasing my skills and experience in extracting valuable data and solving complex challenges.

Size: 19.5 KB - Last synced at: 10 days ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

mominurr/Walmart-Data-Scraping

walmart product data scraping and bypassing ip blocking, press & hold captcha.

Size: 568 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

mominurr/realSelf.com_scraper

realself.com data scraper that scrape website all information and bypass ip blocking and press & hold captcha.

Size: 146 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 1

mominurr/Google-Map-Scraping

google map scraper collect google map all available data and collect email from business website.

Size: 21.5 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

HexNio/booking_scraper

A booking.com Web Scraper for Data Mining/Harvesting and Automation

Language: Python - Size: 53.7 KB - Last synced at: 8 days ago - Pushed at: about 4 years ago - Stars: 33 - Forks: 24

Factoria-F5-dev/webscraper

Web scraper con Python, Selenium, Docker y Cronjob

Language: Python - Size: 69.3 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 2 - Forks: 1

code-yeongyu/TrackPurchase

단 몇줄의 코드로 다양한 쇼핑 플랫폼에서 결제 내역을 긁어오자!

Language: TypeScript - Size: 619 KB - Last synced at: 4 days ago - Pushed at: over 3 years ago - Stars: 17 - Forks: 0

jahnltib/reviewpal.me Fork of julmcardenas/web-scraper-olo

Analyze product quality based on aggregated customer reviews sentiment given a product URL or search term

Language: TypeScript - Size: 9.2 MB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

Zilts345890/Golang-html-parsing

Parse HTML in Go using Node Parser, Tokenizer, and tools like Goquery and Colly, with practical examples and efficient web scraping techniques

Size: 1.95 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

onepointAI/onepoint

An AI assistant tool that integrates coding, writing, and reading functions. For better alternatives see https://monica.im/desktop

Language: TypeScript - Size: 10.1 MB - Last synced at: 3 months ago - Pushed at: about 2 years ago - Stars: 311 - Forks: 45

ZenXen7/Webscraper-with-LLM

This project is a Python-based webscraper utilizing the Ollama Language Model (LLM) to enhance web scraping capabilities with natural language processing. The scraper efficiently extracts data from websites and uses Ollama’s advanced language model to parse, clean, and analyze the data.

Language: Python - Size: 129 MB - Last synced at: 3 months ago - Pushed at: 8 months ago - Stars: 2 - Forks: 0

xvertile/Flux

Flux is a powerful tool designed to monitor proxy providers across the industry, analyzing response times, uptime, outgoing IPs, and more. With Flux, you can uncover the true performance and integrity of proxy providers, ensuring you're working with reliable data and not falling for misleading claims.

Language: Go - Size: 23.8 MB - Last synced at: 3 days ago - Pushed at: 7 months ago - Stars: 19 - Forks: 2

Superjoa10/Maps_Web_Scrapper

Esse projeto é uma automação de Web Scraping do google maps que extrai informações de negocios e salva para um arquivo de tipo solicitado.

Language: Java - Size: 21.5 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

mominurr/stackoverflow.com

A web scraper collecting Stack Overflow questions for NLP, using threading and user-agent rotation

Language: Python - Size: 0 Bytes - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0

TheElevatedOne/ClearOutsideAPY

Webscraper/API for ClearOutside.com

Language: Python - Size: 238 KB - Last synced at: 9 days ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0

andreuvv/myl_scraper

tor.myl.cl web scraper for TCG Mitos y Leyendas (MyL)

Language: Python - Size: 46.9 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 1 - Forks: 0

KenWuqianghao/Internship-Opportunity-Web-Scraper

A web scraper that finds opportunities and internships for highschoolers

Language: Jupyter Notebook - Size: 20.5 KB - Last synced at: about 1 month ago - Pushed at: over 3 years ago - Stars: 1 - Forks: 0

zNitche/ao3-web-reader

ao3 scraper / reader

Language: Python - Size: 303 KB - Last synced at: 3 months ago - Pushed at: 4 months ago - Stars: 8 - Forks: 0

boringPpl/Linkedin-profiles-scraping

Automatically scrape the web data of people profiles on Linkedin based on a specific search query

Language: Jupyter Notebook - Size: 25.4 KB - Last synced at: 3 months ago - Pushed at: over 1 year ago - Stars: 66 - Forks: 31

RodrigoCarrico01/scraper_pagination

Trying to use scraping in multiple pages at the same time.

Language: JavaScript - Size: 6.95 MB - Last synced at: 3 months ago - Pushed at: 5 months ago - Stars: 1 - Forks: 0

RodrigoCarrico01/webscraping_brcgs

I developed this scraper with QuinaRibeiro Lda.

Language: JavaScript - Size: 109 KB - Last synced at: 3 months ago - Pushed at: 5 months ago - Stars: 1 - Forks: 0

Brand07/HP-Printer-Metrics-Scrape

Python script that scrapes the internal web pages of HP Printers for data such as page count, toner level, and maintenance kit level.

Language: Python - Size: 23.4 KB - Last synced at: 3 months ago - Pushed at: 4 months ago - Stars: 1 - Forks: 0

MakeYourLifeEasier/Wuxiaworld-2-eBook

This Python script will download chapters from novels availaible on wuxiaworld.com saves then into the .epub format

Language: Python - Size: 78.1 KB - Last synced at: 3 months ago - Pushed at: over 4 years ago - Stars: 92 - Forks: 38

Alvian121/nepali-news-scraper

Nepali News Scraper is a Node.js package that allows you to effortlessly scrape news articles from Kathmandu Post and Kantipur. It provides rich article data, including titles, authors, published dates, summaries, and full content, with user-defined limits for flexibility.

Size: 1000 Bytes - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0

dimitryzub/py-google-scholar-organic-cite-to-csv-sqlite

Scrape historic Google Scholar Organic and Cite results to CSV, MySQL Lite using Python and SerpApi.

Language: Python - Size: 11.7 KB - Last synced at: 2 months ago - Pushed at: over 3 years ago - Stars: 4 - Forks: 4

DeliciousBoy/web-scraper

Automated Web Scraper With Python

Size: 22.5 KB - Last synced at: 3 months ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0

aryansh3lke/trendscope

An AI tool to gain insight into X (Twitter) trends through web-scraping, natural language processing, and abstractive summarization.

Language: TypeScript - Size: 7.53 MB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 1 - Forks: 0

jibi21212/UFC_web_scraper

Scrapes Fighter data from the UFC website using Python webscraper

Language: Python - Size: 18.6 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0

fadhilyori/pywebscraper

A simple web scraper that uses the BeautifulSoup library to scrape the web. This project is still early in development and initially for personal use.

Language: Python - Size: 13.7 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0

Lennart1978/picturescrape

Collect pictures from a website and download them - powered by Colly and Fyne GUI.

Language: Go - Size: 728 KB - Last synced at: about 2 months ago - Pushed at: over 1 year ago - Stars: 4 - Forks: 1

jakbin/pcdt-scraper

A PyChromeDevTools based WebScraper and selenium like syntax.

Language: Python - Size: 6.84 KB - Last synced at: 6 days ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0

datacollectionspecialist/Scrape-Google-Scholar

Learn how to scrape Google Scholar results effectively with step-by-step instructions and the best practices. Discover tools and techniques to gather academic research data efficiently.

Size: 12.7 KB - Last synced at: 23 days ago - Pushed at: 4 months ago - Stars: 2 - Forks: 0

thepravin/amazon-web-scripting

Language: Jupyter Notebook - Size: 0 Bytes - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0

ikp4success/shopasource

Easiest way to find best lowest price products online.

Language: Python - Size: 3.54 MB - Last synced at: about 2 months ago - Pushed at: about 2 years ago - Stars: 8 - Forks: 0

jamezmca/stock-data-webscraper

Stock Data Web Scraper (Free) w. Node.JS & Cheerio

Language: JavaScript - Size: 415 KB - Last synced at: 3 months ago - Pushed at: over 1 year ago - Stars: 3 - Forks: 0

OliverHennhoefer/quant

UNMAINTAINED | R-package providing access to fundamental data and valuation metrics for thousands of publicly traded companies worldwide.

Language: R - Size: 3.98 MB - Last synced at: 2 months ago - Pushed at: almost 3 years ago - Stars: 9 - Forks: 2

OshekharO/Link-Crawler

Get movie & Tv Shows stream by IMDB, TMDB.

Language: HTML - Size: 636 KB - Last synced at: 4 months ago - Pushed at: 10 months ago - Stars: 15 - Forks: 6

ap-atul/ranobe

light novel scraper in rust

Language: Rust - Size: 11.7 KB - Last synced at: 3 months ago - Pushed at: over 2 years ago - Stars: 2 - Forks: 0

areebbeigh/anime-scraper

[partially working] Scrape and add anime episode stream URLs to uGet (Linux) or IDM (Windows) ~ Python3

Language: Python - Size: 20.1 MB - Last synced at: 3 months ago - Pushed at: over 6 years ago - Stars: 24 - Forks: 5

AlainaMariaJoe/WebScraper

A baisc web scraper built with Python’s BeautifulSoup to extract text from websites.

Language: Python - Size: 1.95 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0

manjushree08/AI-Scraper-crawl4AI-Streamlit

AI-Scraper-crawl4AI is a web scraping application built with Streamlit and Crawl4AI. It allows users to input a URL, scrape the content of the webpage, and download the scraped content as a Markdown file.

Language: Python - Size: 51.8 KB - Last synced at: about 1 month ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0

savnyc/silentscraper

SilentScraper is a web scraping solution built with advanced stealth protocols. It operates undetectably in the background, bypassing anti-scraping mechanisms to collect structured data at scale. It's lightwight architecture mimics humans browsing patterns, rotating IP addresses, spoofing user agents, and more

Language: Python - Size: 13.7 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 1 - Forks: 0

bvoris/osintprofdev

This tool gathers information from the end user about a specific target. First name, last/surname, location, etc. are gathered to generate a list and scrape specific websites to gather additional information about the target. This can be used to build a web report.

Language: Python - Size: 796 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 6 - Forks: 2

agumfanani19/Web-Scraper-CNBC

Web-Scraper for news

Language: Python - Size: 35.2 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0

ccrsxx/autobsi

Attendance script for My Best BSI

Language: Python - Size: 92.8 KB - Last synced at: 3 months ago - Pushed at: over 1 year ago - Stars: 18 - Forks: 8

xtream1101/scraperx

Library for scraping websites or apis at any scale

Language: Python - Size: 304 KB - Last synced at: 3 months ago - Pushed at: over 1 year ago - Stars: 53 - Forks: 11

ranbot-ai/.github

Size: 3.91 KB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 0 - Forks: 0

JadynHax/scpscraper

A Python library designed for scraping data from the SCP wiki.

Language: Python - Size: 216 KB - Last synced at: about 1 month ago - Pushed at: over 4 years ago - Stars: 15 - Forks: 4

Crabmeat8179/ScrapeUIusingGalleryDL

A small UI front end for gallery-dl to make it easier and quicker to use.

Language: C# - Size: 21.5 KB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 0 - Forks: 0

sukhcha-in/dart_web_scraper

Config-based, reusable web scraper for web and API scraping. Scrape multiple pages or APIs without writing parsers or scraping logic, using simple configurations for efficient scraping.

Language: Dart - Size: 86.9 KB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 7 - Forks: 1