GitHub topics: scraping-websites

Repositories

euvitorti/Teste-Intuitive-Care

Teste de Nivelamento - Estágio

Language: Python - Size: 130 MB - Last synced at: about 9 hours ago - Pushed at: about 10 hours ago - Stars: 1 - Forks: 0

brittojo7n/WebScrap-TT2UserDetails

Totem Tribe 2 Jotun User details (ids and account names) dataset and script for scraping and updating the dataset.

Language: Python - Size: 28.4 MB - Last synced at: about 16 hours ago - Pushed at: about 17 hours ago - Stars: 0 - Forks: 0

chapeljuice/scraper

Web app for creating a data scraping website using Vite + React + ExpressJS + Puppeteer

Language: TypeScript - Size: 1.55 MB - Last synced at: about 16 hours ago - Pushed at: about 17 hours ago - Stars: 0 - Forks: 0

Stream4me/addon

Stream4me addon main repository

Language: Python - Size: 75.4 MB - Last synced at: about 18 hours ago - Pushed at: about 20 hours ago - Stars: 104 - Forks: 9

0xAdafang/Book_Scrap

scraping de livre issue de https://books.toscrape.com/index.html

Language: Python - Size: 17.6 KB - Last synced at: about 19 hours ago - Pushed at: about 20 hours ago - Stars: 2 - Forks: 0

jakopako/croncert-config

configuration and github actions for concertcloud.live (fka croncert.ch), a website that shows you concerts in various cities

Size: 8.91 MB - Last synced at: about 24 hours ago - Pushed at: 1 day ago - Stars: 12 - Forks: 6

html2rss/html2rss

📰 Build RSS 2.0 feeds from websites (and JSON APIs) automatically or with a few CSS selectors.

Language: Ruby - Size: 1.19 MB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 127 - Forks: 10

scp-data/scp-api

A dataset of SCP Items, Articles, and Metadata - Updated Daily

Language: Shell - Size: 9.65 GB - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 25 - Forks: 8

MontFerret/ferret

Declarative web scraping

Language: Go - Size: 4.26 MB - Last synced at: 2 days ago - Pushed at: 5 days ago - Stars: 5,808 - Forks: 305

tawfikhammad/Find-Matcher

An app searches on vector database about files matches uploaded file

Language: Python - Size: 1.12 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 0 - Forks: 0

Huolix/web-scraping-beautifulsoup

A Python project that scrapes product data using BeautifulSoup

Language: Jupyter Notebook - Size: 9.77 KB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 0 - Forks: 0

ac-i2i-engineering/access-amherst

An web app to connect Amherst College students to events in a more convenient, intimate, and diverse way! https://amherst-connect.com/

Language: Python - Size: 11.7 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 1 - Forks: 1

irf87/prices-scraper

Lightweight API & scraper for e-commerce on low-power devices, compatible with ARM

Language: JavaScript - Size: 336 KB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 0 - Forks: 0

Ne0bliviscaris/Job-Search-Tool

Organizer for job searching across multiple sites. Fetch offers, measure recruitment progress, collect info about potential employer

Language: Python - Size: 4.69 MB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 3 - Forks: 0

rafabelokurows/github-actions-workflows

Sandbox of automated workflows using GitHub Actions with Python and R code

Language: HTML - Size: 37.5 MB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 2 - Forks: 0

hhuayuan/spiderbuf

Spiderbuf 是一个专注于 Python 爬虫练习的网站。提供丰富的爬虫教程、爬虫案例解析和爬虫练习题。Python爬虫开发强化练习，在矛与盾的攻防中不断提高技术水平，通过大量的爬虫实战掌握常见的爬虫与反爬套路。引导式爬虫案例 + 免费爬虫视频教程，以闯关的形式挑战各个爬虫任务，培养爬虫开发的直觉及经验，验证自身爬虫开发与反爬虫实力的时候到了。

Language: Python - Size: 137 KB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 91 - Forks: 9

evarga/comments-aggregator

This is a teaching material aimed to demonstrate text analysis and generation capabilities of OpenAI.

Language: JavaScript - Size: 4.75 MB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 2 - Forks: 0

Ritesh-p/CaptchaBypass

CaptchaBypass

Size: 1000 Bytes - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 1 - Forks: 0

kibetbrian74/real-estate-data-scrapping-and-machine-learning

This Python script scrapes real estate data, simulates listings, merges datasets, visualizes trends, predicts prices with linear regression, and performs K-Means clustering on property data.

Language: Python - Size: 8.79 KB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 1 - Forks: 0

schmwong/APAC-McDelivery-Menu-Logger

Automatically scrapes McDelivery menu data and records it for future visualisation projects

Language: Python - Size: 36.9 MB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 6 - Forks: 2

MontFerret/worker

Containerized Ferret worker

Language: Go - Size: 1.68 MB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 14 - Forks: 7

erhangundogan/spider

Spider crawling the web

Language: Python - Size: 154 KB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 1 - Forks: 0

p3nnatr4tion/aliexpress-puppeteer

Dynamic Puppeteer Web Scraping for AliExpress Without API Key

Language: JavaScript - Size: 68.4 KB - Last synced at: 5 days ago - Pushed at: 6 days ago - Stars: 0 - Forks: 0

martial-god/Benny-Scraper

Webnovel and Manga Scraper that stores Webnovels as Epubs, and mangas as either PDFs of Comicbook Archives

Language: C# - Size: 761 KB - Last synced at: 4 days ago - Pushed at: about 2 months ago - Stars: 33 - Forks: 6

gustavofariaa/FlashscoreScraping

🤖 Web Scrape to Flashcore

Language: JavaScript - Size: 657 KB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 63 - Forks: 20

Kuria-byte/Scraping

Scraping and Enhancing Kenyan Leaders Data

Language: JavaScript - Size: 1.08 MB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 0 - Forks: 0

tathithienthanh/WomenFashionProductRecommendationSystem

Build a recommendation system for recommending woman fashion's products on e-commerce platforms

Language: Jupyter Notebook - Size: 45.1 MB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 0 - Forks: 0

josephlimtech/linkedin-profile-scraper-api

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON.

Language: TypeScript - Size: 10.8 MB - Last synced at: 7 days ago - Pushed at: about 1 year ago - Stars: 633 - Forks: 162

macbre/mobify

Download a webpage as an e-book

Language: Python - Size: 874 KB - Last synced at: 8 days ago - Pushed at: 8 days ago - Stars: 6 - Forks: 2

shurco/goClone

🌱 goClone - clone websites in seconds

Language: Go - Size: 7.1 MB - Last synced at: 8 days ago - Pushed at: 8 days ago - Stars: 69 - Forks: 4

fcoagz/pydolarvenezuela

Esta librería desarrollada en Python te brinda la capacidad de consultar los precios del dólar y/o euro en diversos monitores en Venezuela, también como las tasas de cambio del BCV.

Language: Python - Size: 757 KB - Last synced at: 1 day ago - Pushed at: 4 months ago - Stars: 51 - Forks: 11

mauricioarcez/Scraping-INDEC-IPC

Pipeline ETL automatizado para extraer datos del INDEC, procesar y publicar en Twitter la información destacada del IPC por regiones sin intervención humana con datos actualizados.

Language: Python - Size: 479 KB - Last synced at: 8 days ago - Pushed at: 8 days ago - Stars: 1 - Forks: 0

sainz1407/Sakurazaka-Blog-Discord-Bot

Automatically monitors Sakurazaka46 official blog and posts newly published images to Discord channels based on the blog author (member)

Language: Python - Size: 18.6 KB - Last synced at: 9 days ago - Pushed at: 9 days ago - Stars: 0 - Forks: 0

jirpo9/jobs.cz_web_scraper

Program, ktery scrapuje inzeraty z webu jobs.cz

Language: Python - Size: 25.8 MB - Last synced at: 9 days ago - Pushed at: 9 days ago - Stars: 0 - Forks: 0

MrBlue-Official-Account/Spatula

Web Emails Scraping

Language: Python - Size: 8.76 MB - Last synced at: 10 days ago - Pushed at: 10 days ago - Stars: 0 - Forks: 0

saulnyongesa/OpenSourceWebScraper

A Django-based web application that allows users to search for trending open-source content across multiple platforms including GitHub and Stack Overflow. The system returns relevant projects with descriptions and direct links.

Language: Python - Size: 54.7 KB - Last synced at: 11 days ago - Pushed at: 11 days ago - Stars: 0 - Forks: 0

edisedis777/Playwright-Web-Scraper

Web scraping framework built with Python and Playwright designed for reliable data extraction from websites with pagination.

Language: Python - Size: 30.3 KB - Last synced at: 11 days ago - Pushed at: 11 days ago - Stars: 1 - Forks: 0

NurulBashar/propertyfinder-scraping-analysis-visualization

propertyfinder-scraping-analysis-visualization

Language: Jupyter Notebook - Size: 1.4 MB - Last synced at: 13 days ago - Pushed at: 15 days ago - Stars: 0 - Forks: 0

marcoshsq/PY4E

This repository contains the projects and assignments developed during the Python for Everybody Specialization available on Coursera.

Language: Python - Size: 4.16 MB - Last synced at: 15 days ago - Pushed at: 15 days ago - Stars: 3 - Forks: 3

ricardocastanho/scrapify

A flexible and decoupled Go package for scraping paginated pages. It supports customizable scrapers, dynamic data processing through user-defined callbacks, and adjustable request intervals.

Language: Go - Size: 10.7 KB - Last synced at: 5 days ago - Pushed at: 16 days ago - Stars: 3 - Forks: 0

RuthGnz/SpyScrap

CLI and GUI for OSINT. Are you very exhibited on the Internet? Check it! Twitter, Tinder, Facebook, Google, Yandex, BOE. It uses facial recognition to provide more accurate results.

Language: Python - Size: 15.5 MB - Last synced at: 14 days ago - Pushed at: over 1 year ago - Stars: 181 - Forks: 27

Aroop-Singh/Selenium_scraper

"A collection of Selenium-based scripts to scrape data from job boards and e-commerce platforms, with CSV export support."

Language: Python - Size: 28.3 KB - Last synced at: 17 days ago - Pushed at: 17 days ago - Stars: 1 - Forks: 0

devlocalhost/pylyrical_api

pylyrical_api - lyrics API scraping from genius.com

Language: Python - Size: 115 KB - Last synced at: 17 days ago - Pushed at: 17 days ago - Stars: 2 - Forks: 0

Decodo/zalando_selenium_scraper

Scrape Zalando utilising Python, Selenium & Residential Proxies

Language: Python - Size: 10.7 KB - Last synced at: 14 days ago - Pushed at: 18 days ago - Stars: 2 - Forks: 1

GoranTopic/spider_jar

🫙🕷️ Webscrawling at scale 🕷️🫙

Language: TypeScript - Size: 246 KB - Last synced at: 18 days ago - Pushed at: 18 days ago - Stars: 0 - Forks: 0

davytheprogrammer/Scrapper

Powerful Web scrapper that saved the data scrapped from a website to Pdf

Language: Python - Size: 78.1 KB - Last synced at: 19 days ago - Pushed at: 19 days ago - Stars: 2 - Forks: 0

Anorov/cloudflare-scrape

A Python module to bypass Cloudflare's anti-bot page.

Language: Python - Size: 455 KB - Last synced at: 18 days ago - Pushed at: over 1 year ago - Stars: 3,459 - Forks: 456

chataize/rabbit-hole

C# library for scraping text content from websites.

Language: C# - Size: 95.7 KB - Last synced at: 19 days ago - Pushed at: 19 days ago - Stars: 1 - Forks: 0

esotericpig/nhkore

🇯🇵📰🗻 NHK News Web (Easy) word frequency (core list) scraper for Japanese language learners.

Language: Ruby - Size: 436 KB - Last synced at: 19 days ago - Pushed at: 19 days ago - Stars: 13 - Forks: 2

aydinnyunus/ScrapIn

LinkedIn Scraper

Language: Python - Size: 3.91 KB - Last synced at: 14 days ago - Pushed at: about 5 years ago - Stars: 18 - Forks: 4

hackersandslackers/beautifulsoup-tutorial

:sparkles: :ramen: Scrape webpage metadata using BeautifulSoup.

Language: Python - Size: 2.82 MB - Last synced at: 9 days ago - Pushed at: 9 days ago - Stars: 48 - Forks: 17

crwlrsoft/crawler

Library for Rapid (Web) Crawler and Scraper Development

Language: PHP - Size: 1.17 MB - Last synced at: 20 days ago - Pushed at: 20 days ago - Stars: 360 - Forks: 13

fernandod1/ProductHunt-scraper

Producthunt.com famous website scraper script. Scrap all offers and save in spreadsheet excel file.

Language: Python - Size: 9.77 KB - Last synced at: 10 days ago - Pushed at: almost 3 years ago - Stars: 26 - Forks: 9

Decodo/Web-Scraping-API

Web Scraping API code examples for Python, PHP and Node.js

Language: JavaScript - Size: 99.6 KB - Last synced at: 21 days ago - Pushed at: 21 days ago - Stars: 23 - Forks: 8

IndexOffy/tor-network-dataset

🐍 Tor-Network - Dataset

Size: 3.5 MB - Last synced at: 23 days ago - Pushed at: 23 days ago - Stars: 1 - Forks: 0

autogram-is/spidergram

Structural analysis tools for complex web sites

Language: TypeScript - Size: 1.64 MB - Last synced at: 5 days ago - Pushed at: about 2 months ago - Stars: 123 - Forks: 7

Mz-scripter/DJANGLE

DJANGLE is a specialized search engine for Django developers, offering curated results for questions, documentation, tutorials, and repositories.

Language: Python - Size: 17.2 MB - Last synced at: 5 days ago - Pushed at: about 1 month ago - Stars: 8 - Forks: 2

proxymesh/scrapy-proxy-headers

Handle custom proxy headers when making HTTPS requests through proxies in scrapy

Language: Python - Size: 8.79 KB - Last synced at: 25 days ago - Pushed at: 25 days ago - Stars: 2 - Forks: 0

i701/gazzette-api

Gazzette API with scraped data from Gazzette website.

Language: Python - Size: 34.2 KB - Last synced at: 25 days ago - Pushed at: 26 days ago - Stars: 3 - Forks: 0

avidLearnerInProgress/python-automation-scripts

Simple yet powerful automation stuffs.

Language: Python - Size: 8.55 MB - Last synced at: 19 days ago - Pushed at: over 4 years ago - Stars: 543 - Forks: 164

fernandod1/Instagram-downloader

Instagram user's photos and videos downloader. Download all media files from any username. Working 2022!

Language: Python - Size: 14.6 KB - Last synced at: 10 days ago - Pushed at: almost 3 years ago - Stars: 72 - Forks: 16

m92vyas/llm-reader

Turn Webpage to LLM friendly input text. Similar to Firecrawl and Jina Reader API. Makes RAG, AI web scraping, image & webpage links extraction easy.

Language: Python - Size: 85 KB - Last synced at: 28 days ago - Pushed at: 28 days ago - Stars: 158 - Forks: 14

Allama272/OLX-Mining-and-analyzing-house-prices

Automatically scraps and collects olx apartment data every 15 days

Language: Python - Size: 5.01 MB - Last synced at: 28 days ago - Pushed at: 28 days ago - Stars: 3 - Forks: 0

SubMaRk/mangascrapy

Download Manga, Manhwa, Manhua from any online website.

Language: Python - Size: 38.1 KB - Last synced at: 28 days ago - Pushed at: 28 days ago - Stars: 0 - Forks: 0

nava2105/soce_scraper

soce_scraper is a web application designed to extract procurement data from the Sistema Oficial de Contratación Pública del Ecuador (SOCE) of the processes related to CELEC EP.

Language: Python - Size: 59.6 KB - Last synced at: 29 days ago - Pushed at: 29 days ago - Stars: 1 - Forks: 0

maxmindlin/scout-lang

A web crawling programming language

Language: Rust - Size: 54.4 MB - Last synced at: 7 days ago - Pushed at: 9 months ago - Stars: 112 - Forks: 6

ThatSINEWAVE/Spy.pet-Info

This repository serves as an index for all info the community has gathered on the Spy.pet situation and as well as my own tables and tools written for these investigations. Spy.pet was taken down by Discord on 11.08.2024, this is just an archive of what bots where in each server.

Language: JavaScript - Size: 3.5 MB - Last synced at: 13 days ago - Pushed at: 3 months ago - Stars: 19 - Forks: 1

BennourNour/tunisie_annonce_srapping-python

Ce projet est une API REST développée avec FastAPI qui permet de scraper et gérer les annonces immobilières du site Tunisie Annonce. L'application automatise la collecte des données immobilières et les rend accessibles via une API structurée.

Language: Python - Size: 174 KB - Last synced at: 30 days ago - Pushed at: 30 days ago - Stars: 0 - Forks: 0

LearnCodingEasy/Web_Scraping

Web Scraping

Language: Vue - Size: 165 KB - Last synced at: 24 days ago - Pushed at: about 1 month ago - Stars: 1 - Forks: 0

gildas-lormeau/single-file-cli

CLI tool for saving a faithful copy of a complete web page in a single HTML file (based on SingleFile)

Language: JavaScript - Size: 5.16 MB - Last synced at: 29 days ago - Pushed at: about 1 month ago - Stars: 769 - Forks: 76

AmmeySaini/Edu-Mail-Generator

Generate Free Edu Mail(s) within minutes

Language: Python - Size: 82 KB - Last synced at: about 1 month ago - Pushed at: over 2 years ago - Stars: 808 - Forks: 396

fschaitel/ETLPipeline

ETL pipeline to analyze notebook listings on Mercado Livre using Scrapy, Pandas, SQLite, and Streamlit.

Language: Python - Size: 0 Bytes - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 1 - Forks: 0

spmedia/CTI-Stuffs

A collection of CTI stuffs - intel scraped from various cybercrime sources

Size: 360 KB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 0 - Forks: 0

elixir-crawly/crawly

Crawly, a high-level web crawling & scraping framework for Elixir.

Language: Elixir - Size: 2.8 MB - Last synced at: 30 days ago - Pushed at: 8 months ago - Stars: 1,020 - Forks: 117

Artboomy/patron_hamster

A content extractor for Patreon and Pixiv Fanbox sites.

Language: TypeScript - Size: 192 KB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 1 - Forks: 0

oxylabs/quick-start-guide

Python quick start guides to get the most out of Oxylabs' Web Scraper API free trial.

Size: 46.9 KB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 523 - Forks: 3

oxylabs/playwright-captcha

A guide on how to use Playwright to bypass CAPTCHA challenges using Python.

Size: 13.7 KB - Last synced at: 5 days ago - Pushed at: about 1 month ago - Stars: 3 - Forks: 0

baptisteArno/tinking 📦

🧶 Extract data from any website without code, just clicks.

Language: TypeScript - Size: 1.36 MB - Last synced at: about 1 month ago - Pushed at: about 4 years ago - Stars: 421 - Forks: 27

ENGRZULQARNAIN/ScrapySub

ScrapySub is a Python library designed to recursively scrape website content, including subpages. It fetches the visible text from web pages and stores it in a structured format for easy access and analysis. This library is particularly useful for NLP and AI developers who need to gather large amounts of web content for their projects.

Language: Python - Size: 25.4 KB - Last synced at: 10 days ago - Pushed at: 10 months ago - Stars: 4 - Forks: 0