Ecosyste.ms: Repos

An open API service providing repository metadata for many open source software ecosystems.

GitHub topics: scraping-websites

Repositories

crwlrsoft/crawler

Library for Rapid (Web) Crawler and Scraper Development

Language: PHP - Size: 845 KB - Last synced: 28 days ago - Pushed: about 2 months ago - Stars: 299 - Forks: 11

Ayloonah/diy_data_science

This is a project I worked on during college where I collect Pokemon deck data from Limitless.com and use said data to generate a 3D graph.

Language: HTML - Size: 1.21 MB - Last synced: 29 days ago - Pushed: 29 days ago - Stars: 0 - Forks: 0

the-homeless-god/NodeScreenshoter

Tool for take screenshot of each node on webpage

Language: JavaScript - Size: 213 KB - Last synced: 29 days ago - Pushed: 3 months ago - Stars: 0 - Forks: 0

naresh5033/Scrapy-python

The repo about the scrapy framework, and how to use it to scrape the sites and demo code

Language: Python - Size: 18.6 KB - Last synced: 29 days ago - Pushed: 11 months ago - Stars: 0 - Forks: 0

aswinshenoy/python-google-scraping

Google Scraping Using Python

Language: Python - Size: 8.79 KB - Last synced: 30 days ago - Pushed: over 5 years ago - Stars: 1 - Forks: 2

tobi303x/Web-crawling-tool-for-NGO

Web crawling GUI app for non-govermental events, originally created in 2023 as Uni project. Enjoy!

Language: Python - Size: 166 KB - Last synced: 30 days ago - Pushed: 30 days ago - Stars: 0 - Forks: 0

ricardocechet/scraping-text-image-table

Language: HTML - Size: 400 KB - Last synced: 30 days ago - Pushed: about 1 month ago - Stars: 1 - Forks: 0

lkuffo/web-scraping

Más de 50 ejemplos de web scraping utilizando: Requests | Scrapy | Selenium | LXML | BeautifulSoup

Language: Python - Size: 60.5 MB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 295 - Forks: 175

TomDoesTech/website-scraping-example

Language: HTML - Size: 351 KB - Last synced: about 1 month ago - Pushed: over 2 years ago - Stars: 3 - Forks: 1

22skowron/Sale-items-scraping

Scrapping with Selenium.

Language: Python - Size: 5.86 KB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 0 - Forks: 0

leogachimu/scraping_a_website_with_pagination_and_popups

The task involves scraping therapists' data from psychologytoday.com. Therapists per State and category are in multiple pages and the script should click on the view button to load and parse the page of a specific therapist.

Language: Jupyter Notebook - Size: 197 KB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 0 - Forks: 0

josejuansanchez/web-scraping-python

Ejemplos sencillos de cómo hacer web scraping en Python utilizados en el módulo de LM de 1º ASIR.

Language: Python - Size: 4.88 KB - Last synced: about 1 month ago - Pushed: almost 2 years ago - Stars: 2 - Forks: 0

lauramatilda/newsScraping

M.A. Thesis work, news scraping framework/pipeline using python, beautifulsoup, newspaper3k, flask and mongodb with a custom api.

Language: Python - Size: 34.2 KB - Last synced: about 1 month ago - Pushed: almost 6 years ago - Stars: 3 - Forks: 0

ahmed0saber/python_console

Language: Python - Size: 143 KB - Last synced: about 1 month ago - Pushed: over 2 years ago - Stars: 1 - Forks: 0

NoumanHere/Scraping-classcentral

Scraping classcentral.com with scrapy.

Language: Python - Size: 4.88 KB - Last synced: about 1 month ago - Pushed: about 4 years ago - Stars: 1 - Forks: 0

NoumanHere/Books-Scrapy

'http://books.toscrape.com/' Scraping books with their name, price,description, image_url.

Language: Python - Size: 550 KB - Last synced: about 1 month ago - Pushed: almost 4 years ago - Stars: 0 - Forks: 0

rafabelokurows/github-actions-r

Template of automated workflows using GitHub Actions with R code

Language: HTML - Size: 13.6 MB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 3 - Forks: 1

egin10/dapodik_go

Command Line App untuk scraping data sekolah dari web dapodik (Data Refrensi) : https://referensi.data.kemdikbud.go.id

Language: Go - Size: 29.4 MB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 0 - Forks: 0

Angel-Karasu/supermarket-scraper-api

An API to scrap several supermarkets

Language: Python - Size: 24.4 KB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 0 - Forks: 0

scp-data/scp-api

A dataset of SCP Items, Articles, and Metadata - Updated Daily

Language: Shell - Size: 5.29 GB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 13 - Forks: 3

schmwong/APAC-McDelivery-Menu-Logger

Automatically scrapes McDelivery menu data and records it for future visualisation projects

Language: Python - Size: 24.5 MB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 5 - Forks: 2

Sobak/scrawler

Declarative, scriptable web robot (crawler) and scrapper

Language: PHP - Size: 248 KB - Last synced: about 1 month ago - Pushed: about 4 years ago - Stars: 10 - Forks: 1

ljnath/getDomainAge

getDomainAge is a web application which can provide the age of a given domain name. The intention of this project is to demonstrate flask for API development, SQLAlchemy for ORM and beautifulsoup for HTML parsing.

Language: Python - Size: 3.48 MB - Last synced: 4 months ago - Pushed: 4 months ago - Stars: 2 - Forks: 0

GettingWhiter/squadronBot

A discord bot built with discord.py for War Thunder Squadrons actively participating in squadron battles (SQBs/SREs - Squadron Realistic Events).

Language: Python - Size: 80.1 KB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 0 - Forks: 0

its0x08/duckduckgo

A simple DuckDuckGo URL scraper.

Language: Python - Size: 43.9 KB - Last synced: 27 days ago - Pushed: 3 months ago - Stars: 21 - Forks: 3

AmmeySaini/Edu-Mail-Generator

Generate Free Edu Mail(s) within minutes

Language: Python - Size: 82 KB - Last synced: about 1 month ago - Pushed: over 1 year ago - Stars: 789 - Forks: 408

aashishops/Football_data-Web_scrapping

Football Data web scrapping

Language: Jupyter Notebook - Size: 112 KB - Last synced: about 1 month ago - Pushed: 8 months ago - Stars: 2 - Forks: 0

MontFerret/ferret

Declarative web scraping

Language: Go - Size: 3.87 MB - Last synced: about 1 month ago - Pushed: about 2 months ago - Stars: 5,615 - Forks: 301

csbun/thal Fork of emadehsan/thal

译文：Puppeteer 与 Chrome Headless —— 从入门到爬虫

Language: JavaScript - Size: 755 KB - Last synced: about 1 month ago - Pushed: over 5 years ago - Stars: 655 - Forks: 49

HayatiYrtgl/pc_prices_scraper

Size: 16.6 KB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 0 - Forks: 0

radical-data/arte-util-scraper

Software to scrape the Arte Útil archive.

Language: Python - Size: 2.93 KB - Last synced: 11 days ago - Pushed: almost 2 years ago - Stars: 0 - Forks: 0

avidLearnerInProgress/python-automation-scripts

Simple yet powerful automation stuffs.

Language: Python - Size: 8.55 MB - Last synced: about 1 month ago - Pushed: over 3 years ago - Stars: 515 - Forks: 161

TheNoiselessNoise/csfd_scraper

Simple scraper for CSFD.cz, a Czech movie database.

Language: Python - Size: 138 KB - Last synced: 3 days ago - Pushed: 2 months ago - Stars: 2 - Forks: 1

ChenlongXu/Github_trending_data

This is a daily updated trial project for web scraping everyday Github trending data and save them in an excel file for that date.

Language: Python - Size: 1.13 MB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 0 - Forks: 0

puutaro/newsSpeecher

News speech by scraping in CommandClick

Language: JavaScript - Size: 30.3 KB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 0 - Forks: 0

farizkiyazid/Mega-Credit-Promo-Scraper

Web scraper that scrapes credit card promos from Bank Mega. (NodeJs)

Language: JavaScript - Size: 860 KB - Last synced: about 1 month ago - Pushed: almost 2 years ago - Stars: 1 - Forks: 0

binDebug3/Casper

(2023) A web scraping program to generate personalized used car recommendations using Deep Learning by Dallin Stewart

Language: Jupyter Notebook - Size: 44 MB - Last synced: about 1 month ago - Pushed: about 1 year ago - Stars: 4 - Forks: 0

kunalnagarco/imdb-scraper 📦

🎬 An attempt at the most complete IMDb API

Language: TypeScript - Size: 1.06 MB - Last synced: 23 days ago - Pushed: 4 months ago - Stars: 34 - Forks: 7

RyuzakiH/CloudflareSolverRe

Cloudflare Javascript & reCaptcha challenge (I'm Under Attack Mode or IUAM) solving / bypass .NET Standard library.

Language: C# - Size: 229 KB - Last synced: 20 days ago - Pushed: almost 3 years ago - Stars: 173 - Forks: 49

rosadigital/Tax-Instruction-Scrap

Scrap the public information on Brazilian Federal Revenue Office website, using OOP, BS and Selenium.

Language: Python - Size: 55.2 MB - Last synced: about 1 month ago - Pushed: over 1 year ago - Stars: 0 - Forks: 0

umasankar1999/Website-Scraping

Language: Python - Size: 1000 Bytes - Last synced: about 1 month ago - Pushed: almost 6 years ago - Stars: 0 - Forks: 0

Smartproxy/Web-Scraping-API

Web Scraping API code examples for Python, PHP and Node.js

Language: JavaScript - Size: 99.6 KB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 21 - Forks: 5

Smartproxy/eCommerce-Scraping-API

eCommerce Scraping API code examples for Python, PHP and Node.js

Language: PHP - Size: 151 KB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 21 - Forks: 4

HopefulHeart2020/zoominfo_scraper

Zoominfo scraper with using of rotating proxies and headless Chrome from ScrapingAnt

Language: Python - Size: 6.84 KB - Last synced: about 1 month ago - Pushed: 8 months ago - Stars: 1 - Forks: 0

sujitmandal/scrape-search-engine

Search anything on the different Search Engine's it will collect all the links.

Language: Python - Size: 49.8 KB - Last synced: 14 days ago - Pushed: 11 months ago - Stars: 14 - Forks: 5

josephlimtech/linkedin-profile-scraper-api

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON.

Language: TypeScript - Size: 10.8 MB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 466 - Forks: 133

RuthGnz/SpyScrap

CLI and GUI for OSINT. Are you very exhibited on the Internet? Check it! Twitter, Tinder, Facebook, Google, Yandex, BOE. It uses facial recognition to provide more accurate results.

Language: Python - Size: 15.5 MB - Last synced: about 1 month ago - Pushed: 6 months ago - Stars: 140 - Forks: 27

Esubaalew/AAUROBOT

Simple Telegram bot for Addis Ababa University Students

Language: Python - Size: 110 KB - Last synced: about 2 months ago - Pushed: 2 months ago - Stars: 1 - Forks: 0

behitek/social-scraper

Vietnamese text data crawler scripts for various sites (including Youtube, Facebook, 4rum, news, ...)

Language: Python - Size: 22 MB - Last synced: about 2 months ago - Pushed: over 1 year ago - Stars: 65 - Forks: 37

thoth2357/Dynamic-Job-Board-Application

A jobBoard Web Application That helps scrape Jobs From multiple Sites at Certain Set Time And displays it

Language: HTML - Size: 17.4 MB - Last synced: about 2 months ago - Pushed: about 2 months ago - Stars: 1 - Forks: 0

vkbandari/internshala_scraping

scraping of internships from internshala using python

Language: Jupyter Notebook - Size: 12.3 MB - Last synced: about 2 months ago - Pushed: almost 4 years ago - Stars: 1 - Forks: 0

mrseanryan/style-scraper

Scrape styles from a website using reliable computed styling via a headless browser [fonts, colors, border styles ...]

Language: JavaScript - Size: 432 KB - Last synced: 30 days ago - Pushed: 3 months ago - Stars: 1 - Forks: 0

AtalaySezen/Google-Email-Scraper

Google Email scraper is a web scraper that searches google search result based on a specific keyword to find email adresses.

Language: JavaScript - Size: 7.76 MB - Last synced: about 2 months ago - Pushed: about 2 months ago - Stars: 2 - Forks: 0

qiyoulu/wiki-scraper

scrapes wikipedia articles from a list and plots their keywords using bs4 + nlpir

Language: Python - Size: 1.37 MB - Last synced: about 2 months ago - Pushed: almost 2 years ago - Stars: 0 - Forks: 0

TheJaeLal/buyredminote4

An automated checkout script to buy redmi note 4 from flipkart.

Language: Python - Size: 9.75 MB - Last synced: about 2 months ago - Pushed: almost 7 years ago - Stars: 4 - Forks: 2

MaxBubblegum47/VictoriaCinemaBot

Little bot that displays information about the latest movies in my favourite movie theater

Language: HTML - Size: 8.79 MB - Last synced: about 2 months ago - Pushed: about 2 months ago - Stars: 0 - Forks: 0

Covantec/examples.diazo

Ejemplo del tutorial "Creando Temas con diazo" de publicado en la Documentación oficial de Plone en Español

Size: 234 KB - Last synced: about 2 months ago - Pushed: over 9 years ago - Stars: 0 - Forks: 0

HackSocNotts/membership-scraper

Scraping service for SUMS

Language: TypeScript - Size: 1.35 MB - Last synced: about 2 months ago - Pushed: over 1 year ago - Stars: 0 - Forks: 1

thijswillemmoens/scraping_nba_stats

Trying to do some data scraping from the Basketball Reference website.

Language: Jupyter Notebook - Size: 473 KB - Last synced: about 2 months ago - Pushed: about 2 months ago - Stars: 0 - Forks: 0

scrapehero/yelp-search

Code to search for business listings on yelp.com

Language: Python - Size: 1.95 KB - Last synced: about 2 months ago - Pushed: over 5 years ago - Stars: 3 - Forks: 3

JusticeRage/ApkTrack 📦

ApkTrack is an Android app which checks if updates for installed APKs are available.

Language: Java - Size: 1.92 MB - Last synced: about 2 months ago - Pushed: almost 2 years ago - Stars: 121 - Forks: 27

danangfir/playwrght-scrap

Ini contoh buat scrap halaman https://bachacoffee.com/en/coffee-beans

Language: JavaScript - Size: 3.91 KB - Last synced: about 2 months ago - Pushed: about 2 months ago - Stars: 0 - Forks: 0

pnguyen215/instagram-crawler

Instagram Crawler is a Python script to download posts from a specified Instagram account.

Language: Python - Size: 20.5 KB - Last synced: about 2 months ago - Pushed: about 2 months ago - Stars: 1 - Forks: 0

GoodbyeKittyy/Web-Scraper-For-Cagematch-Website

An improved way to find and filter professional wrestling matches

Size: 0 Bytes - Last synced: about 2 months ago - Pushed: about 2 months ago - Stars: 0 - Forks: 0

e43b/Kemono-Downloader

This script allows you to download images from posts on the Kemono website https://kemono.su/ and https://kemono.party/

Language: Python - Size: 95.7 KB - Last synced: about 2 months ago - Pushed: about 2 months ago - Stars: 1 - Forks: 1

fa8i/Web_Scraping

Scripts de ejemplo para extraccion de texto, tablas o imagenes de diferentes webs.

Language: Jupyter Notebook - Size: 11.7 KB - Last synced: about 2 months ago - Pushed: about 2 months ago - Stars: 0 - Forks: 0

paulapereda/AnalyzingMoweekTrendsSummer

"Data but make it fashion" presented at the LatinR conference. LatinR is the Latin American Conference on the Use of R in Research + Development.

Language: R - Size: 68.5 MB - Last synced: about 2 months ago - Pushed: about 2 months ago - Stars: 0 - Forks: 0

jpablogomezb/Simple-Web-Scraping

A simple python script to scrape a website, parse data, count words and store data into a local CSV file.

Language: Python - Size: 13.7 KB - Last synced: about 2 months ago - Pushed: about 3 years ago - Stars: 0 - Forks: 0

murilogmamaral/brasileiro

Comparing points of Brazilian football teams in the last four seasons

Language: R - Size: 255 KB - Last synced: about 2 months ago - Pushed: about 2 years ago - Stars: 0 - Forks: 0

AndyTheFactory/article-extraction-dataset

Article title, authors, date and body extraction dataset.

Language: HTML - Size: 31.9 MB - Last synced: about 2 months ago - Pushed: about 2 months ago - Stars: 1 - Forks: 0

lyndon160/cpe-scraper

Script for scraping websites

Language: Python - Size: 78.1 KB - Last synced: 2 months ago - Pushed: over 5 years ago - Stars: 1 - Forks: 0

NelcifranMagalhaes/SharinganApi

An API in rails

Language: Ruby - Size: 64.5 KB - Last synced: 2 months ago - Pushed: over 1 year ago - Stars: 0 - Forks: 0

VanHakobyan/Scrapping

Scrapping

Language: C# - Size: 53.7 MB - Last synced: 2 months ago - Pushed: over 1 year ago - Stars: 4 - Forks: 0

outscraper/google-maps-reviews-python

Python SDK that allows fetching Google Maps Reviews from any place. Fetch reviews from any business on Google Maps without limitations.

Language: Python - Size: 15.6 KB - Last synced: about 1 month ago - Pushed: over 2 years ago - Stars: 6 - Forks: 2

nitinjosephrepo/scrapy-crawl-spider

Project using Scrapy spider with Rules that crawls Website/Pages of Interest and extracts Product info, pagination is dealt by configuring Link Extractor method

Language: Python - Size: 6.84 KB - Last synced: 2 months ago - Pushed: about 3 years ago - Stars: 0 - Forks: 0

shreydan/Unsplash-Scraper

downloads specified no. of photos with credits list from Unsplash

Language: Python - Size: 4.88 KB - Last synced: 2 months ago - Pushed: over 3 years ago - Stars: 1 - Forks: 4

dancab13/web-scraping

Materials for the web scraping Module 11 challenge for the BCS Data Analytics boot camp

Language: Jupyter Notebook - Size: 188 KB - Last synced: 2 months ago - Pushed: 2 months ago - Stars: 0 - Forks: 0

teilhard8/Automatic-Udemy-Course-Enroller-GET-PAID-UDEMY-COURSES-for-FREE Fork of aapatre/Automatic-Udemy-Course-Enroller-GET-PAID-UDEMY-COURSES-for-FREE

Do you want to LEARN NEW STUFF for FREE? Don't worry, with the power of web-scraping and automation, this script will find the necessary Udemy coupons & enroll you for PAID UDEMY COURSES, ABSOLUTELY FREE!

Size: 138 KB - Last synced: 2 months ago - Pushed: over 3 years ago - Stars: 0 - Forks: 0

DataKind-BLR/QuestAlliance 📦

DKBLR with collaboration with Quest Alliance. Blue collar job related stuff.

Language: HTML - Size: 7.25 MB - Last synced: 2 months ago - Pushed: over 3 years ago - Stars: 0 - Forks: 0

rudrakshsyal/Craigslist-Job-Listing-Transformation-via-Text-Modeling

Improved quality and presentation of job listings on Craigslist website via scraping and training data from Indeed’s job listings’, to enhance user experience, drive more traffic and thus increase revenue

Language: Jupyter Notebook - Size: 4.54 MB - Last synced: 2 months ago - Pushed: over 5 years ago - Stars: 1 - Forks: 0

EnthusiasticXcoder/Django-carrierAPI

Django-carrierAPI is a powerful RESTful API built with Django and Django Rest Framework for user authentication and serving blog and career data. With token-based authentication and standardized endpoints, it simplifies user authentication and data retrieval for client applications.

Language: Python - Size: 651 KB - Last synced: about 2 months ago - Pushed: about 2 months ago - Stars: 0 - Forks: 0

codeberrypro/my_parsers

This is a small scripts for parsing data from the site. Where I bypassed the captcha, used cookies and bypassed the site's security. I hope the code will be useful for beginners and not only

Language: Python - Size: 6.43 MB - Last synced: 2 months ago - Pushed: 2 months ago - Stars: 6 - Forks: 0

ed-word/Linkedin-Scraper

Selenium based LinkedIn profile data scraper

Language: Python - Size: 8.79 KB - Last synced: 2 months ago - Pushed: over 6 years ago - Stars: 12 - Forks: 0

devallasaitej/WebScrapers

Python code for scraping Amazon Bestselling Book reviews, BestBuy product reviews

Language: Jupyter Notebook - Size: 1.58 MB - Last synced: 2 months ago - Pushed: 2 months ago - Stars: 0 - Forks: 0

noobscode/kalel 📦

Kal El Network Stress Test and Penetration Testing Toolkit

Language: Python - Size: 4.46 MB - Last synced: 2 months ago - Pushed: 2 months ago - Stars: 57 - Forks: 18

sharad1126/WebScrapper

web scrapping with selenium using chrome driver

Language: Python - Size: 217 KB - Last synced: 16 days ago - Pushed: 11 months ago - Stars: 3 - Forks: 2

07Rinat07/autoparser

tasktest

Language: PHP - Size: 3.91 KB - Last synced: 2 months ago - Pushed: 2 months ago - Stars: 0 - Forks: 0

madipta/web-scraping

web scraping and search portal using postgres full-text search, web admin built with angular + ng-zorro, web portal built with reactjs + tailwindcss

Language: TypeScript - Size: 5.29 MB - Last synced: 2 months ago - Pushed: 2 months ago - Stars: 13 - Forks: 0

zefr0x/python-telegram2rss

A python library to fetch data from public Telegram channels to use them as a python object or RSS feed.

Language: Python - Size: 82 KB - Last synced: 12 days ago - Pushed: 18 days ago - Stars: 7 - Forks: 0

w0ltage/subchase

Chase subdomains by parsing the results of Google and Yandex search results

Language: Go - Size: 517 KB - Last synced: 5 days ago - Pushed: 8 months ago - Stars: 16 - Forks: 0

ArkaNiightt/scraper_vagas_bot

Este repositório contém um script para varrer um site específico em busca de vagas de emprego. O script coleta e organiza informações sobre oportunidades disponíveis neste site, facilitando a busca por emprego para os candidatos interessados. Os usuários podem personalizar a busca de acordo com suas preferências e qualificações específicas.

Language: Python - Size: 73.2 KB - Last synced: 2 months ago - Pushed: 2 months ago - Stars: 1 - Forks: 0