Ecosyste.ms: Repos

An open API service providing repository metadata for many open source software ecosystems.

GitHub topics: scraping-websites

Repositories

html2rss/html2rss

📰 Build RSS 2.0 feeds from websites (and JSON APIs) with a few CSS selectors.

Language: Ruby - Size: 908 KB - Last synced: about 15 hours ago - Pushed: about 16 hours ago - Stars: 111 - Forks: 9

rahulbhoyar1995/Web-Scraping-with-Scrapy

Repository showcasing diverse web scraping projects, extracting valuable data from websites for analysis and insights. #WebScraping #DataExtraction

Language: Jupyter Notebook - Size: 916 KB - Last synced: 38 minutes ago - Pushed: 1 day ago - Stars: 0 - Forks: 0

kodiondemand/addon Fork of lozioangie/addon

Kodi on Demand addon main repository. Kodi is a registered trademark of the XBMC Foundation.

Language: Python - Size: 75.4 MB - Last synced: 1 day ago - Pushed: 1 day ago - Stars: 62 - Forks: 55

brittojo7n/TotemTribe2-UserDetails

Totem Tribe 2 Jotun User details (ids and account names) dataset and script for scraping and updating the dataset.

Language: Python - Size: 7.98 MB - Last synced: 1 day ago - Pushed: 1 day ago - Stars: 0 - Forks: 0

hridaydutta123/the-youtube-scraper

Download YouTube video description and video comments without using the YouTube API.

Language: Python - Size: 313 KB - Last synced: 1 day ago - Pushed: 1 day ago - Stars: 125 - Forks: 19

GoranTopic/spider_jar

🫙🕷️ Webscrawling with at scale 🫙🕷️

Language: TypeScript - Size: 246 KB - Last synced: 1 day ago - Pushed: 2 days ago - Stars: 0 - Forks: 0

Allama272/OLX-Mining-and-analyzing-house-prices

Automatically scraps and collects olx apartment data every 15 days

Language: Python - Size: 2.5 MB - Last synced: 1 day ago - Pushed: 2 days ago - Stars: 4 - Forks: 0

bobikhsann/indonesia-election-news

displays 2024 election news data from web scraping at detiknews.com

Language: Jupyter Notebook - Size: 601 KB - Last synced: 2 days ago - Pushed: 3 days ago - Stars: 0 - Forks: 0

lunarwhite/hello-scrapy-mongo

A simple crawler for stackoverflow newest questions.

Language: Python - Size: 45.9 KB - Last synced: 2 days ago - Pushed: 3 days ago - Stars: 0 - Forks: 1

macbre/mobify

Download a webpage as an e-book

Language: Python - Size: 464 KB - Last synced: 2 days ago - Pushed: 3 days ago - Stars: 5 - Forks: 2

borbiuk/band-spider

Bandcamp sraper: extracd data from bandcamp website

Language: TypeScript - Size: 127 MB - Last synced: 3 days ago - Pushed: 3 days ago - Stars: 3 - Forks: 0

hackersandslackers/beautifulsoup-tutorial

:sparkles: :ramen: Scrape webpage metadata using BeautifulSoup.

Language: Python - Size: 2.83 MB - Last synced: 3 days ago - Pushed: 3 days ago - Stars: 48 - Forks: 19

evarga/comments-aggregator

This is a teaching material aimed to demonstrate text analysis and generation capabilities of OpenAI.

Language: JavaScript - Size: 4.81 MB - Last synced: 4 days ago - Pushed: 4 days ago - Stars: 2 - Forks: 0

alexis-brosseau/Ebay-Scraper

Get the average price of a product on eBay or a list of products on sale.

Language: Python - Size: 62.5 KB - Last synced: 4 days ago - Pushed: 4 days ago - Stars: 11 - Forks: 4

jakopako/croncert-config

configuration and github actions for concertcloud.live (fka croncert.ch), a website that shows you concerts in various cities

Size: 8.75 MB - Last synced: 4 days ago - Pushed: 5 days ago - Stars: 10 - Forks: 4

ZOMBOTRON/reegex

- O projeto consiste em coletar dados de várias cidades do Tocantins usando web scraping.- Os dados coletados incluem o nome da cidade, população, gentílico e território.

Language: JavaScript - Size: 25.4 KB - Last synced: 5 days ago - Pushed: 6 days ago - Stars: 0 - Forks: 0

davidgao7/research_assistant

your personal research assistant!

Language: Python - Size: 8.79 KB - Last synced: 4 days ago - Pushed: 7 days ago - Stars: 0 - Forks: 0

outscraper/outscraper-node

The library provides convenient access to the Outscraper API from applications written in server-side JavaScript. Allows using Outscraper's services from your code.

Language: JavaScript - Size: 37.1 KB - Last synced: 10 days ago - Pushed: 10 days ago - Stars: 5 - Forks: 0

fcoagz/pydolarvenezuela

Esta librería desarrollada en Python te brinda la capacidad de consultar los precios del dólar y/o euro en diversos monitores en Venezuela, también como las tasas de cambio del BCV.

Language: Python - Size: 443 KB - Last synced: 9 days ago - Pushed: 10 days ago - Stars: 25 - Forks: 6

HubTou/libgh

GitHub scraping tool and library

Language: Python - Size: 119 KB - Last synced: 10 days ago - Pushed: 10 days ago - Stars: 0 - Forks: 1

gustavofariaa/FlashscoreScraping

🤖 Web Scrape to Flashcore

Language: JavaScript - Size: 94.7 KB - Last synced: 10 days ago - Pushed: 10 days ago - Stars: 22 - Forks: 6

SilentDemonSD/FZBypassBot

A Elegant Fast Multi Threaded Bypass Bot for Bigger Deeds. Try Now !!

Language: Python - Size: 281 KB - Last synced: 10 days ago - Pushed: 10 days ago - Stars: 65 - Forks: 145

jonesrussell/page-prowler

Experimental mashup of golang commands to scrape the web

Language: Go - Size: 10.8 MB - Last synced: 10 days ago - Pushed: 10 days ago - Stars: 0 - Forks: 1

timan-z/Scrape-Letterboxd-Ratings-with-Flask-on-Chrome

This Chrome extension leverages the Flask web framework for Python, as well as the automated-testing tool Selenium, to scrape film ratings from a given Letterboxd profile. Upon clicking the extension icon while on a valid Letterboxd profile, the extension fetches the ratings data, converts it into a CSV file and downloads it in the current browser.

Language: Python - Size: 5.04 MB - Last synced: 10 days ago - Pushed: 11 days ago - Stars: 0 - Forks: 0

UtshoData/BestMovieDetailsAnalyticsUsingTableau-main-

Best Movie Details analysis using Tableau

Language: Python - Size: 55.5 MB - Last synced: 10 days ago - Pushed: 11 days ago - Stars: 0 - Forks: 0

Indigo-Coder-github/Korean_News_Crawler

Python Library for Crawling News Artircles in Korean Top 10 News Websites with Utilities

Language: Python - Size: 53.7 KB - Last synced: 11 days ago - Pushed: 11 days ago - Stars: 0 - Forks: 0

ThatSINEWAVE/Spy.pet-Info

This repository serves as an index for all info the community has gathered on the Spy.pet situation and as well as my own tables and tools written for these investigations.

Language: JavaScript - Size: 3.48 MB - Last synced: 12 days ago - Pushed: 12 days ago - Stars: 5 - Forks: 0

d3sc/anime-scraping

Scraping anime website with Javascript

Language: JavaScript - Size: 0 Bytes - Last synced: 13 days ago - Pushed: 13 days ago - Stars: 0 - Forks: 0

sudoDeVinci/MAPper

MAPper: Twitter Follower-Network Visualizer using The Tweepy Library

Language: Python - Size: 461 MB - Last synced: 13 days ago - Pushed: 13 days ago - Stars: 2 - Forks: 0

rike422/kirinuki-cli

Language: TypeScript - Size: 1.85 MB - Last synced: 13 days ago - Pushed: 13 days ago - Stars: 2 - Forks: 0

codegratia/react-node-web-scraper

Final Year project, scraping data of e-commerce stores and display in ReactJS app.

Language: JavaScript - Size: 13.9 MB - Last synced: 13 days ago - Pushed: 13 days ago - Stars: 38 - Forks: 14

unixfox/pupflare

A webpage proxy that request through Chromium (puppeteer) - can be used to bypass Cloudflare anti bot / anti ddos on any application (like curl)

Language: JavaScript - Size: 39.1 KB - Last synced: 14 days ago - Pushed: 14 days ago - Stars: 321 - Forks: 78

biplavpoudel/TMDB-Webscraper-using-RPA

A webscraping bot that uses RPA framework and scraps movie data from tmdb.org based on the movie names provided.

Language: Python - Size: 70.3 KB - Last synced: 14 days ago - Pushed: 14 days ago - Stars: 0 - Forks: 0

yacineberkani/scraping-ArXiv

Ce code est conçu pour télécharger des articles de la plateforme arXiv en fonction d'une requête donnée par l'utilisateur

Language: Jupyter Notebook - Size: 38.1 KB - Last synced: 14 days ago - Pushed: 14 days ago - Stars: 0 - Forks: 0

corentinbouton/ETF-scrapper

Automated web scraper for extracting information about Exchange-Traded Funds (ETFs) eligible for investment in the French PEA account from the Boursorama website.

Language: Python - Size: 17.6 KB - Last synced: 15 days ago - Pushed: 15 days ago - Stars: 0 - Forks: 0

zefr0x/python-telegram2rss

A python library to fetch data from public Telegram channels to use them as a python object or RSS feed.

Language: Python - Size: 82 KB - Last synced: 12 days ago - Pushed: 18 days ago - Stars: 7 - Forks: 0

elixir-crawly/crawly

Crawly, a high-level web crawling & scraping framework for Elixir.

Language: Elixir - Size: 2.79 MB - Last synced: 16 days ago - Pushed: 18 days ago - Stars: 842 - Forks: 104

lautarovculic/zScrap

Simple Python scripts to do Web Scraping when I'm bored.

Language: Python - Size: 10.7 KB - Last synced: 19 days ago - Pushed: 19 days ago - Stars: 1 - Forks: 0

SubMaRk/scrapy

Download any Manga Manhwa Manhua from online website with Custom Extension.

Language: Python - Size: 91.8 KB - Last synced: 20 days ago - Pushed: 20 days ago - Stars: 1 - Forks: 1

FabienBounoir/Lightshot-Lows

Think twice before using Lightshot! Our investigations reveal that all captures taken with Lightshot are automatically made public on the web for a lifetime. Protect your privacy and opt for a more secure screenshot solution. Your moments should stay yours, not become public domain!

Language: Svelte - Size: 2.05 MB - Last synced: 16 days ago - Pushed: 20 days ago - Stars: 0 - Forks: 0

LawrenceQuarshie/DataScience

This is a repository of plotting, charting, data visualization, manipulation, analytics and anything about data in typical engineering setting.

Language: HTML - Size: 3.91 KB - Last synced: 21 days ago - Pushed: 21 days ago - Stars: 0 - Forks: 0

ozangulll/PythonScraping

Language: Python - Size: 7.81 KB - Last synced: 13 days ago - Pushed: 21 days ago - Stars: 0 - Forks: 0

GiorgosG1an/SongLyricsSearchEngine

Using pylucene in order to search songs and albums based on artist, song name, lyrics, album name. Songs and album data from AZlyrics

Language: Python - Size: 20.6 MB - Last synced: 21 days ago - Pushed: 21 days ago - Stars: 1 - Forks: 0

AhmedAniba/WebScrapping

Web scrapping & Excel extraction

Language: Python - Size: 4.88 KB - Last synced: 22 days ago - Pushed: 22 days ago - Stars: 0 - Forks: 0

Ali-Hasan-Khan/Scrape-Whoscored-Event-Data

Get match event data from whoscored.com

Language: Jupyter Notebook - Size: 37.3 MB - Last synced: 22 days ago - Pushed: 22 days ago - Stars: 22 - Forks: 6

samircd4/Brainy-Quotes-Scrapy

Scrap 190k data from https://www.brainyquote.com/

Language: Python - Size: 7.25 MB - Last synced: 25 days ago - Pushed: 25 days ago - Stars: 1 - Forks: 0

ayoubzulfiqar/go-scraper

This repo show how to Scrape different type of data

Language: Go - Size: 31.3 KB - Last synced: 28 days ago - Pushed: 28 days ago - Stars: 0 - Forks: 0

oxylabs/scrape-google-shopping

Learn how to gather accurate real-time data from Google Shopping hassle-free.

Language: Python - Size: 1.07 MB - Last synced: 27 days ago - Pushed: 28 days ago - Stars: 6 - Forks: 1

Ayloonah/diy_data_science

This is a project I worked on during college where I collect Pokemon deck data from Limitless.com and use said data to generate a 3D graph.

Language: HTML - Size: 1.21 MB - Last synced: 29 days ago - Pushed: 29 days ago - Stars: 0 - Forks: 0

tobi303x/Web-crawling-tool-for-NGO

Web crawling GUI app for non-govermental events, originally created in 2023 as Uni project. Enjoy!

Language: Python - Size: 166 KB - Last synced: 30 days ago - Pushed: 30 days ago - Stars: 0 - Forks: 0

ricardocechet/scraping-text-image-table

Language: HTML - Size: 400 KB - Last synced: 30 days ago - Pushed: about 1 month ago - Stars: 1 - Forks: 0

sxge-repo/mbti-profession

Ever wonder which professions attract certain types of personality? Find out in this project.

Language: Jupyter Notebook - Size: 2 MB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 0 - Forks: 1

lkuffo/web-scraping

Más de 50 ejemplos de web scraping utilizando: Requests | Scrapy | Selenium | LXML | BeautifulSoup

Language: Python - Size: 60.5 MB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 295 - Forks: 175

22skowron/Sale-items-scraping

Scrapping with Selenium.

Language: Python - Size: 5.86 KB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 0 - Forks: 0

ChrisMichaelPerezSantiago/ryuanime

A free anime streaming , using the jkanime content by scraping the jkanime website.

Language: Vue - Size: 3.61 MB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 21 - Forks: 13

Sermuns/undervisningsnummer

Se antalet föreläsningar, lektioner, etc som har passerat i en kurs på Linköping universitet.

Language: JavaScript - Size: 416 KB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 2 - Forks: 0

leogachimu/scraping_a_website_with_pagination_and_popups

The task involves scraping therapists' data from psychologytoday.com. Therapists per State and category are in multiple pages and the script should click on the view button to load and parse the page of a specific therapist.

Language: Jupyter Notebook - Size: 197 KB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 0 - Forks: 0

schmwong/APAC-McDelivery-Menu-Logger

Automatically scrapes McDelivery menu data and records it for future visualisation projects

Language: Python - Size: 24.5 MB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 5 - Forks: 2

Angel-Karasu/supermarket-scraper-api

An API to scrap several supermarkets

Language: Python - Size: 24.4 KB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 0 - Forks: 0

rafabelokurows/github-actions-r

Template of automated workflows using GitHub Actions with R code

Language: HTML - Size: 13.6 MB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 3 - Forks: 1

scp-data/scp-api

A dataset of SCP Items, Articles, and Metadata - Updated Daily

Language: Shell - Size: 5.29 GB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 13 - Forks: 3

kinoute/scraper-allocine

Just playing with BeautifulSoup and Python to scrap some movies on Allocine.fr.

Language: Python - Size: 74.2 KB - Last synced: 16 days ago - Pushed: about 1 month ago - Stars: 1 - Forks: 1

GettingWhiter/squadronBot

A discord bot built with discord.py for War Thunder Squadrons actively participating in squadron battles (SQBs/SREs - Squadron Realistic Events).

Language: Python - Size: 80.1 KB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 0 - Forks: 0

ChenlongXu/Github_trending_data

This is a daily updated trial project for web scraping everyday Github trending data and save them in an excel file for that date.

Language: Python - Size: 1.13 MB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 0 - Forks: 0

egin10/dapodik_go

Command Line App untuk scraping data sekolah dari web dapodik (Data Refrensi) : https://referensi.data.kemdikbud.go.id

Language: Go - Size: 29.4 MB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 0 - Forks: 0

HayatiYrtgl/pc_prices_scraper

Size: 16.6 KB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 0 - Forks: 0

Ashbyt/Python

Ashley Bythell - Python

Language: Jupyter Notebook - Size: 5.68 MB - Last synced: 17 days ago - Pushed: about 1 month ago - Stars: 1 - Forks: 0

spekulatius/PHPScraper

A universal web-util for PHP.

Language: PHP - Size: 6.53 MB - Last synced: 14 days ago - Pushed: about 1 month ago - Stars: 497 - Forks: 72

Ximaz/instagram-api

This project is an Instagram API guest-oriented, meaning you can perform actions of data collection without having to register. However, you may be facing some Ratelimits, so I recommend you to put some delay between operations.

Language: Python - Size: 776 KB - Last synced: 16 days ago - Pushed: about 1 month ago - Stars: 6 - Forks: 1

hebertcisco/pensador-promise

Frases do Pensador via WebScraping: https://github.com/hebertcisco/pensador-api

Language: TypeScript - Size: 762 KB - Last synced: 16 days ago - Pushed: about 1 month ago - Stars: 2 - Forks: 1

puutaro/newsSpeecher

News speech by scraping in CommandClick

Language: JavaScript - Size: 30.3 KB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 0 - Forks: 0

autogram-is/spidergram

Structural analysis tools for complex web sites

Language: TypeScript - Size: 1.85 MB - Last synced: 12 days ago - Pushed: about 1 month ago - Stars: 101 - Forks: 5

Smartproxy/Web-Scraping-API

Web Scraping API code examples for Python, PHP and Node.js

Language: JavaScript - Size: 99.6 KB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 21 - Forks: 5

Smartproxy/eCommerce-Scraping-API

eCommerce Scraping API code examples for Python, PHP and Node.js

Language: PHP - Size: 151 KB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 21 - Forks: 4

josephlimtech/linkedin-profile-scraper-api

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON.

Language: TypeScript - Size: 10.8 MB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 466 - Forks: 133

rianlucascs/webscraper_e_enriquecedor_de_dados

Este projeto apresenta um software dedicado à automação da coleta de dados da web. Utilizando o nome da empresa e o domínio fornecidos, ele extrai informações relevantes e realiza uma filtragem para garantir a precisão dos dados obtidos. Essa abordagem simplifica significativamente o processo de obtenção de dados valiosos diretamente da internet.

Language: Python - Size: 962 KB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 0 - Forks: 0

oxylabs/free-trial-guides

Learn how to set up and get the most of your e-commerce, web scraper and serp scraper free trial.

Size: 854 KB - Last synced: 27 days ago - Pushed: about 1 month ago - Stars: 1 - Forks: 3

oxylabs/custom-parser-instructions

Learn the fundamentals of writing parsing instructions with Oxylabs' Custom Parser.

Language: Python - Size: 1.73 MB - Last synced: 27 days ago - Pushed: about 1 month ago - Stars: 1 - Forks: 0

outscraper/outscraper-ruby

The library provides convenient access to the Outscraper API from applications written in the Ruby language. Allows using Outscraper's services from your code.

Language: Ruby - Size: 34.2 KB - Last synced: 14 days ago - Pushed: about 1 month ago - Stars: 2 - Forks: 1

brahma-dev/metafetch

NodeJS package that fetches a given URL's title, description, images, links etc.

Language: JavaScript - Size: 235 KB - Last synced: 14 days ago - Pushed: about 1 month ago - Stars: 21 - Forks: 7

fa8i/Web_Scraping

Scripts de ejemplo para extraccion de texto, tablas o imagenes de diferentes webs.

Language: Jupyter Notebook - Size: 11.7 KB - Last synced: about 2 months ago - Pushed: about 2 months ago - Stars: 0 - Forks: 0

MontFerret/ferret

Declarative web scraping

Language: Go - Size: 3.87 MB - Last synced: about 1 month ago - Pushed: about 2 months ago - Stars: 5,615 - Forks: 301

thoth2357/Dynamic-Job-Board-Application

A jobBoard Web Application That helps scrape Jobs From multiple Sites at Certain Set Time And displays it

Language: HTML - Size: 17.4 MB - Last synced: about 2 months ago - Pushed: about 2 months ago - Stars: 1 - Forks: 0

EnthusiasticXcoder/Django-carrierAPI

Django-carrierAPI is a powerful RESTful API built with Django and Django Rest Framework for user authentication and serving blog and career data. With token-based authentication and standardized endpoints, it simplifies user authentication and data retrieval for client applications.

Language: Python - Size: 651 KB - Last synced: about 2 months ago - Pushed: about 2 months ago - Stars: 0 - Forks: 0