scraping-python | Topic | Ecosyste.ms: Repos

Topic: "scraping-python"

ScrapeGraphAI/Scrapegraph-ai

Python scraper based on AI

Language: Python - Size: 14 MB - Last synced at: 5 days ago - Pushed at: about 1 month ago - Stars: 20,886 - Forks: 1,775

lorien/awesome-web-scraping

List of libraries, tools and APIs for web scraping and data processing.

Language: Makefile - Size: 473 KB - Last synced at: 7 days ago - Pushed at: 8 months ago - Stars: 7,106 - Forks: 814

omkarcloud/botasaurus

The All in One Framework to Build Undefeatable Scrapers

Language: Python - Size: 77.9 MB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 2,307 - Forks: 203

kaliiiiiiiiii/Selenium-Driverless

a stealthy browser automation framework

Language: Python - Size: 19.1 MB - Last synced at: 5 days ago - Pushed at: 4 months ago - Stars: 810 - Forks: 80

scrapfly/scrapfly-scrapers

Scalable Python web scraping scripts for +40 popular domains

Language: Python - Size: 4.94 MB - Last synced at: 3 days ago - Pushed at: 6 days ago - Stars: 594 - Forks: 140

franckferman/MetaDetective

🕵️ Unleash Metadata Intelligence with MetaDetective. Your Assistant Beyond Metagoofil.

Language: Python - Size: 19.2 MB - Last synced at: 11 days ago - Pushed at: 4 months ago - Stars: 377 - Forks: 39

lkuffo/web-scraping

Más de 50 ejemplos de web scraping utilizando: Requests | Scrapy | Selenium | LXML | BeautifulSoup

Language: Python - Size: 60.6 MB - Last synced at: 3 months ago - Pushed at: 10 months ago - Stars: 361 - Forks: 208

furas/python-examples

Python examples from my answers on Stackoverflow and other short scripts.

Language: Jupyter Notebook - Size: 203 MB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 186 - Forks: 69

lobstrio/shadow-useragent

Pick the most common user-agents on the Internet 👻

Language: Python - Size: 62.5 KB - Last synced at: 23 days ago - Pushed at: over 4 years ago - Stars: 170 - Forks: 11

fernandod1/Instagram-to-discord

Monitor instagram user account and automatically post new images to discord channel via a webhook. Working 2022!

Language: Python - Size: 15.6 KB - Last synced at: 19 days ago - Pushed at: over 4 years ago - Stars: 127 - Forks: 59

hhuayuan/spiderbuf

Spiderbuf 是一个专注于 Python 爬虫练习的网站。提供丰富的爬虫教程、爬虫案例解析和爬虫练习题。Python爬虫开发强化练习，在矛与盾的攻防中不断提高技术水平，通过大量的爬虫实战掌握常见的爬虫与反爬套路。引导式爬虫案例 + 免费爬虫视频教程，以闯关的形式挑战各个爬虫任务，培养爬虫开发的直觉及经验，验证自身爬虫开发与反爬虫实力的时候到了。

Language: Python - Size: 157 KB - Last synced at: 8 days ago - Pushed at: 8 days ago - Stars: 103 - Forks: 10

ScrapingAnt/amazon_scraper

Amazon products scraper with using of rotating proxies and headless Chrome from ScrapingAnt

Language: JavaScript - Size: 52.7 KB - Last synced at: 3 days ago - Pushed at: over 1 year ago - Stars: 85 - Forks: 19

outscraper/outscraper-python

The library provides convenient access to the Outscraper API from applications written in the Python language. Allows using Outscraper's services from your code.

Language: Python - Size: 162 KB - Last synced at: 24 days ago - Pushed at: 24 days ago - Stars: 80 - Forks: 19

fabriziomiano/linkedin-scraper 📦

Tool to scrape linkedin

Language: Python - Size: 42 KB - Last synced at: over 1 year ago - Pushed at: over 3 years ago - Stars: 76 - Forks: 13

fernandod1/Instagram-downloader

Instagram user's photos and videos downloader. Download all media files from any username. Working 2022!

Language: Python - Size: 14.6 KB - Last synced at: 3 months ago - Pushed at: about 3 years ago - Stars: 72 - Forks: 16

Steelio/Telegram-Post-Scraper

Python program designed to scrape posts from Telegram channels using HTTP requests and HTML parsing, rather than Telegrams API. This is useful, as selfbots are against Telegram's ToS.

Language: Python - Size: 30.3 KB - Last synced at: 9 months ago - Pushed at: 9 months ago - Stars: 70 - Forks: 7

thewebscraping/tls-requests

TLS Requests is a powerful Python library for secure HTTP requests, offering browser-like TLS client, fingerprinting, anti-bot page bypass, and high performance.

Language: Python - Size: 3.7 MB - Last synced at: about 1 month ago - Pushed at: 3 months ago - Stars: 68 - Forks: 5

fcoagz/pydolarvenezuela 📦

Esta librería desarrollada en Python te brinda la capacidad de consultar los precios del dólar y/o euro en diversos monitores en Venezuela, también como las tasas de cambio del BCV.

Language: Python - Size: 757 KB - Last synced at: 12 days ago - Pushed at: 7 months ago - Stars: 52 - Forks: 11

CrapTheCoder/UpCode

Upload accepted solutions from CodeChef and CodeForces to GitHub

Language: Python - Size: 52.7 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 45 - Forks: 15

mike-gee/webtranspose

Web scraping API for building AI applications.

Language: Python - Size: 1.43 MB - Last synced at: about 1 month ago - Pushed at: over 1 year ago - Stars: 41 - Forks: 2

xtekky/TikDown

Fast TikTok NO Watermark Video Downloader (username or url)

Language: Python - Size: 64.5 KB - Last synced at: 4 months ago - Pushed at: about 3 years ago - Stars: 41 - Forks: 12

wael-sudo2/facebook-page-info-scraper

Free Facebook pages MetaData Scraping Library - Unlimited Calls

Language: Python - Size: 106 KB - Last synced at: about 1 month ago - Pushed at: 10 months ago - Stars: 37 - Forks: 6

poudelroshan/BOTVID-19

Messenger Bot that scrapes for COVID-19 data and periodically updates subscribers via Facebook Messages. Created using Python/Flask, MYSQL, HTML, Heroku

Language: Python - Size: 7.36 MB - Last synced at: almost 2 years ago - Pushed at: about 2 years ago - Stars: 34 - Forks: 6

officialpm/scrape-amazon

🤩 Python Package for Scraping Amazon Product Reviews ✨

Language: Python - Size: 134 KB - Last synced at: about 1 year ago - Pushed at: over 2 years ago - Stars: 34 - Forks: 11

alessio-ds/BSCWhalesMonitor

BSC whales monitor and detector written in Python.

Language: Python - Size: 48.8 KB - Last synced at: 9 days ago - Pushed at: over 3 years ago - Stars: 34 - Forks: 18

sd3v/openinsiderData

An Openinsider Scraper written in Python. Get the full Database behind the infamous SEC site :)

Language: Python - Size: 14.6 KB - Last synced at: 6 months ago - Pushed at: 6 months ago - Stars: 32 - Forks: 6

serpwow/google-search-results-python

Scrape and parse Google search results in Python

Language: Python - Size: 209 KB - Last synced at: 23 days ago - Pushed at: about 2 years ago - Stars: 31 - Forks: 9

kevinam99/Instagram-Like-Bot

A simple Instagram like bot that likes posts based on the hashtags you provide.

Language: Python - Size: 113 KB - Last synced at: 4 months ago - Pushed at: over 5 years ago - Stars: 27 - Forks: 5

omkarcloud/botasaurus-starter

🚀 OFFICIAL STARTER TEMPLATE FOR BOTASAURUS SCRAPING FRAMEWORK 🤖

Language: TypeScript - Size: 402 KB - Last synced at: 12 days ago - Pushed at: about 1 month ago - Stars: 26 - Forks: 9

mattdood/web_scraping_example

Simple RSS feed reader for HackerNews.

Language: Python - Size: 41 KB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 26 - Forks: 11

fernandod1/ProductHunt-scraper

Producthunt.com famous website scraper script. Scrap all offers and save in spreadsheet excel file.

Language: Python - Size: 9.77 KB - Last synced at: about 2 months ago - Pushed at: about 3 years ago - Stars: 26 - Forks: 9

adgramigna/job-board-scraper

Scrapes job listings from popular job boards Greenhouse, Lever, Ashby, and Rippling

Language: Python - Size: 3.76 MB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 24 - Forks: 3

asepscareer/ycnbc

Get Real Time News and Market Data From CNBC Using ycnbc !!!

Language: Python - Size: 70.3 KB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 23 - Forks: 3

TheWebScrapingClub/AI-Cursor-Scraping-Assistant

A powerful tool that leverages Cursor AI and MCP (Model Context Protocol) to easily generate web scrapers for various types of websites.

Language: Python - Size: 14.6 KB - Last synced at: about 2 months ago - Pushed at: 4 months ago - Stars: 23 - Forks: 3

alexis-brosseau/Ebay-Scraper

Get the average price of a product on eBay or a list of products on sale.

Language: Python - Size: 71.3 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 18 - Forks: 6

MasterCruelty/robbot

My personal Telegram bot made in Python. It has several features and it's based on Pyrogram.

Language: Python - Size: 671 KB - Last synced at: 28 days ago - Pushed at: 4 months ago - Stars: 18 - Forks: 1

ArthurVerrez/linkedin-scraping-tools

A set of simple tools leveraging Selenium for scraping LinkedIn through LinkedIn Sales Navigator or LinkedIn Recruiter.

Language: Python - Size: 28.3 KB - Last synced at: about 2 months ago - Pushed at: almost 2 years ago - Stars: 18 - Forks: 6

outscraper/outscraper-php

The library provides convenient access to the Outscraper API from applications written in the PHP language. Allows using Outscraper's services from your code.

Language: PHP - Size: 67.4 KB - Last synced at: 24 days ago - Pushed at: 24 days ago - Stars: 17 - Forks: 7

wetcork/raiplay-dl

The most advanced raiplay.it downloader

Language: Python - Size: 34.6 MB - Last synced at: over 2 years ago - Pushed at: almost 3 years ago - Stars: 17 - Forks: 3

easonlai/facebook_post_scraping_and_text_analytics

This is demo repo to demostrate how to scrape post data from Facebook by Python with library facebook_scraper. And then use Azure Text Analytics to perform sentiment analysis for post text content.

Language: Jupyter Notebook - Size: 1.98 MB - Last synced at: 3 months ago - Pushed at: over 3 years ago - Stars: 17 - Forks: 14

EnsembleData/ensembledata-python

Python library to scrape social media data via the EnsembleData API.

Language: Python - Size: 163 KB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 16 - Forks: 2

JonusNattapong/Crewzombitx64

This project was inspired by the unclecode/crawl4ai repository. It provided valuable insights and ideas that helped shape the development of Crewzombitx64.

Language: Python - Size: 682 KB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 16 - Forks: 0

gurjeet29/telegram_members_scrapper

Python Script to scrape members from a selected Telegram group.

Language: Python - Size: 10.7 KB - Last synced at: 20 days ago - Pushed at: over 3 years ago - Stars: 16 - Forks: 10

morpheusthewhite/jwscraper

A python library for scraping videos from JW Player

Language: Python - Size: 40 KB - Last synced at: over 2 years ago - Pushed at: about 4 years ago - Stars: 16 - Forks: 6

hhhrrrttt222111/Selenium_python

Language: Python - Size: 19.3 MB - Last synced at: 4 months ago - Pushed at: almost 4 years ago - Stars: 15 - Forks: 6

An anonymous Instagram stories, posts, reels, IGTV videos, and profile pictures effortlessly. Supports cookie-based authentication for stories and allows downloading media without authentication for other types.

Language: Python - Size: 21.5 KB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 14 - Forks: 1

sawyerclick/scrapers

Scrapin' some data, man

Language: Python - Size: 72.8 MB - Last synced at: 10 months ago - Pushed at: 10 months ago - Stars: 14 - Forks: 4

ohn0/youtube-livechat-scraper

grab youtube live chat data from existing VODs

Language: Python - Size: 492 KB - Last synced at: about 1 month ago - Pushed at: over 1 year ago - Stars: 14 - Forks: 5

baronet2/FirstCyclingAPI

An unofficial Python API wrapper for firstcycling.com

Language: Python - Size: 5.88 MB - Last synced at: about 2 years ago - Pushed at: about 2 years ago - Stars: 14 - Forks: 1

DmytroNorth/Linkedin-Job-Postings-Visualization-and-Analysis-Python

This Python script scrapes up to 100 most recent Linkedin job postings of any job title and creates sentiment visualization in a form of a word cloud.

Language: Jupyter Notebook - Size: 15 MB - Last synced at: 2 months ago - Pushed at: about 4 years ago - Stars: 14 - Forks: 1

liebki/MinecraftServerForkDownloads

Easy access to (almost) all minecraft server types/loaders with direct links.

Language: Python - Size: 181 KB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 12 - Forks: 1

xLe0x/AcquiFinder

Get acquisitions by scraping titles of crunchbase.

Language: Python - Size: 11.7 KB - Last synced at: 8 months ago - Pushed at: 8 months ago - Stars: 12 - Forks: 2

Leixien/TrackerPriceAmazon-Bot

This Telegram Bot checks the price of a given Amazon Product.

Language: Python - Size: 13.7 KB - Last synced at: 10 months ago - Pushed at: about 1 year ago - Stars: 12 - Forks: 3

Titan1911/Google-Meet-Attendance-using-Python

Take the GoogleMeet Attendance without much effort!

Language: Python - Size: 26.4 KB - Last synced at: about 1 year ago - Pushed at: about 4 years ago - Stars: 12 - Forks: 8

mateuszbuda/instagram-scraping-fish

A tutorial for scraping Instagram profile information and posts using Scraping Fish API: https://scrapingfish.com

Language: Jupyter Notebook - Size: 26.4 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 11 - Forks: 1

spicyparrot/kafka_scrapy_connect

A custom library that integrates Scrapy with Kafka.

Language: Python - Size: 59.6 KB - Last synced at: 9 months ago - Pushed at: about 1 year ago - Stars: 10 - Forks: 1

lakshyaag/India-Trade-Data

A web scraper written in Python to gather trade data for India across commodities and countries

Language: Jupyter Notebook - Size: 5.86 MB - Last synced at: over 2 years ago - Pushed at: almost 3 years ago - Stars: 10 - Forks: 5

sbmagar13/raidfscrape

Code to Scraping some portion of Data from forum(RaidForums[seized by FBI]) with Python SCRAPY spiders bypassing recaptcha and storing to PostgreSQL database.(used scrapper-API as captcha, proxy bypass) SQLAlchemy as ORM for PostgreSQL - Python.

Size: 482 KB - Last synced at: 9 days ago - Pushed at: over 4 years ago - Stars: 10 - Forks: 3

mancunian1792/ipl_mock_auction

Mock IPL auction - Automated decision making

Language: Jupyter Notebook - Size: 313 KB - Last synced at: over 2 years ago - Pushed at: over 5 years ago - Stars: 10 - Forks: 2

regulad/freebooter 📦

freebooter downloads photos & videos from the internet and uploads it onto your social media accounts.

Language: Python - Size: 356 KB - Last synced at: 6 months ago - Pushed at: over 1 year ago - Stars: 9 - Forks: 0

mateuszbuda/webscraping-benchmark

Web scraping API benchmark

Language: Python - Size: 55.7 KB - Last synced at: over 1 year ago - Pushed at: about 3 years ago - Stars: 9 - Forks: 1

adhityavenkatraman/PageRank

A power iteration algorithm to sort webpages. Builds off of the PageRank function developed by Sergey Brin and Larry Page.

Language: Python - Size: 89.8 KB - Last synced at: 22 days ago - Pushed at: over 4 years ago - Stars: 9 - Forks: 0

tonyjward/plagiarism_detection

Using PyTorch to identify plagiarism in articles written about Data Science on the Medium platform

Language: Jupyter Notebook - Size: 2.04 MB - Last synced at: over 1 year ago - Pushed at: over 2 years ago - Stars: 8 - Forks: 2

palahsu/ScraperImages

Simple Web Images Scraper From Websites!

Language: Python - Size: 55.7 KB - Last synced at: about 1 month ago - Pushed at: over 4 years ago - Stars: 8 - Forks: 0

StfBlanchet/Agoraphon

A Flask application for analyzing activity on an online discussion forum, using scraping, indexing, analytics, relational graph and NLP.

Size: 1.77 MB - Last synced at: over 2 years ago - Pushed at: over 4 years ago - Stars: 8 - Forks: 1

schmwong/APAC-McDelivery-Menu-Logger

Automatically scrapes McDelivery menu data and records it for future visualisation projects

Language: Python - Size: 42.3 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 7 - Forks: 2

gayanukabulegoda/Web-Scraping-Starter-Kit

Repository designed to help freshers easily grasp the basics of web scripting, offering simple guides and examples to build a strong foundation.

Language: Python - Size: 10.7 KB - Last synced at: 4 months ago - Pushed at: 6 months ago - Stars: 7 - Forks: 0

SamRB-dev/AutoSeekOut

A simple web scraping bot for scraping information from seekout.com written in Python and Selenium

Language: Python - Size: 11.7 MB - Last synced at: 4 months ago - Pushed at: about 1 year ago - Stars: 7 - Forks: 0

umihico/minigun-requests

Web scraping API to outsource tons of GET & xpath to cloud computing

Language: Python - Size: 392 KB - Last synced at: 2 months ago - Pushed at: about 1 year ago - Stars: 7 - Forks: 0

Wyndace/WynScrape

Wyndace's python scraping dev kit

Language: Python - Size: 37.1 KB - Last synced at: 3 months ago - Pushed at: almost 2 years ago - Stars: 7 - Forks: 1

omkarcloud/web-scraping-template

🚀 THIS WEB SCRAPING TEMPLATE PROVIDES YOU WITH A GREAT STARTING POINT WHEN CREATING WEB SCRAPING BOTS. 🤖

Language: Python - Size: 104 KB - Last synced at: 12 days ago - Pushed at: about 2 years ago - Stars: 7 - Forks: 3

dhmine/tripadvisor-hotels-scraper

How to scrap hotels in tripAdvisor (I took France as an example)

Language: Python - Size: 3.01 MB - Last synced at: about 2 years ago - Pushed at: about 2 years ago - Stars: 7 - Forks: 1

MarekCichy/pt-br-classifier

Classifying Portuguese texts according to their variety (European or Brazilian Portuguese)

Language: Jupyter Notebook - Size: 96.7 KB - Last synced at: 5 days ago - Pushed at: over 2 years ago - Stars: 7 - Forks: 3

Viswalahiri/Results_Fetcher

Facilitates retrieval of large amounts of data from GITAM's results platform

Language: Python - Size: 43 KB - Last synced at: over 2 years ago - Pushed at: over 5 years ago - Stars: 7 - Forks: 5

ShiqinHuo/wuhan_house_price_crawler

武汉东湖高新片区光谷&软件园二手房房价爬虫。data source: 房天下

Language: Jupyter Notebook - Size: 14 MB - Last synced at: 2 months ago - Pushed at: over 6 years ago - Stars: 7 - Forks: 3

Solihatun1/AI-Cursor-Scraping-Assistant

A powerful tool that leverages Cursor AI and MCP (Model Context Protocol) to easily generate web scrapers for various types of websites.

Language: Python - Size: 16.6 KB - Last synced at: 29 days ago - Pushed at: 29 days ago - Stars: 6 - Forks: 0

dbaio/portsfallout

An easy way to search/query the FreeBSD pkg-fallout archive

Language: Python - Size: 445 KB - Last synced at: 7 days ago - Pushed at: 5 months ago - Stars: 6 - Forks: 0

Saadmairaj/OAuthBrowser

This module will let you authenticate OAuth 2.0 via the system browser and get the response URL.

Language: Python - Size: 20.5 KB - Last synced at: 4 months ago - Pushed at: over 2 years ago - Stars: 6 - Forks: 0

1x6/lezhin-downloader

Simple requests based manhwa downloader which uses the Lezhin API.

Language: Python - Size: 13.7 KB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 6 - Forks: 2

dori-dev/quotes-crawler

Quotes crawler using scrapy and python.

Language: Python - Size: 13.7 KB - Last synced at: 4 months ago - Pushed at: about 3 years ago - Stars: 6 - Forks: 0

AymaneLk/Python-Data-Scraping-IMDb-Movies-2020-using-BeautifulSoup

Scraping IMDb Movie 2020 site Using BeautifulSoup

Language: Python - Size: 13.7 KB - Last synced at: almost 2 years ago - Pushed at: about 4 years ago - Stars: 6 - Forks: 2

Santa-Clara-Imaginarium-Lab/twitter-scraping-with-python

Twitter Scraping with Python!

Language: Python - Size: 36.1 KB - Last synced at: about 1 year ago - Pushed at: about 4 years ago - Stars: 6 - Forks: 2

pylyf/stockscrape

Getting stock price data using a python script.

Language: Python - Size: 23.4 KB - Last synced at: about 2 years ago - Pushed at: over 5 years ago - Stars: 6 - Forks: 4

M-Taghizadeh/Persian_Question_Answering_Voice2Voice_AI

This repository hosts BonyadAI, a Persian question answering AI Model. We developed an initial web crawler and scraper to gather the dataset. The second phase involved building a machine learning model based on word embeddings and NLP techniques. This AI model operates end-to-end, receiving user voice input and providing responses in Persian voice.

Language: Jupyter Notebook - Size: 89.4 MB - Last synced at: 3 months ago - Pushed at: about 1 year ago - Stars: 5 - Forks: 3

TheNoiselessNoise/csfd_scraper

Simple scraper for CSFD.cz, a Czech movie database.

Language: Python - Size: 141 KB - Last synced at: 25 days ago - Pushed at: 5 months ago - Stars: 4 - Forks: 1

shallvhack/Flight-Tracker

Flight Tracker: Real-time flight updates and interactive map for seamless tracking and staying in the loop.

Language: Python - Size: 20.5 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 4 - Forks: 1

epythonlab/github-search-tool

Github Repository Search Tool

Language: Python - Size: 281 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 4 - Forks: 1

danielfrees/scrapemed

ScrapeMed: Data scraping for PubMed Central.

Language: Python - Size: 13.6 MB - Last synced at: about 1 year ago - Pushed at: over 1 year ago - Stars: 4 - Forks: 1

spartypkp/legalAI

LegalAI is a passion project which explores and simplifies the complexities of obtaining legal information using LLMs.

Language: Jupyter Notebook - Size: 4.29 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 4 - Forks: 0

pierreleocadie/ScrapeGram 📦

OSINT tool to scraping data from someone's Instagram account.

Language: Python - Size: 62.5 KB - Last synced at: about 2 years ago - Pushed at: almost 3 years ago - Stars: 4 - Forks: 5