scraping-python | Topic | Ecosyste.ms: Repos

Topic: "scraping-python"

ScrapeGraphAI/Scrapegraph-ai

Python scraper based on AI

Language: Python - Size: 15.1 MB - Last synced at: 6 days ago - Pushed at: 7 days ago - Stars: 21,613 - Forks: 1,866

lorien/awesome-web-scraping

List of libraries, tools and APIs for web scraping and data processing.

Language: Makefile - Size: 427 KB - Last synced at: 7 days ago - Pushed at: 16 days ago - Stars: 7,384 - Forks: 825

omkarcloud/botasaurus

The All in One Framework to Build Undefeatable Scrapers

Language: Python - Size: 86 MB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 3,143 - Forks: 263

kaliiiiiiiiii/Selenium-Driverless

a stealthy browser automation framework

Language: Python - Size: 19.1 MB - Last synced at: 13 days ago - Pushed at: 6 months ago - Stars: 831 - Forks: 82

scrapfly/scrapfly-scrapers

Scalable Python web scraping scripts for +40 popular domains

Language: Python - Size: 6.58 MB - Last synced at: 12 days ago - Pushed at: 12 days ago - Stars: 693 - Forks: 156

franckferman/MetaDetective

🕵️ Unleash Metadata Intelligence with MetaDetective. Your Assistant Beyond Metagoofil.

Language: Python - Size: 19.2 MB - Last synced at: about 2 months ago - Pushed at: 6 months ago - Stars: 389 - Forks: 41

lkuffo/web-scraping

Más de 50 ejemplos de web scraping utilizando: Requests | Scrapy | Selenium | LXML | BeautifulSoup

Language: Python - Size: 60.5 MB - Last synced at: 8 days ago - Pushed at: 8 days ago - Stars: 377 - Forks: 215

furas/python-examples

Python examples from my answers on Stackoverflow and other short scripts.

Language: Jupyter Notebook - Size: 203 MB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 186 - Forks: 69

lobstrio/shadow-useragent

Pick the most common user-agents on the Internet 👻

Language: Python - Size: 62.5 KB - Last synced at: 2 days ago - Pushed at: almost 5 years ago - Stars: 170 - Forks: 11

fernandod1/Instagram-to-discord

Monitor instagram user account and automatically post new images to discord channel via a webhook. Working 2022!

Language: Python - Size: 15.6 KB - Last synced at: 3 months ago - Pushed at: over 4 years ago - Stars: 127 - Forks: 59

hhuayuan/spiderbuf

Spiderbuf 是一个专注于 Python 爬虫练习的网站。提供丰富的爬虫教程、爬虫案例解析和爬虫练习题。Python爬虫开发强化练习，在矛与盾的攻防中不断提高技术水平，通过大量的爬虫实战掌握常见的爬虫与反爬套路。引导式爬虫案例 + 免费爬虫视频教程，以闯关的形式挑战各个爬虫任务，培养爬虫开发的直觉及经验，验证自身爬虫开发与反爬虫实力的时候到了。

Language: Python - Size: 229 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 108 - Forks: 10

thewebscraping/tls-requests

TLS Requests is a powerful Python library for secure HTTP requests, offering browser-like TLS client, fingerprinting, anti-bot page bypass, and high performance.

Language: Python - Size: 3.71 MB - Last synced at: 6 days ago - Pushed at: 15 days ago - Stars: 90 - Forks: 7

outscraper/outscraper-python

The library provides convenient access to the Outscraper API from applications written in the Python language. Allows using Outscraper's services from your code.

Language: Python - Size: 154 KB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 85 - Forks: 21

ScrapingAnt/amazon_scraper

Amazon products scraper with using of rotating proxies and headless Chrome from ScrapingAnt

Language: JavaScript - Size: 52.7 KB - Last synced at: about 2 months ago - Pushed at: over 1 year ago - Stars: 85 - Forks: 19

fabriziomiano/linkedin-scraper 📦

Tool to scrape linkedin

Language: Python - Size: 42 KB - Last synced at: over 1 year ago - Pushed at: almost 4 years ago - Stars: 76 - Forks: 13

fernandod1/Instagram-downloader

Instagram user's photos and videos downloader. Download all media files from any username. Working 2022!

Language: Python - Size: 14.6 KB - Last synced at: 6 months ago - Pushed at: over 3 years ago - Stars: 72 - Forks: 16

Steelio/Telegram-Post-Scraper

Python program designed to scrape posts from Telegram channels using HTTP requests and HTML parsing, rather than Telegrams API. This is useful, as selfbots are against Telegram's ToS.

Language: Python - Size: 30.3 KB - Last synced at: 12 months ago - Pushed at: 12 months ago - Stars: 70 - Forks: 7

Srivaths-P/UpCode 📦

Upload accepted solutions from CodeChef and CodeForces to GitHub

Language: Python - Size: 52.7 KB - Last synced at: 2 months ago - Pushed at: about 1 year ago - Stars: 51 - Forks: 15

mike-gee/webtranspose

Web scraping API for building AI applications.

Language: Python - Size: 1.43 MB - Last synced at: about 2 months ago - Pushed at: almost 2 years ago - Stars: 41 - Forks: 2

xtekky/TikDown

Fast TikTok NO Watermark Video Downloader (username or url)

Language: Python - Size: 64.5 KB - Last synced at: 7 months ago - Pushed at: over 3 years ago - Stars: 41 - Forks: 12

wael-sudo2/facebook-page-info-scraper

Free Facebook pages MetaData Scraping Library - Unlimited Calls

Language: Python - Size: 106 KB - Last synced at: about 2 months ago - Pushed at: about 1 year ago - Stars: 40 - Forks: 8

officialpm/scrape-amazon

🤩 Python Package for Scraping Amazon Product Reviews ✨

Language: Python - Size: 134 KB - Last synced at: 30 days ago - Pushed at: almost 3 years ago - Stars: 37 - Forks: 13

poudelroshan/BOTVID-19

Messenger Bot that scrapes for COVID-19 data and periodically updates subscribers via Facebook Messages. Created using Python/Flask, MYSQL, HTML, Heroku

Language: Python - Size: 7.36 MB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 34 - Forks: 6

alessio-ds/BSCWhalesMonitor

BSC whales monitor and detector written in Python.

Language: Python - Size: 48.8 KB - Last synced at: 19 days ago - Pushed at: almost 4 years ago - Stars: 34 - Forks: 15

sd3v/openinsiderData

An Openinsider Scraper written in Python. Get the full Database behind the infamous SEC site :)

Language: Python - Size: 14.6 KB - Last synced at: 9 months ago - Pushed at: 9 months ago - Stars: 32 - Forks: 6

omkarcloud/botasaurus-starter

🚀 OFFICIAL STARTER TEMPLATE FOR BOTASAURUS SCRAPING FRAMEWORK 🤖

Language: TypeScript - Size: 402 KB - Last synced at: 24 days ago - Pushed at: 4 months ago - Stars: 27 - Forks: 9

kevinam99/Instagram-Like-Bot

A simple Instagram like bot that likes posts based on the hashtags you provide.

Language: Python - Size: 113 KB - Last synced at: 7 months ago - Pushed at: over 5 years ago - Stars: 27 - Forks: 5

mattdood/web_scraping_example

Simple RSS feed reader for HackerNews.

Language: Python - Size: 41 KB - Last synced at: over 2 years ago - Pushed at: almost 3 years ago - Stars: 26 - Forks: 11

fernandod1/ProductHunt-scraper

Producthunt.com famous website scraper script. Scrap all offers and save in spreadsheet excel file.

Language: Python - Size: 9.77 KB - Last synced at: 4 months ago - Pushed at: over 3 years ago - Stars: 26 - Forks: 9

asepscareer/ycnbc

Get Real Time News and Market Data From CNBC Using ycnbc !!!

Language: Python - Size: 70.3 KB - Last synced at: about 1 month ago - Pushed at: 4 months ago - Stars: 24 - Forks: 4

adgramigna/job-board-scraper

Scrapes job listings from popular job boards Greenhouse, Lever, Ashby, and Rippling

Language: Python - Size: 3.76 MB - Last synced at: 6 months ago - Pushed at: 6 months ago - Stars: 24 - Forks: 3

TheWebScrapingClub/AI-Cursor-Scraping-Assistant

A powerful tool that leverages Cursor AI and MCP (Model Context Protocol) to easily generate web scrapers for various types of websites.

Language: Python - Size: 14.6 KB - Last synced at: 5 months ago - Pushed at: 7 months ago - Stars: 23 - Forks: 3

An anonymous Instagram stories, posts, reels, IGTV videos, and profile pictures effortlessly. Supports cookie-based authentication for stories and allows downloading media without authentication for other types.

Language: Python - Size: 26.4 KB - Last synced at: 2 days ago - Pushed at: 4 months ago - Stars: 21 - Forks: 2

EnsembleData/ensembledata-python

Python library to scrape social media data via the EnsembleData API.

Language: Python - Size: 167 KB - Last synced at: 9 days ago - Pushed at: 10 days ago - Stars: 18 - Forks: 4

alexis-brosseau/Ebay-Scraper

Get the average price of a product on eBay or a list of products on sale.

Language: Python - Size: 71.3 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 18 - Forks: 6

MasterCruelty/robbot

My personal Telegram bot made in Python. It has several features and it's based on Pyrogram.

Language: Python - Size: 671 KB - Last synced at: 4 months ago - Pushed at: 6 months ago - Stars: 18 - Forks: 1

ArthurVerrez/linkedin-scraping-tools

A set of simple tools leveraging Selenium for scraping LinkedIn through LinkedIn Sales Navigator or LinkedIn Recruiter.

Language: Python - Size: 28.3 KB - Last synced at: 4 months ago - Pushed at: about 2 years ago - Stars: 18 - Forks: 6

outscraper/outscraper-php

The library provides convenient access to the Outscraper API from applications written in the PHP language. Allows using Outscraper's services from your code.

Language: PHP - Size: 94.7 KB - Last synced at: about 1 month ago - Pushed at: 2 months ago - Stars: 17 - Forks: 7

wetcork/raiplay-dl

The most advanced raiplay.it downloader

Language: Python - Size: 34.6 MB - Last synced at: over 2 years ago - Pushed at: about 3 years ago - Stars: 17 - Forks: 3

easonlai/facebook_post_scraping_and_text_analytics

This is demo repo to demostrate how to scrape post data from Facebook by Python with library facebook_scraper. And then use Azure Text Analytics to perform sentiment analysis for post text content.

Language: Jupyter Notebook - Size: 1.98 MB - Last synced at: 5 months ago - Pushed at: almost 4 years ago - Stars: 17 - Forks: 14

JonusNattapong/Crewzombitx64

This project was inspired by the unclecode/crawl4ai repository. It provided valuable insights and ideas that helped shape the development of Crewzombitx64.

Language: Python - Size: 682 KB - Last synced at: 8 months ago - Pushed at: 8 months ago - Stars: 16 - Forks: 0

sawyerclick/scrapers

Scrapin' some data, man

Language: Python - Size: 77.1 MB - Last synced at: 3 months ago - Pushed at: 8 months ago - Stars: 16 - Forks: 4

gurjeet29/telegram_members_scrapper

Python Script to scrape members from a selected Telegram group.

Language: Python - Size: 10.7 KB - Last synced at: 3 months ago - Pushed at: over 3 years ago - Stars: 16 - Forks: 10

morpheusthewhite/jwscraper

A python library for scraping videos from JW Player

Language: Python - Size: 40 KB - Last synced at: over 2 years ago - Pushed at: over 4 years ago - Stars: 16 - Forks: 6

hhhrrrttt222111/Selenium_python

Language: Python - Size: 19.3 MB - Last synced at: 6 days ago - Pushed at: about 4 years ago - Stars: 15 - Forks: 6

ohn0/youtube-livechat-scraper

grab youtube live chat data from existing VODs

Language: Python - Size: 492 KB - Last synced at: 23 days ago - Pushed at: almost 2 years ago - Stars: 14 - Forks: 5

baronet2/FirstCyclingAPI

An unofficial Python API wrapper for firstcycling.com

Language: Python - Size: 5.88 MB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 14 - Forks: 1

DmytroNorth/Linkedin-Job-Postings-Visualization-and-Analysis-Python

This Python script scrapes up to 100 most recent Linkedin job postings of any job title and creates sentiment visualization in a form of a word cloud.

Language: Jupyter Notebook - Size: 15 MB - Last synced at: 5 months ago - Pushed at: over 4 years ago - Stars: 14 - Forks: 1

liebki/MinecraftServerForkDownloads

Easy access to (almost) all minecraft server types/loaders with direct links.

Language: Python - Size: 311 KB - Last synced at: 12 days ago - Pushed at: 13 days ago - Stars: 13 - Forks: 1

danielfrees/scrapemed

ScrapeMed: Data scraping for PubMed Central.

Language: Python - Size: 13.6 MB - Last synced at: 30 days ago - Pushed at: almost 2 years ago - Stars: 13 - Forks: 1

xLe0x/AcquiFinder

Get acquisitions by scraping titles of crunchbase.

Language: Python - Size: 11.7 KB - Last synced at: 10 months ago - Pushed at: 10 months ago - Stars: 12 - Forks: 2

spicyparrot/kafka_scrapy_connect

A custom library that integrates Scrapy with Kafka.

Language: Python - Size: 59.6 KB - Last synced at: about 1 month ago - Pushed at: about 1 year ago - Stars: 12 - Forks: 1

Leixien/TrackerPriceAmazon-Bot

This Telegram Bot checks the price of a given Amazon Product.

Language: Python - Size: 13.7 KB - Last synced at: 12 months ago - Pushed at: over 1 year ago - Stars: 12 - Forks: 3

Titan1911/Google-Meet-Attendance-using-Python

Take the GoogleMeet Attendance without much effort!

Language: Python - Size: 26.4 KB - Last synced at: about 1 year ago - Pushed at: over 4 years ago - Stars: 12 - Forks: 8

mateuszbuda/instagram-scraping-fish

A tutorial for scraping Instagram profile information and posts using Scraping Fish API: https://scrapingfish.com

Language: Jupyter Notebook - Size: 26.4 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 11 - Forks: 1

lakshyaag/India-Trade-Data

A web scraper written in Python to gather trade data for India across commodities and countries

Language: Jupyter Notebook - Size: 5.86 MB - Last synced at: over 2 years ago - Pushed at: almost 3 years ago - Stars: 10 - Forks: 5

sbmagar13/raidfscrape

Code to Scraping some portion of Data from forum(RaidForums[seized by FBI]) with Python SCRAPY spiders bypassing recaptcha and storing to PostgreSQL database.(used scrapper-API as captcha, proxy bypass) SQLAlchemy as ORM for PostgreSQL - Python.

Size: 482 KB - Last synced at: 20 days ago - Pushed at: over 4 years ago - Stars: 10 - Forks: 3

mancunian1792/ipl_mock_auction

Mock IPL auction - Automated decision making

Language: Jupyter Notebook - Size: 313 KB - Last synced at: over 2 years ago - Pushed at: over 5 years ago - Stars: 10 - Forks: 2

regulad/freebooter 📦

freebooter downloads photos & videos from the internet and uploads it onto your social media accounts.

Language: Python - Size: 356 KB - Last synced at: 25 days ago - Pushed at: over 1 year ago - Stars: 9 - Forks: 0

mateuszbuda/webscraping-benchmark

Web scraping API benchmark

Language: Python - Size: 55.7 KB - Last synced at: almost 2 years ago - Pushed at: over 3 years ago - Stars: 9 - Forks: 1

adhityavenkatraman/PageRank

A power iteration algorithm to sort webpages. Builds off of the PageRank function developed by Sergey Brin and Larry Page.

Language: Python - Size: 89.8 KB - Last synced at: 3 months ago - Pushed at: almost 5 years ago - Stars: 9 - Forks: 0

omkarcloud/web-scraping-template

🚀 THIS WEB SCRAPING TEMPLATE PROVIDES YOU WITH A GREAT STARTING POINT WHEN CREATING WEB SCRAPING BOTS. 🤖

Language: Python - Size: 104 KB - Last synced at: 5 days ago - Pushed at: over 2 years ago - Stars: 8 - Forks: 4

tonyjward/plagiarism_detection

Using PyTorch to identify plagiarism in articles written about Data Science on the Medium platform

Language: Jupyter Notebook - Size: 2.04 MB - Last synced at: over 1 year ago - Pushed at: almost 3 years ago - Stars: 8 - Forks: 2

palahsu/ScraperImages

Simple Web Images Scraper From Websites!

Language: Python - Size: 55.7 KB - Last synced at: 4 months ago - Pushed at: almost 5 years ago - Stars: 8 - Forks: 0

StfBlanchet/Agoraphon

A Flask application for analyzing activity on an online discussion forum, using scraping, indexing, analytics, relational graph and NLP.

Size: 1.77 MB - Last synced at: over 2 years ago - Pushed at: almost 5 years ago - Stars: 8 - Forks: 1

schmwong/APAC-McDelivery-Menu-Logger

Automatically scrapes McDelivery menu data and records it for future visualisation projects

Language: Python - Size: 43.7 MB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 7 - Forks: 2

Solihatun1/AI-Cursor-Scraping-Assistant

A powerful tool that leverages Cursor AI and MCP (Model Context Protocol) to easily generate web scrapers for various types of websites.

Language: Python - Size: 16.6 KB - Last synced at: 2 months ago - Pushed at: 2 months ago - Stars: 7 - Forks: 0

gayanukabulegoda/Web-Scraping-Starter-Kit

Repository designed to help freshers easily grasp the basics of web scripting, offering simple guides and examples to build a strong foundation.

Language: Python - Size: 10.7 KB - Last synced at: about 1 month ago - Pushed at: 9 months ago - Stars: 7 - Forks: 0

SamRB-dev/AutoSeekOut

A simple web scraping bot for scraping information from seekout.com written in Python and Selenium

Language: Python - Size: 11.7 MB - Last synced at: 18 days ago - Pushed at: over 1 year ago - Stars: 7 - Forks: 0

umihico/minigun-requests

Web scraping API to outsource tons of GET & xpath to cloud computing

Language: Python - Size: 392 KB - Last synced at: about 1 month ago - Pushed at: over 1 year ago - Stars: 7 - Forks: 0

Wyndace/WynScrape

Wyndace's python scraping dev kit

Language: Python - Size: 37.1 KB - Last synced at: 6 months ago - Pushed at: about 2 years ago - Stars: 7 - Forks: 1

dhmine/tripadvisor-hotels-scraper

How to scrap hotels in tripAdvisor (I took France as an example)

Language: Python - Size: 3.01 MB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 7 - Forks: 1

MarekCichy/pt-br-classifier

Classifying Portuguese texts according to their variety (European or Brazilian Portuguese)

Language: Jupyter Notebook - Size: 96.7 KB - Last synced at: 14 days ago - Pushed at: almost 3 years ago - Stars: 7 - Forks: 3

Viswalahiri/Results_Fetcher

Facilitates retrieval of large amounts of data from GITAM's results platform

Language: Python - Size: 43 KB - Last synced at: over 2 years ago - Pushed at: almost 6 years ago - Stars: 7 - Forks: 5

ShiqinHuo/wuhan_house_price_crawler

武汉东湖高新片区光谷&软件园二手房房价爬虫。data source: 房天下

Language: Jupyter Notebook - Size: 14 MB - Last synced at: 5 months ago - Pushed at: over 6 years ago - Stars: 7 - Forks: 3

dbaio/portsfallout

An easy way to search/query the FreeBSD pkg-fallout archive

Language: Python - Size: 448 KB - Last synced at: 16 days ago - Pushed at: 24 days ago - Stars: 6 - Forks: 0

Saadmairaj/OAuthBrowser

This module will let you authenticate OAuth 2.0 via the system browser and get the response URL.

Language: Python - Size: 20.5 KB - Last synced at: 6 months ago - Pushed at: over 2 years ago - Stars: 6 - Forks: 0

1x6/lezhin-downloader

Simple requests based manhwa downloader which uses the Lezhin API.

Language: Python - Size: 13.7 KB - Last synced at: over 2 years ago - Pushed at: almost 3 years ago - Stars: 6 - Forks: 2

dori-dev/quotes-crawler

Quotes crawler using scrapy and python.

Language: Python - Size: 13.7 KB - Last synced at: 21 days ago - Pushed at: over 3 years ago - Stars: 6 - Forks: 0

AymaneLk/Python-Data-Scraping-IMDb-Movies-2020-using-BeautifulSoup

Scraping IMDb Movie 2020 site Using BeautifulSoup

Language: Python - Size: 13.7 KB - Last synced at: about 2 years ago - Pushed at: about 4 years ago - Stars: 6 - Forks: 2

Santa-Clara-Imaginarium-Lab/twitter-scraping-with-python

Twitter Scraping with Python!

Language: Python - Size: 36.1 KB - Last synced at: about 1 year ago - Pushed at: about 4 years ago - Stars: 6 - Forks: 2

pylyf/stockscrape

Getting stock price data using a python script.

Language: Python - Size: 23.4 KB - Last synced at: about 2 years ago - Pushed at: over 5 years ago - Stars: 6 - Forks: 4

M-Taghizadeh/Persian_Question_Answering_Voice2Voice_AI

This repository hosts BonyadAI, a Persian question answering AI Model. We developed an initial web crawler and scraper to gather the dataset. The second phase involved building a machine learning model based on word embeddings and NLP techniques. This AI model operates end-to-end, receiving user voice input and providing responses in Persian voice.

Language: Jupyter Notebook - Size: 89.4 MB - Last synced at: 6 months ago - Pushed at: over 1 year ago - Stars: 5 - Forks: 3

TheNoiselessNoise/csfd_scraper

Simple scraper for CSFD.cz, a Czech movie database.

Language: Python - Size: 141 KB - Last synced at: 2 months ago - Pushed at: 8 months ago - Stars: 4 - Forks: 1

shallvhack/Flight-Tracker

Flight Tracker: Real-time flight updates and interactive map for seamless tracking and staying in the loop.

Language: Python - Size: 20.5 KB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 4 - Forks: 1

epythonlab/github-search-tool

Github Repository Search Tool

Language: Python - Size: 281 KB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 4 - Forks: 1

spartypkp/legalAI

LegalAI is a passion project which explores and simplifies the complexities of obtaining legal information using LLMs.

Language: Jupyter Notebook - Size: 4.29 MB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 4 - Forks: 0

pierreleocadie/ScrapeGram 📦

OSINT tool to scraping data from someone's Instagram account.

Language: Python - Size: 62.5 KB - Last synced at: over 2 years ago - Pushed at: about 3 years ago - Stars: 4 - Forks: 5