scraping-python | Topic | Ecosyste.ms: Repos

Topic: "scraping-python"

ScrapeGraphAI/Scrapegraph-ai

Python scraper based on AI

Language: Python - Size: 15 MB - Last synced at: 6 days ago - Pushed at: 13 days ago - Stars: 19,468 - Forks: 1,646

lorien/awesome-web-scraping

List of libraries, tools and APIs for web scraping and data processing.

Language: Makefile - Size: 473 KB - Last synced at: 11 days ago - Pushed at: 5 months ago - Stars: 6,982 - Forks: 808

omkarcloud/botasaurus

The All in One Framework to Build Undefeatable Scrapers

Language: Python - Size: 62.3 MB - Last synced at: 7 days ago - Pushed at: 15 days ago - Stars: 1,861 - Forks: 163

kaliiiiiiiiii/Selenium-Driverless

a stealthy browser automation framework

Language: Python - Size: 19.1 MB - Last synced at: 1 day ago - Pushed at: 17 days ago - Stars: 774 - Forks: 77

scrapfly/scrapfly-scrapers

Scalable Python web scraping scripts for +40 popular domains

Language: Python - Size: 4.69 MB - Last synced at: 4 days ago - Pushed at: about 2 months ago - Stars: 505 - Forks: 127

franckferman/MetaDetective

🕵️ Unleash Metadata Intelligence with MetaDetective. Your Assistant Beyond Metagoofil.

Language: Python - Size: 19.2 MB - Last synced at: 24 days ago - Pushed at: 25 days ago - Stars: 360 - Forks: 35

lkuffo/web-scraping

Más de 50 ejemplos de web scraping utilizando: Requests | Scrapy | Selenium | LXML | BeautifulSoup

Language: Python - Size: 60.6 MB - Last synced at: about 1 month ago - Pushed at: 7 months ago - Stars: 353 - Forks: 206

furas/python-examples

Python examples from my answers on Stackoverflow and other short scripts.

Language: Python - Size: 197 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 172 - Forks: 73

lobstrio/shadow-useragent

Pick the most common user-agents on the Internet 👻

Language: Python - Size: 62.5 KB - Last synced at: 26 days ago - Pushed at: over 4 years ago - Stars: 168 - Forks: 12

fernandod1/Instagram-to-discord

Monitor instagram user account and automatically post new images to discord channel via a webhook. Working 2022!

Language: Python - Size: 15.6 KB - Last synced at: 27 days ago - Pushed at: about 4 years ago - Stars: 129 - Forks: 59

hhuayuan/spiderbuf

Spiderbuf 是一个专注于 Python 爬虫练习的网站。提供丰富的爬虫教程、爬虫案例解析和爬虫练习题。Python爬虫开发强化练习，在矛与盾的攻防中不断提高技术水平，通过大量的爬虫实战掌握常见的爬虫与反爬套路。引导式爬虫案例 + 免费爬虫视频教程，以闯关的形式挑战各个爬虫任务，培养爬虫开发的直觉及经验，验证自身爬虫开发与反爬虫实力的时候到了。

Language: Python - Size: 137 KB - Last synced at: 3 days ago - Pushed at: 4 days ago - Stars: 91 - Forks: 9

ScrapingAnt/amazon_scraper

Amazon products scraper with using of rotating proxies and headless Chrome from ScrapingAnt

Language: JavaScript - Size: 52.7 KB - Last synced at: 4 days ago - Pushed at: about 1 year ago - Stars: 81 - Forks: 19

fabriziomiano/linkedin-scraper 📦

Tool to scrape linkedin

Language: Python - Size: 42 KB - Last synced at: about 1 year ago - Pushed at: over 3 years ago - Stars: 76 - Forks: 13

outscraper/outscraper-python

The library provides convenient access to the Outscraper API from applications written in the Python language. Allows using Outscraper's services from your code.

Language: Python - Size: 146 KB - Last synced at: 14 days ago - Pushed at: about 1 month ago - Stars: 75 - Forks: 21

fernandod1/Instagram-downloader

Instagram user's photos and videos downloader. Download all media files from any username. Working 2022!

Language: Python - Size: 14.6 KB - Last synced at: 9 days ago - Pushed at: almost 3 years ago - Stars: 72 - Forks: 16

Steelio/Telegram-Post-Scraper

Python program designed to scrape posts from Telegram channels using HTTP requests and HTML parsing, rather than Telegrams API. This is useful, as selfbots are against Telegram's ToS.

Language: Python - Size: 30.3 KB - Last synced at: 6 months ago - Pushed at: 6 months ago - Stars: 70 - Forks: 7

fcoagz/pydolarvenezuela

Esta librería desarrollada en Python te brinda la capacidad de consultar los precios del dólar y/o euro en diversos monitores en Venezuela, también como las tasas de cambio del BCV.

Language: Python - Size: 757 KB - Last synced at: 7 days ago - Pushed at: 4 months ago - Stars: 50 - Forks: 11

thewebscraping/tls-requests

TLS Requests is a powerful Python library for secure HTTP requests, offering browser-like TLS client, fingerprinting, anti-bot page bypass, and high performance.

Language: Python - Size: 3.68 MB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 48 - Forks: 3

CrapTheCoder/UpCode

Upload accepted solutions from CodeChef and CodeForces to GitHub

Language: Python - Size: 52.7 KB - Last synced at: 9 months ago - Pushed at: 9 months ago - Stars: 45 - Forks: 15

mike-gee/webtranspose

Web scraping API for building AI applications.

Language: Python - Size: 1.43 MB - Last synced at: 12 days ago - Pushed at: over 1 year ago - Stars: 41 - Forks: 2

xtekky/TikDown

Fast TikTok NO Watermark Video Downloader (username or url)

Language: Python - Size: 64.5 KB - Last synced at: about 1 month ago - Pushed at: almost 3 years ago - Stars: 41 - Forks: 12

poudelroshan/BOTVID-19

Messenger Bot that scrapes for COVID-19 data and periodically updates subscribers via Facebook Messages. Created using Python/Flask, MYSQL, HTML, Heroku

Language: Python - Size: 7.36 MB - Last synced at: over 1 year ago - Pushed at: almost 2 years ago - Stars: 34 - Forks: 6

officialpm/scrape-amazon

🤩 Python Package for Scraping Amazon Product Reviews ✨

Language: Python - Size: 134 KB - Last synced at: 9 months ago - Pushed at: over 2 years ago - Stars: 34 - Forks: 11

alessio-ds/BSCWhalesMonitor

BSC whales monitor and detector written in Python.

Language: Python - Size: 48.8 KB - Last synced at: 2 days ago - Pushed at: over 3 years ago - Stars: 34 - Forks: 18

sd3v/openinsiderData

An Openinsider Scraper written in Python. Get the full Database behind the infamous SEC site :)

Language: Python - Size: 14.6 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 32 - Forks: 6

wael-sudo2/facebook-page-info-scraper

Free Facebook pages MetaData Scraping Library - Unlimited Calls

Language: Python - Size: 106 KB - Last synced at: 13 days ago - Pushed at: 7 months ago - Stars: 32 - Forks: 5

serpwow/google-search-results-python

Scrape and parse Google search results in Python

Language: Python - Size: 209 KB - Last synced at: 28 days ago - Pushed at: almost 2 years ago - Stars: 31 - Forks: 9

kevinam99/Instagram-Like-Bot

A simple Instagram like bot that likes posts based on the hashtags you provide.

Language: Python - Size: 113 KB - Last synced at: 30 days ago - Pushed at: about 5 years ago - Stars: 27 - Forks: 5

mattdood/web_scraping_example

Simple RSS feed reader for HackerNews.

Language: Python - Size: 41 KB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 26 - Forks: 11

fernandod1/ProductHunt-scraper

Producthunt.com famous website scraper script. Scrap all offers and save in spreadsheet excel file.

Language: Python - Size: 9.77 KB - Last synced at: 9 days ago - Pushed at: almost 3 years ago - Stars: 26 - Forks: 9

omkarcloud/botasaurus-starter

🚀 OFFICIAL STARTER TEMPLATE FOR BOTASAURUS SCRAPING FRAMEWORK 🤖

Language: TypeScript - Size: 397 KB - Last synced at: 8 days ago - Pushed at: 8 days ago - Stars: 25 - Forks: 8

adgramigna/job-board-scraper

Scrapes job listings from popular job boards Greenhouse, Lever, Ashby, and Rippling

Language: Python - Size: 3.76 MB - Last synced at: 13 days ago - Pushed at: 13 days ago - Stars: 24 - Forks: 3

asepscareer/ycnbc

Get Real Time News and Market Data From CNBC Using ycnbc !!!

Language: Python - Size: 45.9 KB - Last synced at: 22 days ago - Pushed at: 9 months ago - Stars: 20 - Forks: 3

MasterCruelty/robbot

My personal Telegram bot made in Python. It has several features and it's based on Pyrogram.

Language: Python - Size: 671 KB - Last synced at: 26 days ago - Pushed at: 4 months ago - Stars: 18 - Forks: 1

outscraper/outscraper-php

The library provides convenient access to the Outscraper API from applications written in the PHP language. Allows using Outscraper's services from your code.

Language: PHP - Size: 44.9 KB - Last synced at: 7 days ago - Pushed at: 3 months ago - Stars: 17 - Forks: 6

wetcork/raiplay-dl

The most advanced raiplay.it downloader

Language: Python - Size: 34.6 MB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 17 - Forks: 3

easonlai/facebook_post_scraping_and_text_analytics

This is demo repo to demostrate how to scrape post data from Facebook by Python with library facebook_scraper. And then use Azure Text Analytics to perform sentiment analysis for post text content.

Language: Jupyter Notebook - Size: 1.98 MB - Last synced at: 16 days ago - Pushed at: over 3 years ago - Stars: 17 - Forks: 14

EnsembleData/ensembledata-python

Python library to scrape social media data via the EnsembleData API.

Language: Python - Size: 163 KB - Last synced at: 3 days ago - Pushed at: about 1 month ago - Stars: 16 - Forks: 2

JonusNattapong/Crewzombitx64

This project was inspired by the unclecode/crawl4ai repository. It provided valuable insights and ideas that helped shape the development of Crewzombitx64.

Language: Python - Size: 682 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 16 - Forks: 0

ArthurVerrez/linkedin-scraping-tools

A set of simple tools leveraging Selenium for scraping LinkedIn through LinkedIn Sales Navigator or LinkedIn Recruiter.

Language: Python - Size: 28.3 KB - Last synced at: 2 months ago - Pushed at: over 1 year ago - Stars: 16 - Forks: 6

gurjeet29/telegram_members_scrapper

Python Script to scrape members from a selected Telegram group.

Language: Python - Size: 10.7 KB - Last synced at: 5 months ago - Pushed at: about 3 years ago - Stars: 16 - Forks: 11

morpheusthewhite/jwscraper

A python library for scraping videos from JW Player

Language: Python - Size: 40 KB - Last synced at: about 2 years ago - Pushed at: almost 4 years ago - Stars: 16 - Forks: 6

alexis-brosseau/Ebay-Scraper

Get the average price of a product on eBay or a list of products on sale.

Language: Python - Size: 71.3 KB - Last synced at: 2 months ago - Pushed at: 2 months ago - Stars: 15 - Forks: 4

hhhrrrttt222111/Selenium_python

Language: Python - Size: 19.3 MB - Last synced at: about 1 month ago - Pushed at: over 3 years ago - Stars: 15 - Forks: 6

sawyerclick/scrapers

Scrapin' some data, man

Language: Python - Size: 72.8 MB - Last synced at: 6 months ago - Pushed at: 6 months ago - Stars: 14 - Forks: 4

baronet2/FirstCyclingAPI

An unofficial Python API wrapper for firstcycling.com

Language: Python - Size: 5.88 MB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 14 - Forks: 1

ohn0/youtube-livechat-scraper

grab youtube live chat data from existing VODs

Language: Python - Size: 492 KB - Last synced at: 15 days ago - Pushed at: over 1 year ago - Stars: 13 - Forks: 3

xLe0x/AcquiFinder

Get acquisitions by scraping titles of crunchbase.

Language: Python - Size: 11.7 KB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 12 - Forks: 2

Leixien/TrackerPriceAmazon-Bot

This Telegram Bot checks the price of a given Amazon Product.

Language: Python - Size: 13.7 KB - Last synced at: 6 months ago - Pushed at: 12 months ago - Stars: 12 - Forks: 3

Titan1911/Google-Meet-Attendance-using-Python

Take the GoogleMeet Attendance without much effort!

Language: Python - Size: 26.4 KB - Last synced at: 10 months ago - Pushed at: almost 4 years ago - Stars: 12 - Forks: 8

mateuszbuda/instagram-scraping-fish

A tutorial for scraping Instagram profile information and posts using Scraping Fish API: https://scrapingfish.com

Language: Jupyter Notebook - Size: 26.4 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 11 - Forks: 1

spicyparrot/kafka_scrapy_connect

A custom library that integrates Scrapy with Kafka.

Language: Python - Size: 59.6 KB - Last synced at: 6 months ago - Pushed at: 9 months ago - Stars: 10 - Forks: 1

lakshyaag/India-Trade-Data

A web scraper written in Python to gather trade data for India across commodities and countries

Language: Jupyter Notebook - Size: 5.86 MB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 10 - Forks: 5

mancunian1792/ipl_mock_auction

Mock IPL auction - Automated decision making

Language: Jupyter Notebook - Size: 313 KB - Last synced at: about 2 years ago - Pushed at: about 5 years ago - Stars: 10 - Forks: 2

regulad/freebooter 📦

freebooter downloads photos & videos from the internet and uploads it onto your social media accounts.

Language: Python - Size: 356 KB - Last synced at: 3 months ago - Pushed at: about 1 year ago - Stars: 9 - Forks: 0

mateuszbuda/webscraping-benchmark

Web scraping API benchmark

Language: Python - Size: 55.7 KB - Last synced at: over 1 year ago - Pushed at: almost 3 years ago - Stars: 9 - Forks: 1

Code to Scraping some portion of Data from forum(RaidForums[seized by FBI]) with Python SCRAPY spiders bypassing recaptcha and storing to PostgreSQL database.(used scrapper-API as captcha, proxy bypass) SQLAlchemy as ORM for PostgreSQL - Python.

Size: 482 KB - Last synced at: 6 days ago - Pushed at: about 4 years ago - Stars: 9 - Forks: 3

tonyjward/plagiarism_detection

Using PyTorch to identify plagiarism in articles written about Data Science on the Medium platform

Language: Jupyter Notebook - Size: 2.04 MB - Last synced at: about 1 year ago - Pushed at: over 2 years ago - Stars: 8 - Forks: 2

palahsu/ScraperImages

Simple Web Images Scraper From Websites!

Language: Python - Size: 55.7 KB - Last synced at: 17 days ago - Pushed at: over 4 years ago - Stars: 8 - Forks: 0

StfBlanchet/Agoraphon

A Flask application for analyzing activity on an online discussion forum, using scraping, indexing, analytics, relational graph and NLP.

Size: 1.77 MB - Last synced at: about 2 years ago - Pushed at: over 4 years ago - Stars: 8 - Forks: 1

liebki/MinecraftServerForkDownloads

Easy access to (almost) all minecraft server types/loaders with direct links.

Language: Python - Size: 228 KB - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 7 - Forks: 1

gayanukabulegoda/Web-Scraping-Starter-Kit

Repository designed to help freshers easily grasp the basics of web scripting, offering simple guides and examples to build a strong foundation.

Language: Python - Size: 10.7 KB - Last synced at: about 1 month ago - Pushed at: 3 months ago - Stars: 7 - Forks: 0

SamRB-dev/AutoSeekOut

A simple web scraping bot for scraping information from seekout.com written in Python and Selenium

Language: Python - Size: 11.7 MB - Last synced at: about 1 month ago - Pushed at: 11 months ago - Stars: 7 - Forks: 0

umihico/minigun-requests

Web scraping API to outsource tons of GET & xpath to cloud computing

Language: Python - Size: 392 KB - Last synced at: 29 days ago - Pushed at: 11 months ago - Stars: 7 - Forks: 0

Wyndace/WynScrape

Wyndace's python scraping dev kit

Language: Python - Size: 37.1 KB - Last synced at: about 1 month ago - Pushed at: over 1 year ago - Stars: 7 - Forks: 1

omkarcloud/web-scraping-template

🚀 THIS WEB SCRAPING TEMPLATE PROVIDES YOU WITH A GREAT STARTING POINT WHEN CREATING WEB SCRAPING BOTS. 🤖

Language: Python - Size: 104 KB - Last synced at: 7 days ago - Pushed at: almost 2 years ago - Stars: 7 - Forks: 3

dhmine/tripadvisor-hotels-scraper

How to scrap hotels in tripAdvisor (I took France as an example)

Language: Python - Size: 3.01 MB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 7 - Forks: 1

MarekCichy/pt-br-classifier

Classifying Portuguese texts according to their variety (European or Brazilian Portuguese)

Language: Jupyter Notebook - Size: 96.7 KB - Last synced at: 1 day ago - Pushed at: over 2 years ago - Stars: 7 - Forks: 3

Viswalahiri/Results_Fetcher

Facilitates retrieval of large amounts of data from GITAM's results platform

Language: Python - Size: 43 KB - Last synced at: about 2 years ago - Pushed at: over 5 years ago - Stars: 7 - Forks: 5

schmwong/APAC-McDelivery-Menu-Logger

Automatically scrapes McDelivery menu data and records it for future visualisation projects

Language: Python - Size: 36.9 MB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 6 - Forks: 2

dbaio/portsfallout

An easy way to search/query the FreeBSD pkg-fallout archive

Language: Python - Size: 445 KB - Last synced at: 5 days ago - Pushed at: 2 months ago - Stars: 6 - Forks: 0

Saadmairaj/OAuthBrowser

This module will let you authenticate OAuth 2.0 via the system browser and get the response URL.

Language: Python - Size: 20.5 KB - Last synced at: 23 days ago - Pushed at: about 2 years ago - Stars: 6 - Forks: 0

1x6/lezhin-downloader

Simple requests based manhwa downloader which uses the Lezhin API.

Language: Python - Size: 13.7 KB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 6 - Forks: 2

dori-dev/quotes-crawler

Quotes crawler using scrapy and python.

Language: Python - Size: 13.7 KB - Last synced at: 21 days ago - Pushed at: almost 3 years ago - Stars: 6 - Forks: 0

AymaneLk/Python-Data-Scraping-IMDb-Movies-2020-using-BeautifulSoup

Scraping IMDb Movie 2020 site Using BeautifulSoup

Language: Python - Size: 13.7 KB - Last synced at: over 1 year ago - Pushed at: almost 4 years ago - Stars: 6 - Forks: 2

Santa-Clara-Imaginarium-Lab/twitter-scraping-with-python

Twitter Scraping with Python!

Language: Python - Size: 36.1 KB - Last synced at: 9 months ago - Pushed at: almost 4 years ago - Stars: 6 - Forks: 2

pylyf/stockscrape

Getting stock price data using a python script.

Language: Python - Size: 23.4 KB - Last synced at: almost 2 years ago - Pushed at: about 5 years ago - Stars: 6 - Forks: 4

M-Taghizadeh/Persian_Question_Answering_Voice2Voice_AI

This repository hosts BonyadAI, a Persian question answering AI Model. We developed an initial web crawler and scraper to gather the dataset. The second phase involved building a machine learning model based on word embeddings and NLP techniques. This AI model operates end-to-end, receiving user voice input and providing responses in Persian voice.

Language: Jupyter Notebook - Size: 89.4 MB - Last synced at: about 22 hours ago - Pushed at: 10 months ago - Stars: 5 - Forks: 3

ShiqinHuo/wuhan_house_price_crawler

武汉东湖高新片区光谷&软件园二手房房价爬虫。data source: 房天下

Language: Jupyter Notebook - Size: 14 MB - Last synced at: about 2 years ago - Pushed at: about 6 years ago - Stars: 5 - Forks: 2

TheNoiselessNoise/csfd_scraper

Simple scraper for CSFD.cz, a Czech movie database.

Language: Python - Size: 141 KB - Last synced at: about 7 hours ago - Pushed at: about 2 months ago - Stars: 4 - Forks: 1

shallvhack/Flight-Tracker

Flight Tracker: Real-time flight updates and interactive map for seamless tracking and staying in the loop.

Language: Python - Size: 20.5 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 4 - Forks: 1

epythonlab/github-search-tool

Github Repository Search Tool

Language: Python - Size: 281 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 4 - Forks: 1

danielfrees/scrapemed

ScrapeMed: Data scraping for PubMed Central.

Language: Python - Size: 13.6 MB - Last synced at: 9 months ago - Pushed at: over 1 year ago - Stars: 4 - Forks: 1

spartypkp/legalAI

LegalAI is a passion project which explores and simplifies the complexities of obtaining legal information using LLMs.

Language: Jupyter Notebook - Size: 4.29 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 4 - Forks: 0

pierreleocadie/ScrapeGram 📦

OSINT tool to scraping data from someone's Instagram account.

Language: Python - Size: 62.5 KB - Last synced at: almost 2 years ago - Pushed at: over 2 years ago - Stars: 4 - Forks: 5