GitHub topics: apify | Ecosyste.ms: Repos

apify/crawlee

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Language: TypeScript - Size: 143 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 18,568 - Forks: 891

apify/apify-sdk-python

The Apify SDK for Python is the official library for creating Apify Actors in Python. It provides useful features like actor lifecycle management, local storage emulation, and actor event handling.

Language: Python - Size: 8.55 MB - Last synced at: 2 days ago - Pushed at: 3 days ago - Stars: 136 - Forks: 13

apify/actor-whitepaper-web

Documentation site for the Actor Programming Model – a fresh take on serverless microapps. Built with Astro.

Language: MDX - Size: 101 MB - Last synced at: 2 days ago - Pushed at: 3 days ago - Stars: 4 - Forks: 0

apify/apify-cli

Apify command-line interface helps you create, develop, build and run Apify actors, and manage the Apify cloud platform.

Language: TypeScript - Size: 17.7 MB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 144 - Forks: 28

apify/apify-zapier-integration

Apify integration for Zapier

Language: JavaScript - Size: 567 KB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 14 - Forks: 2

apify/crawlee-python

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.

Language: Python - Size: 29.9 MB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 5,947 - Forks: 404

systemfsoftware/youtube-autocomplete-scraper

YouTube AutoComplete Scraper - An Apify actor that scrapes YouTube's search suggestions with intelligent deduplication using pglite and trigram similarity matching. Perfect for content research, SEO, and trend analysis.

Language: TypeScript - Size: 867 KB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 6 - Forks: 1

apify/apify-sdk-js

Apify SDK monorepo

Language: TypeScript - Size: 314 MB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 146 - Forks: 48

apify/actor-content-checker

You can use this act to monitor any page's content and get a notification when content changes.

Language: JavaScript - Size: 86.9 KB - Last synced at: 10 days ago - Pushed at: about 3 years ago - Stars: 21 - Forks: 17

apify/actor-scraper

House of Apify Scrapers. Generic scraping actors with a simple UI to handle complex web crawling and scraping use cases.

Language: JavaScript - Size: 22.5 MB - Last synced at: 10 days ago - Pushed at: over 2 years ago - Stars: 122 - Forks: 48

apify/actor-scrapy-executor

Apify actor to run web spiders written in Python in the Scrapy library

Language: Python - Size: 316 KB - Last synced at: 10 days ago - Pushed at: over 2 years ago - Stars: 11 - Forks: 5

apify/super-scraper

Generic REST API for scraping websites. Drop-in replacement for ScrapingBee, ScrapingAnt, and ScraperAPI services. And it is open-source!

Language: TypeScript - Size: 101 KB - Last synced at: 10 days ago - Pushed at: 8 months ago - Stars: 29 - Forks: 7

apify/actor-scrapy-books-example

Example of Python Scrapy project. It scrapes book data from https://books.toscrape.com/.

Language: Python - Size: 90.8 KB - Last synced at: 10 days ago - Pushed at: 19 days ago - Stars: 3 - Forks: 1

apify/apify-client-python

Apify API client for Python

Language: Python - Size: 3.57 MB - Last synced at: 10 days ago - Pushed at: 11 days ago - Stars: 71 - Forks: 14

devblack/curlx

CurlX a basic Curl syntax

Language: PHP - Size: 70.3 KB - Last synced at: 18 days ago - Pushed at: 18 days ago - Stars: 14 - Forks: 13

apify/actor-legacy-phantomjs-crawler

The actor implements the legacy Apify Crawler product. It uses PhantomJS headless browser to recursively crawl websites and extract data from them using a piece of JavaScript code.

Language: JavaScript - Size: 1020 KB - Last synced at: 10 days ago - Pushed at: over 2 years ago - Stars: 8 - Forks: 4

pocesar/actor-shopify-scraper

Automate monitoring prices on the most popular solution for building online stores and selling products online. Crawl arbitrary Shopify-powered online stores and extract a list of all products in a structured form, including product title, price, description, etc

Language: JavaScript - Size: 36.1 KB - Last synced at: 22 days ago - Pushed at: about 2 years ago - Stars: 14 - Forks: 12

Luna-BLox/Luna-Crawler

✨🎮 Turn a Roblox Page into a JSON/CSV

Language: JavaScript - Size: 158 KB - Last synced at: 25 days ago - Pushed at: 25 days ago - Stars: 1 - Forks: 0

docaxess/php-apify-sdk

Community driven PHP SDK for Apify

Language: PHP - Size: 145 KB - Last synced at: 28 days ago - Pushed at: 28 days ago - Stars: 1 - Forks: 1

MrXujiang/crawel

基于Apify+node+react搭建的有点意思的爬虫平台

Language: JavaScript - Size: 9.09 MB - Last synced at: about 4 hours ago - Pushed at: about 5 years ago - Stars: 42 - Forks: 15

JuroOravec/crawlee-one

Professional scrapers that provide full control to the users. Crawlee One builds on top of Crawlee and Apify and extends them with features for robust and highly configurable web scrapers.

Language: TypeScript - Size: 1.96 MB - Last synced at: 2 days ago - Pushed at: about 1 year ago - Stars: 31 - Forks: 2

DoniLite/holder-bing

the official crawler for the ghostify platform service

Language: TypeScript - Size: 84 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 0 - Forks: 0

superryeti/Hands-on-WebScraping

This repo is a part of blog series on several web scraping projects where we will explore scraping techniques to crawl data from simple websites to websites using advanced protection.

Language: Python - Size: 74.2 KB - Last synced at: 27 days ago - Pushed at: over 2 years ago - Stars: 81 - Forks: 73

ma593y/Seeklyzer-Dash-App

AI-powered job search platform featuring natural language queries, semantic matching, and automated resume assessment. Built with Python, integrates OpenAI GPT, xAI Grok, and ChromaDB for intelligent job-candidate compatibility analysis.

Language: Python - Size: 88.9 KB - Last synced at: 2 months ago - Pushed at: 2 months ago - Stars: 0 - Forks: 0

Azquaier/phone-number-formatter

Bulk phone number formatter: clean & convert to E.164, International, National & RFC3966. Configurable region, batch, concurrency & rate-limit. Perfect for CRMs, SMS campaigns, and data migration.

Language: Python - Size: 1.03 MB - Last synced at: 2 months ago - Pushed at: 2 months ago - Stars: 0 - Forks: 0

systemfsoftware/essentials

Essential TypeScript libraries and utilities for building production-ready AI SaaS applications

Language: TypeScript - Size: 224 KB - Last synced at: 18 days ago - Pushed at: about 1 month ago - Stars: 2 - Forks: 0

RamiWalidou/Apify-Scraper

Extract complete data from the Apify Store marketplace, including actor details, pricing, statistics, and user information. Features multi-query support, state persistence, and customizable filters. Perfect for market research, competitor analysis, and discovering the best project tools.

Language: JavaScript - Size: 69.3 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

NakerTheFirst/Sentiment-analysis

Analyse social media sentiment of OpenAI using LinkedIn data with NLP and transfer learning

Language: Python - Size: 24.4 MB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 1 - Forks: 0

Azquaier/imprint-contact-scraper

An Apify Actor that automatically finds and extracts company information (name, address, phone, email, optional decision-makers) from German Impressum pages.

Language: Python - Size: 97.7 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

bala-ceg/ai-travelassistant

AI Travel Assistant is a powerful AI-powered tool that helps users plan their trips effortlessly. It integrates flight search, hotel booking, and sightseeing recommendations to generate a structured travel itinerary using LangChain, Apify, and OpenAI

Language: Python - Size: 25.4 KB - Last synced at: 6 days ago - Pushed at: 5 months ago - Stars: 2 - Forks: 1

thenetaji/apify-actors

Actors to ease your Apify journey through automation, scrapping and more...

Language: JavaScript - Size: 353 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0

VaclavRut/actor-amazon-crawler

Amazon crawler - this configuration will extract items for a keywords that you will specify in the input, and it will automatically extract all pages for the given keyword. You can specify more keywords on the input for one run.

Language: JavaScript - Size: 144 KB - Last synced at: 2 months ago - Pushed at: over 4 years ago - Stars: 76 - Forks: 35

apify/apify-haystack

The official integration for Apify and Haystack 2.0

Language: Python - Size: 133 KB - Last synced at: 10 days ago - Pushed at: 8 months ago - Stars: 3 - Forks: 0

metalwarrior665/actor-rust-scraper

Experimental scraper in Rust suited for running locally or on the Apify platform. Inspired by Apify SDK.

Language: Rust - Size: 170 KB - Last synced at: 4 months ago - Pushed at: over 1 year ago - Stars: 13 - Forks: 3

lhotanok/zoot-scraper

Apify actor extracting data from Zoot.cz, Zoot.sk and Zoot.ro

Language: TypeScript - Size: 79.1 KB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 1 - Forks: 1

ZinRicky/nesc-code-dump

All the scripts and data for the “Network Science” group project (A.Y. 2024-25)

Language: Python - Size: 88.3 MB - Last synced at: 6 months ago - Pushed at: 6 months ago - Stars: 0 - Forks: 2

Strajk/actor-github-repositories-search-scraper 📦

Apify actor for extracting repositories from GitHub based on search queries

Language: JavaScript - Size: 52.7 KB - Last synced at: 8 days ago - Pushed at: over 4 years ago - Stars: 5 - Forks: 0

metalwarrior665/actor-keywords-extractor

Extract list of all keywords fro many website

Language: JavaScript - Size: 33.2 KB - Last synced at: about 2 months ago - Pushed at: almost 5 years ago - Stars: 4 - Forks: 1

SatyamVyas04/Prizey

Prizey simplifies online shopping with real-time price tracking, comparisons, alerts, and insights, ensuring users get the best deals effortlessly.

Language: JavaScript - Size: 464 KB - Last synced at: 4 months ago - Pushed at: 6 months ago - Stars: 0 - Forks: 0

Nikolay-Lysenko/servifier 📦

An easy-to-use tool for making web service with API from your own Python functions.

Language: Python - Size: 30.3 KB - Last synced at: 3 days ago - Pushed at: 7 months ago - Stars: 8 - Forks: 0

ImBIOS/actor-bun-node-docker

Fork Base Docker images of Apify actors using Bun and Node

Language: Dockerfile - Size: 503 KB - Last synced at: 10 days ago - Pushed at: 10 months ago - Stars: 1 - Forks: 0

kulkultech/asos-crawler

Asos Crawler for Apify

Language: JavaScript - Size: 769 KB - Last synced at: 22 days ago - Pushed at: over 2 years ago - Stars: 4 - Forks: 1

ScaleLeap/zine-not-amazon-scraper 📦

How to Scrape Amazon Search Results

Language: JavaScript - Size: 142 KB - Last synced at: 10 days ago - Pushed at: over 2 years ago - Stars: 7 - Forks: 1

metalwarrior665/actor-article-extractor-smart

Combines Apify's crawling system and article parsing with unfluff library.

Language: JavaScript - Size: 218 KB - Last synced at: 4 months ago - Pushed at: about 1 year ago - Stars: 11 - Forks: 5

Kalana99/web_scrapping

A news and web scraping project using React, Django and OpenAI

Language: JavaScript - Size: 354 KB - Last synced at: 5 months ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

scrapefulldotcom/crunchbase-scraper

Scrape Crunchbase company data reliably without an account.

Size: 6.84 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

pocesar/apify-login-session

Grab a session for any website for usage on your own actor

Language: TypeScript - Size: 437 KB - Last synced at: 9 months ago - Pushed at: about 1 year ago - Stars: 10 - Forks: 4

yanis-git/php-apify-sdk

Unofficial SDK for Apify service

Language: PHP - Size: 43 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

sauermar/web-browser-recorder

Web application for recording, management and editing of inteligent RPA workflows using Playwright technology

Language: TypeScript - Size: 7.86 MB - Last synced at: 12 months ago - Pushed at: almost 3 years ago - Stars: 13 - Forks: 1

lhotanok/zalando-scraper

Apify actor extracting data from Zalando

Language: TypeScript - Size: 149 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 6 - Forks: 3

dreyescabrera/ip_tracker

Track an IP with ipify. Taken from frontendmentor.io

Language: CSS - Size: 84 KB - Last synced at: about 1 year ago - Pushed at: about 3 years ago - Stars: 1 - Forks: 0

NoahFinberg/google_kg_movie_scraper

Uses Apify to scrape Google Search Engine Results pages for all American Films between 1950-2020 and then parses the knowledge panel structured data.

Language: Python - Size: 255 KB - Last synced at: about 1 year ago - Pushed at: over 4 years ago - Stars: 3 - Forks: 0

ShruAgarwal/Activeloop-LangChain-CourseBot

Ask questions related to langchain course brought by Activeloop.

Language: Python - Size: 10.7 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

petrpatek/airbnb-scraper

Apify public actor for scraping Airbnb homes.

Language: JavaScript - Size: 761 KB - Last synced at: 4 months ago - Pushed at: over 2 years ago - Stars: 11 - Forks: 6

maxCopell/tripadvisor-scraper

Scrape Tripadvisor restaurant, hotels, and places.

Language: JavaScript - Size: 420 KB - Last synced at: over 1 year ago - Pushed at: almost 3 years ago - Stars: 47 - Forks: 26

apify-projects/store-website-checker

Analyzes target website for anti-scraping protections and performance. Saves screenshots/HTML snapshots.

Language: TypeScript - Size: 689 KB - Last synced at: about 1 year ago - Pushed at: over 1 year ago - Stars: 6 - Forks: 5

aweworkz/html-web-media-scraper

Extracts various media files, such as images, videos, audio, and other related media elements, from multiple websites. It then provides the corresponding descriptions or the alt="" content. You may need to use proxies to run this actor for some websites with bot blocking features.

Size: 3.91 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

redoper1/covid-sk-3

Apify.com actor for scraping data from website https://www.arcgis.com/apps/opsdashboard/index.html#/5fe83e34abc14349b7d2fcd5c48c6c85

Language: JavaScript - Size: 77.1 KB - Last synced at: about 2 months ago - Pushed at: over 1 year ago - Stars: 2 - Forks: 0

bernardro/actor-youtube-scraper

Apify actor to scrape Youtube search results. You can set the maximum videos to scrape per page as well as the date from which to start scraping.

Language: JavaScript - Size: 293 KB - Last synced at: over 1 year ago - Pushed at: almost 3 years ago - Stars: 23 - Forks: 19

chaimaebouyarmane/Bank-Sentiment-Analysis

Our project leverages data mining techniques, including Apify, Airflow, BERT, PostgreSQL, and Power BI, to conduct in-depth sentiment analysis for bank branches. We successfully identify customer sentiments, highlight positive and negative aspects, and provide valuable insights for enhancing the customer experience and gaining a competitive edge.

Language: Python - Size: 728 KB - Last synced at: over 1 year ago - Pushed at: almost 2 years ago - Stars: 1 - Forks: 1

metalwarrior665/actor-google-sheets

No more dealing with Google API. Simple Node.js program to automate access to Google Sheets.

Language: JavaScript - Size: 576 KB - Last synced at: 4 months ago - Pushed at: almost 3 years ago - Stars: 16 - Forks: 4

G-Glitch404/investing.com-scraper

scrape news articles from investing.com as much as you want

Language: Python - Size: 1.83 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 0

CodeLeom/docker-ppt

This code runs puppeteer in a docker environment. After the docker build, a simple puppeteer script will visit a website, take a screenshot and save it.

Language: Dockerfile - Size: 2.93 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

cermak-petr/act-anti-captcha-recaptcha

Apify act for solving google recaptcha using the anti-captcha.com service.

Language: JavaScript - Size: 21.5 KB - Last synced at: over 1 year ago - Pushed at: almost 3 years ago - Stars: 8 - Forks: 8

wchen02/apify-etl-server

Scrape data using apify, normalize and load data into db.

Language: JavaScript - Size: 547 KB - Last synced at: over 1 year ago - Pushed at: over 2 years ago - Stars: 2 - Forks: 0

Coelurus/bezRealitky-Apify-Scraper

Scrap all important propety market info in just few clicks

Language: JavaScript - Size: 802 KB - Last synced at: almost 2 years ago - Pushed at: almost 4 years ago - Stars: 1 - Forks: 0

AakashPandey/json-viewer

View rendered json data fetched using apify google-maps-scraper https://apify.com/compass/crawler-google-places

Language: Svelte - Size: 463 KB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0

pocesar/actor-twitter-scraper Fork of vdrmota/actor-twitter-scraper

Scrape any Twitter user profile. Extract tweets, retweets, replies, favorites, and conversation threads with no Twitter API limits

Language: TypeScript - Size: 244 KB - Last synced at: 2 months ago - Pushed at: almost 3 years ago - Stars: 12 - Forks: 11

bijankhazaei/apify-sdk-scraper

scrap shop with apify and expressjs

Language: JavaScript - Size: 56.6 KB - Last synced at: almost 2 years ago - Pushed at: over 3 years ago - Stars: 1 - Forks: 0

scrapians/airbnb-by-location-scraper

Scrapes the properties details of airbnb by searching it's location developed for Apify.

Size: 1000 Bytes - Last synced at: about 2 years ago - Pushed at: about 2 years ago - Stars: 0 - Forks: 0

nicooprat/nicooprat

My own website, built with Nuxt 3, Vite 3, Tailwind 3, Puppeteer & Netlify

Language: Vue - Size: 24.5 MB - Last synced at: 4 months ago - Pushed at: almost 3 years ago - Stars: 1 - Forks: 2

pocesar/apify-covid19-brazil

Get Coronavirus epidemilogy data for Brazil in CSV

Language: JavaScript - Size: 278 KB - Last synced at: 9 months ago - Pushed at: over 2 years ago - Stars: 4 - Forks: 2

lhotanok/erasmus-plus-organisation-scraper

Search organisations from Erasmus+ and European Solidarity Corps programmes. Define search criteria such as legal name, business name, hyperlink, PIC or an OID from: https://webgate.ec.europa.eu/erasmus-esc/index/organisations/search-for-an-organisation

Language: TypeScript - Size: 122 KB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 2 - Forks: 0

g-pets/unsplash-scraper

Apify actor that crawls Unsplash photos.

Language: JavaScript - Size: 910 KB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0

mstephen19/apify-click-events

Like TypeScript, but for clicking ;) Manage automated clicks, and ensure your Apify web-crawler is only clicking exactly what you allow it to

Language: TypeScript - Size: 205 KB - Last synced at: 4 months ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0

mstephen19/apify-global-store

Easily access, manage, and manipulate a global state/multiple global states for an Apify actor's run

Language: TypeScript - Size: 363 KB - Last synced at: 4 months ago - Pushed at: over 2 years ago - Stars: 1 - Forks: 1

0xjgv/act-classifier

Naive & Logistic regression classifiers. (Supervised learning)

Language: JavaScript - Size: 6.84 KB - Last synced at: almost 2 years ago - Pushed at: over 7 years ago - Stars: 1 - Forks: 0

0xjgv/apify-code-base

Apify's best practices, creative solutions, error handling and more.

Language: JavaScript - Size: 18.6 KB - Last synced at: almost 2 years ago - Pushed at: over 7 years ago - Stars: 2 - Forks: 0

0xjgv/act-pdf-crawl

Apify.com Act that extracts text from a PDF files.

Language: JavaScript - Size: 146 KB - Last synced at: almost 2 years ago - Pushed at: about 7 years ago - Stars: 1 - Forks: 0

ganevdev/actor-webdesignernews-scraper

Scraper for www.webdesignernews.com, using Apify.

Language: TypeScript - Size: 513 KB - Last synced at: 4 months ago - Pushed at: about 6 years ago - Stars: 6 - Forks: 0

asowder3943/craigslist-scraper

A Simple Craigslist Scraper Created using Crawlee to Test the Apify Platform

Language: TypeScript - Size: 271 KB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0

asowder3943/r6-map-scraper

A Scraper for https://www.ubisoft.com/en-us/game/rainbow-six/siege/game-info/maps

Language: JavaScript - Size: 79.1 KB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0

lhotanok/erasmus-plus-project-scraper

Searches Erasmus+ projects from: https://erasmus-plus.ec.europa.eu/projects/search

Language: TypeScript - Size: 142 KB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0

lhotanok/fler-marketplace-scraper

Extracts products from sellers at Fler.cz

Language: TypeScript - Size: 71.3 KB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0

seifbhxc65/Strass

this a university java project made with javaFX and mysql for the database , its idea is to find the nearest and most rated restaurants in a provided region in tunisia based on apify which a website used for web scraping , I've used it to scrape google maps.

Language: Java - Size: 93.8 KB - Last synced at: about 2 years ago - Pushed at: over 3 years ago - Stars: 1 - Forks: 0

nicooprat/nicooprat-old

My previous website. Using Nuxt, Apify & Netlify.

Language: Vue - Size: 693 KB - Last synced at: 4 months ago - Pushed at: over 2 years ago - Stars: 1 - Forks: 0

opop999/TI_monitoring_social_media_communication

Monitoring of political and media communication before 2021 elections in the Czech Republic for Transparency International. This repository focuses on the extraction and analysis of political communication on Facebook, Yotube, and Twitter through the use of hlidacstatu.cz API and Twitter API.

Language: HTML - Size: 1.21 GB - Last synced at: over 2 years ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0