cyberagiinc/DevDocs

Completely free, private, UI based Tech Documentation MCP server. Designed for coders and software developers in mind. Easily integrate into Cursor, Windsurf, Cline, Roo Code, Claude Desktop App

Language: TypeScript - Size: 3.33 MB - Last synced at: about 10 hours ago - Pushed at: 8 days ago - Stars: 1,576 - Forks: 150

Gint367/webscraping_marketing

scraping fixed asset from Jahresabschluss and machine keywords from the company websites

Language: Python - Size: 42.9 MB - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 0 - Forks: 1

MalikMalcolm1/PuppetMaster

Puppeteer & Crawl4AI microservice for web automation, scraping, and AI processing with Bull queues

Language: Python - Size: 41 KB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 1 - Forks: 0

bigsk1/supa-crawl-chat

Integrates Supabase with Crawl4AI and AI Chat to create a powerful web crawling and semantic search solution. Streamlit supabase data visualization. Run all in Docker. API and more!

Language: Python - Size: 1.69 MB - Last synced at: 8 days ago - Pushed at: 29 days ago - Stars: 14 - Forks: 1

PangPangGod/crawl4ai-sample-codes

crawl4ai sample codes

Language: Python - Size: 0 Bytes - Last synced at: 10 days ago - Pushed at: 10 days ago - Stars: 0 - Forks: 0

NhanPhamThanh-IT/Deepnews-Summarizer

A feature-rich web application for automated news scraping and summarization. It allows users to enter article URLs, fetches the full content, and generates concise summaries. The system supports both local inference with custom models and remote deployment via FastAPI or Streamlit interfaces.

Language: Jupyter Notebook - Size: 309 KB - Last synced at: 13 days ago - Pushed at: 13 days ago - Stars: 1 - Forks: 0

mzazakeith/PuppetMaster

Puppeteer & Crawl4AI microservice for web automation, scraping, and AI processing with Bull queues

Language: Python - Size: 47.9 KB - Last synced at: 16 days ago - Pushed at: 23 days ago - Stars: 3 - Forks: 0

lymagics/generic-ai-scraper

Just provide schema and let your AI scrape

Language: Python - Size: 4.88 KB - Last synced at: 27 days ago - Pushed at: 27 days ago - Stars: 0 - Forks: 0

Jeanetted3v/Web-Crawler-Playground

A playground to testing out website crawling tools

Language: Python - Size: 16.6 KB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 0 - Forks: 0

wenkil/langgraph_study_web_agent

LangGraph Web Agent: A LangGraph based intelligent AI assistant that can perform network searches, crawl web content, and intelligently summarize, providing users with a deeper and more comprehensive network information acquisition experience than regular searches.

Language: Python - Size: 43.9 KB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 0 - Forks: 0

tecno-consultores/llm-lab

LLM laboratory

Language: Shell - Size: 203 KB - Last synced at: 22 days ago - Pushed at: 22 days ago - Stars: 3 - Forks: 2

Svantevith/talk-to-your-website

RAG agent enhanced with LLM-optimized website crawler built using Crawl4AI, Langchain, ChromaDB and Ollama.

Language: Python - Size: 189 KB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 0 - Forks: 0

TrueMan777/upwork_scraper

Web-scraper for Upwork jobs using Selenium-driverless

Language: Python - Size: 50.8 KB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 0 - Forks: 0

JamesN-dev/Scroll-Scribe

ScrollScribe is a Python tool designed to scrape internal URLs from targeted website, read those URLs from a text file, scrape the content of each URL using `crawl4ai`, and generate cleaned Markdown output suitable for ingestion into RAG (Retrieval-Augmented Generation) systems or other text-processing pipelines.

Language: Python - Size: 646 KB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 0 - Forks: 0

Karthick-840/Crawl4ai-RAG-with-Local-LLM

A tool for scraping web documentation using Crawl4AI, converting it to Markdown, and preparing it for integration with local LLMs (like Ollama) to enhance their knowledge for learning and "vibe coding" workflows.

Language: Python - Size: 33.1 MB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 0 - Forks: 0

e-d-i-n-i/ai-data-extraction

AI-driven system for structured data extraction, storage, and vector search, leveraging Crawl4AI, PydanticAI, and Supabase to enable efficient retrieval and RAG-based AI applications.

Size: 2.93 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 0 - Forks: 0

thevladdo/rag-backend

Retrieval-Augmented Generation server with Pinecone and OpenAI

Language: HTML - Size: 46.9 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 0 - Forks: 0

oussemabenhassena5/Crawl4DeepSeek

Crawl4DeepSeek = Crawl4AI + DeepSeek 🚀 Smart, efficient, and built for deep web exploration! 🌐🤖

Language: Python - Size: 23.4 KB - Last synced at: about 2 months ago - Pushed at: 4 months ago - Stars: 7 - Forks: 1

Swish78/linkedin-insider-

Linkedin Info scrapper using crawl4ai

Language: Python - Size: 13.7 KB - Last synced at: about 1 month ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

sidhyaashu/scrapy-deepseek

Scrap any website using deepseek

Language: Python - Size: 12.7 KB - Last synced at: about 10 hours ago - Pushed at: 4 months ago - Stars: 1 - Forks: 0

yashpinjarkar10/Pinescript-Agent

PineScript Agentic RAG system that provides an interactive chat interface to answer questions about PineScript. It combines a Streamlit-based UI with an asynchronous AI agent to retrieve and summarize documentation stored in a Supabase database.

Language: Python - Size: 25.4 KB - Last synced at: 9 days ago - Pushed at: 3 months ago - Stars: 1 - Forks: 0

kaymen99/ai-web-scraper

AI web scraper built with Crawl4AI for extracting structured leads data from websites.

Language: Python - Size: 19.5 KB - Last synced at: 3 months ago - Pushed at: 4 months ago - Stars: 14 - Forks: 1

balaji1233/WEB_MASTER

AI tool to transforms any URL into a structured knowledge source by: extracting content using Crawl4AI ,vectorizing and summarizing data , running Retrieval-Augmented Generation (RAG) for deep information discovery, enabling a smart chatbot for interactive Q&A.

Language: Python - Size: 8.79 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

laurentvv/crawl4ai-mcp

Web crawling tool that integrates with AI assistants via the MCP

Language: Python - Size: 0 Bytes - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

shubhampandit/ai-web-scraper

Web Scraper using Gen-AI

Language: Python - Size: 7.81 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

GreatHayat/yt-shorts-generator

Generate YouTube Shorts Video From an Article Link using Pydantic AI

Language: Python - Size: 21.5 KB - Last synced at: 2 months ago - Pushed at: 4 months ago - Stars: 4 - Forks: 0

ebrown-32/webcrawleb

URL to Markdown for AI Processing. Provide better quality input to LLMs for better output.

Language: CSS - Size: 42 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0

Saurabh7636/crawl2md

A streamlined Python tool that crawls websites and converts them into organized Markdown files.

Language: Python - Size: 3.91 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0

sitamgithub-MSIT/crawl4ai-txtfy

Easily retrieve the full-text content from web pages using Crawl4AI.

Language: Python - Size: 389 KB - Last synced at: 2 months ago - Pushed at: 6 months ago - Stars: 1 - Forks: 0

muzzlol/review-radar

Language: TypeScript - Size: 14.6 MB - Last synced at: 6 months ago - Pushed at: 6 months ago - Stars: 0 - Forks: 0

varunsaagar/crawlwithagents

The Web Metadata Extraction Toolkit is designed to streamline the process of extracting, cleaning, and analyzing metadata from websites. Utilizing advanced AI models and custom extraction strategies, this toolkit helps users efficiently gather data like titles, descriptions, and keywords, which are crucial for SEO and content strategy.

Language: Python - Size: 9.77 KB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 0 - Forks: 0