Ecosyste.ms: Repos

An open API service providing repository metadata for many open source software ecosystems.

GitHub topics: markdown-crawler

paulpierre/markdown-crawler

A multithreaded 🕸️ web crawler that recursively crawls a website and creates a 🔽 markdown file for each page, designed for LLM RAG

Language: Python - Size: 1.14 MB - Last synced: 22 days ago - Pushed: 7 months ago - Stars: 168 - Forks: 16

Related Keywords
html-to-markdown 1 html-to-markdown-converter 1 html2md 1 llm 1 llmops 1 markdown 1 markdown-crawler 1 markdown-parser 1 markdown-scraper 1 md-crawler 1 rag 1 web-scraper 1