Topic: "structured-data-storage"
simonpierreboucher/Crawler
A robust, modular web crawler built in Python for extracting and saving content from websites. This crawler is specifically designed to extract text content from both HTML and PDF files, saving them in a structured format with metadata.
Language: Python - Size: 87.9 KB - Last synced at: about 2 months ago - Pushed at: 6 months ago - Stars: 0 - Forks: 0
Related Topics
concurrent-crawling
1
content-extraction
1
data-collection
1
data-extraction-pipeline
1
data-preservation-and-recovery
1
data-scraping
1
error-handling
1
html-parsing
1
http-requests
1
metadata-storage
1
modular-design
1
pdf-text-extraction
1
python-crawler
1
rate-limiting
1
text-processing
1
url-normalization
1
web-crawling
1
yaml-configuration
1