BaseMax/my-site-url-finders

A simple Python-based web crawler that extracts and filters URLs from a given website while avoiding unwanted paths and file types. The crawler follows links recursively within the same domain and provides a clean list of URLs found across the website.

Language: Python - Size: 24.4 KB - Last synced at: 8 days ago - Pushed at: 2 months ago - Stars: 1 - Forks: 0