BaseMax/my-site-url-finders

A simple Python-based web crawler that extracts and filters URLs from a given website while avoiding unwanted paths and file types. The crawler follows links recursively within the same domain and provides a clean list of URLs found across the website.

Language: Python - Size: 24.4 KB - Last synced at: about 16 hours ago - Pushed at: 5 months ago - Stars: 1 - Forks: 0