tberlin-om/robots-txt-checker

robots.txt checker/monitoring - The python-script checks the robots.txt content and statuscode and sents an emails if the check fails

Language: Python - Size: 5.86 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0