X-SLAYER/Website-Cloner

It allows you to download a website from the Internet to a local directory, building recursively all directories, getting HTML, images, and other files from the server to your computer.

Language: Visual Basic .NET - Size: 1.11 MB - Last synced: about 1 month ago - Pushed: about 1 year ago - Stars: 269 - Forks: 83

oxylabs/web-scraper-api-guide

A quick-start guide on using Web Scraper API

Size: 18.6 KB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 2 - Forks: 1

oxylabs/web-scraping-php

A tutorial and code samples of web scraping with PHP

Language: PHP - Size: 23.4 KB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 8 - Forks: 3

flulemon/sneakpeek

Sneakpeek is a framework that helps to quickly and conviniently develop scrapers. It’s the best choice for scrapers that have some specific complex scraping logic that needs to be run on a constant basis

Language: Python - Size: 19.7 MB - Last synced: about 1 month ago - Pushed: 10 months ago - Stars: 35 - Forks: 0

vlmaier/marvel-snap-scrapr

Scraper for https://marvelsnapzone.com to retrieve metadata of Marvel SNAP cards.

Language: Python - Size: 31.3 KB - Last synced: about 2 months ago - Pushed: about 2 months ago - Stars: 21 - Forks: 5

sergeymusenko/simple-crawler

Simple website crawler to get Meta tags and <H1> on Python

Language: Python - Size: 20.5 KB - Last synced: 4 months ago - Pushed: 4 months ago - Stars: 0 - Forks: 0

JohnDiGriz/WebstoreParser

Parses data using json file as instruction and writes to SQL server database

Language: C# - Size: 16.6 KB - Last synced: 6 months ago - Pushed: over 1 year ago - Stars: 0 - Forks: 0

spypunk/sponge

sponge is a website crawler and links downloader command-line tool

Language: Kotlin - Size: 267 KB - Last synced: 6 months ago - Pushed: 7 months ago - Stars: 0 - Forks: 0

oxylabs/ecommerce-scraper-api-guide

A tutorial on using Oxylabs' E-Commerce Scraper

Size: 18.6 KB - Last synced: about 1 month ago - Pushed: about 1 month ago - Stars: 2 - Forks: 1

MLArtist/WebScraper

Python-based web crawling script with randomized intervals, user-agent rotation, and proxy server IP rotation to outsmart website bots and prevent blocking.

Language: Python - Size: 40 KB - Last synced: 8 months ago - Pushed: 8 months ago - Stars: 19 - Forks: 7

vlOd2/LightshotScraper

The most advanced Lightshot (or prnt.sc) scraper ever!

Language: Java - Size: 3.35 MB - Last synced: 6 months ago - Pushed: 6 months ago - Stars: 1 - Forks: 3

AmaanHaider/News-crawler

Language: JavaScript - Size: 3.71 MB - Last synced: 9 months ago - Pushed: 9 months ago - Stars: 0 - Forks: 0

dinocajic/bash-crawler

Created a website-crawler in bash. Note, it's for a specific website and will not work unless you know the site.

Language: Shell - Size: 19.5 KB - Last synced: 10 months ago - Pushed: about 7 years ago - Stars: 0 - Forks: 0

github-1970/link-crawler

Web Link Crawler: A Python script to crawl websites and collect links based on a regex pattern. Efficient and customizable.

Language: Python - Size: 31.3 KB - Last synced: 11 months ago - Pushed: 11 months ago - Stars: 0 - Forks: 0

vlOd2/ImgurScraper

The most advanced Imgur scraper ever!

Language: Java - Size: 189 KB - Last synced: about 1 year ago - Pushed: about 1 year ago - Stars: 0 - Forks: 0

radityaharya/sitesweeper

Sitesweeper is a python package to help you automate your web scraping process, outputting pages to a file

Language: Python - Size: 9.77 KB - Last synced: about 13 hours ago - Pushed: about 1 year ago - Stars: 0 - Forks: 0

ZKAW/website-crawler

Recursive website crawler

Language: Python - Size: 2.93 KB - Last synced: almost 1 year ago - Pushed: about 2 years ago - Stars: 1 - Forks: 0

Deependra-Patel/websiteCrawler

Crawls a website to generate insights

Language: Go - Size: 11.7 KB - Last synced: 9 months ago - Pushed: about 5 years ago - Stars: 3 - Forks: 0

foomo/walker

walks websites

Language: Go - Size: 188 KB - Last synced: 3 months ago - Pushed: over 1 year ago - Stars: 2 - Forks: 0

Dyzio18/java-web-bot-library

Java website crawler - library for analyze and testing websites

Language: Java - Size: 885 KB - Last synced: about 1 year ago - Pushed: over 2 years ago - Stars: 1 - Forks: 0

Mediashare/crawler

:dizzy: Crawl urls from a webpage and provide a DomCrawler with Scraper Library

Language: PHP - Size: 40 KB - Last synced: 28 days ago - Pushed: over 1 year ago - Stars: 3 - Forks: 1

Hem1700/Website-crawler

Language: Python - Size: 6.07 MB - Last synced: about 1 year ago - Pushed: about 3 years ago - Stars: 0 - Forks: 0

MattMoony/image-grabber

Grabs images off webpages.

Language: Python - Size: 1.95 KB - Last synced: about 1 year ago - Pushed: over 5 years ago - Stars: 1 - Forks: 0

shubham-gaur/Crawler

Crawler for "www.mydala.com"

Language: Python - Size: 37.1 KB - Last synced: about 1 year ago - Pushed: over 5 years ago - Stars: 0 - Forks: 0