GitHub topics: data-gathering
vil/H4X-Tools
Open source toolkit for scraping, OSINT and more.
Language: Python - Size: 3.53 MB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 593 - Forks: 79
fhamborg/news-please
news-please - an integrated web crawler and information extractor for news that just works
Language: Python - Size: 2.99 MB - Last synced at: 6 days ago - Pushed at: about 1 month ago - Stars: 2,340 - Forks: 448
Cacti/cacti
Cacti ™
Language: PHP - Size: 273 MB - Last synced at: 10 days ago - Pushed at: 10 days ago - Stars: 1,766 - Forks: 426
darsan-in/Job-Crawler
The Job Crawler is an integral component of the Job RAID project, designed to automatically scrape and collect data from various job listing websites. This crawler enables Job RAID to aggregate comprehensive job listings, ensuring that users have access to up-to-date and relevant job opportunities.
Language: Python - Size: 6.83 MB - Last synced at: 9 days ago - Pushed at: 12 months ago - Stars: 7 - Forks: 1
OSINT-TECHNOLOGIES/dpulse
DPULSE - Tool for complex approach to domain OSINT
Language: Python - Size: 1.56 MB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 153 - Forks: 10
Garboko/zoka-api
Zoka API is the backend REST API for the Zoka ecosystem - a modern, open-source data collection platform designed for field surveys, research projects, and data gathering operations.
Size: 40 KB - Last synced at: 17 days ago - Pushed at: 18 days ago - Stars: 0 - Forks: 0
anjupriya-v/employee-data-gatherer
Developed using angular. It has no local storage or database storage. It was developed just for learning purpose
Language: TypeScript - Size: 1.4 MB - Last synced at: about 2 months ago - Pushed at: almost 3 years ago - Stars: 2 - Forks: 0
shadawck/glit
Retrieve all mails of users related to a git repository, a git user or a git organization
Language: Rust - Size: 266 MB - Last synced at: 12 days ago - Pushed at: over 1 year ago - Stars: 53 - Forks: 7
asamir12/DECI_Lvl3_Wrangling_And_Analyze_Data_Project
DECI-Udacity Lvl3 Wrangling and Analyze Data Project
Language: HTML - Size: 2.25 MB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0
tatthinhle97/data-gathering-and-warehousing Fork of MelissaLaurino/DSSA-5102_Spring2025
Assignments in Data Gathering & Warehousing class at Stockton University
Language: Jupyter Notebook - Size: 24.6 MB - Last synced at: 5 months ago - Pushed at: 7 months ago - Stars: 0 - Forks: 0
aadityasikder/Object-Detection-with-raspberry-pi-implementing-TinyML-models
Repository for Raspberry Pi-based object detection with TinyML models like TensorFlow Lite, PyTorch Nano, including data gathering, mAP evaluation, and image data preparation in Jupyter notebooks.
Language: Jupyter Notebook - Size: 35.7 MB - Last synced at: 22 days ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
Decodo/Decodo
HTTP(S)/SOCKS5 rotating residential proxies - code examples & general information.
Language: Java - Size: 320 KB - Last synced at: 6 months ago - Pushed at: 6 months ago - Stars: 1,125 - Forks: 43
iwansal64/instaf1nder-py
An open source Instagram profile lookup.
Language: Python - Size: 17.6 KB - Last synced at: 6 months ago - Pushed at: 7 months ago - Stars: 1 - Forks: 0
ibrahimceyisakar/hotel-finder
Hotel finder system with Python includes data gathering, analyzing, and visualization.
Language: Python - Size: 30.3 KB - Last synced at: 28 days ago - Pushed at: 8 months ago - Stars: 0 - Forks: 0
speckly/sucorn
ML/DL dataset collection utilities
Language: Python - Size: 663 KB - Last synced at: 10 months ago - Pushed at: 10 months ago - Stars: 4 - Forks: 2
DanielOladipupo/Earth-To-Home-Agricultural-Research-Analysis
An Insightful Analysis given to an Agricultural Start-Up Company in South Africa for Decision Making
Size: 10.7 KB - Last synced at: 10 months ago - Pushed at: 10 months ago - Stars: 0 - Forks: 0
rafa-br34/MCSF
Minecraft Server Finder is a small toolkit which helps in finding Minecraft servers and tracking players using the "sample" parameter.
Language: Python - Size: 5.37 MB - Last synced at: 10 months ago - Pushed at: 10 months ago - Stars: 1 - Forks: 0
iabdullah215/ReconPro
This tool performs a comprehensive security reconnaissance on a given domain, gathering information such as subdomains, SSL certificate details, open ports, HTTP headers, WHOIS data, and more. It generates a detailed JSON report of the findings for further analysis.
Language: Python - Size: 16.6 KB - Last synced at: 3 months ago - Pushed at: 10 months ago - Stars: 0 - Forks: 0
viralvaghela/Jwiki
Java tool to get wikipedia data
Language: Java - Size: 601 KB - Last synced at: 7 months ago - Pushed at: over 3 years ago - Stars: 36 - Forks: 1
lucaromagnoli/dataservice
Python async data gathering
Language: Python - Size: 630 KB - Last synced at: 12 months ago - Pushed at: 12 months ago - Stars: 11 - Forks: 0
sondosaabed/Oil-vs-BigTech-stock-investigation
💹📈Investigating the oils market prices in addition to the stock market prices between the start of 2001 to the end of 2023. 💰📉
Language: Jupyter Notebook - Size: 10.2 MB - Last synced at: 7 months ago - Pushed at: over 1 year ago - Stars: 5 - Forks: 0
tampucci/nautilos-CS-public
Application to send data acquired through Citizen Science activities to Nautilos erddap data server
Language: JavaScript - Size: 602 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0
ranjithkumarravikumar52/master-thesis
A process to parse survey data to elastic search to perform data visualization on Kibana
Language: JavaScript - Size: 105 MB - Last synced at: about 1 year ago - Pushed at: over 7 years ago - Stars: 1 - Forks: 1
garvitjain-02/Data-Gathering_DataScience
All Data Gathering Techniques are implemented here from scratch! Welcome to this Data Gathering techniques repository which is a comprehensive guide for anyone looking to understand how to gather data by various methods.
Language: Jupyter Notebook - Size: 3.23 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
grip-on-software/data-gathering
Modules used to gather data from different data sources in software development processes
Language: Python - Size: 2.15 MB - Last synced at: about 1 month ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
AqueeqAzam/web-scraping-for-data-gathering-and-mining
Web scraping is used by data mining experts and hackers to imitate conventional browsers and visit websites by following their hypertext structure. They then extract HTML content and data according to predetermined settings and store the data in local databases.
Language: Jupyter Notebook - Size: 4.88 KB - Last synced at: 8 months ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
AyaFergany/Netflix-Project
Language: Jupyter Notebook - Size: 3.15 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
alx3dev/ctfc 📦
Crypto to Fiat Currency data gathering
Language: Ruby - Size: 143 KB - Last synced at: 26 days ago - Pushed at: over 3 years ago - Stars: 1 - Forks: 0
Dhrumi-Kansara-1/data-wrangling-diabetes-patient-dataset
Applying data wrangling steps such as gathering data, assessing data and cleaning data on diabetes patient dataset
Language: Jupyter Notebook - Size: 92.8 KB - Last synced at: over 1 year ago - Pushed at: almost 3 years ago - Stars: 0 - Forks: 0
Dhrumi-Kansara-1/data-gathering-movies-dataset
Data gathering using api and web scrapping
Language: Jupyter Notebook - Size: 104 KB - Last synced at: over 1 year ago - Pushed at: almost 3 years ago - Stars: 0 - Forks: 0
Dhrumi-Kansara-1/data-wrangling-audible-dataset
Data gathering, Assessing and cleaning on audible dataset
Language: Jupyter Notebook - Size: 38.1 KB - Last synced at: over 1 year ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0
Sadiq-marcelo/investigate-TMDB-movies-dataset
Investigate TMDB Movies Dataset Project
Language: Jupyter Notebook - Size: 3.29 MB - Last synced at: over 1 year ago - Pushed at: about 2 years ago - Stars: 0 - Forks: 0
rdempsey/data-gathering-and-wrangling
Code and slides for my class: Data Gathering & Wrangling
Language: Python - Size: 24.6 MB - Last synced at: over 1 year ago - Pushed at: over 10 years ago - Stars: 5 - Forks: 11
nagen1/teamportal
Ideas, Collaborate and Data Analytics
Language: HTML - Size: 3.09 MB - Last synced at: over 1 year ago - Pushed at: about 7 years ago - Stars: 1 - Forks: 0
nicolaswehmeyer/corpcrawler
A sales assistant for automatic information gathering on enterprise accounts
Language: Python - Size: 2.53 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
ParthThakur/Wrangle-WeRateDogs
Identifying and Cleaning issues found in the @WeRateDogs twitter archive.
Language: HTML - Size: 1.9 MB - Last synced at: over 1 year ago - Pushed at: almost 7 years ago - Stars: 0 - Forks: 0
Corruptex/booru-dataset-gatherer
A .NET Core 3.1 Console application to gather tags and relevant information from Booru websites for Machine Learning.
Language: C# - Size: 59.6 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 4 - Forks: 1
sorrychoe/covid19today
Today's World covid-19 Data Gathering Tool
Language: R - Size: 5.73 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 0
m-nanda/ilt-1
Language: Jupyter Notebook - Size: 1.19 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
dbrennand/twitter-stream-bot-data-gatherer
An application to watch the Twitter stream and send accounts to the Botometer API for analysis. The results are stored in a SQLite database.
Language: Python - Size: 35.2 KB - Last synced at: 13 days ago - Pushed at: over 3 years ago - Stars: 5 - Forks: 0
dannyskillzz/TonyElumelu
This project presents an analysis of Mr. Tony Elumelu’s engagements on LinkedIn for the first half of 2023 (January to June, 2023).
Size: 3.77 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
guptaharshnavin/WeRateDogs-Dog-Rating-Tweets-Analysis
Detailed Data Wrangling and Analysis of tweets of Dog Ratings from WeRateDogs.
Language: Jupyter Notebook - Size: 870 KB - Last synced at: over 1 year ago - Pushed at: almost 5 years ago - Stars: 0 - Forks: 1
manbau10/Data_wrangling
This report details the steps employed for wrangling the data used in the "WeRateDogs" project. The three steps in the wrangling phase of data analysis – gathering, assessing, and cleaning – were strictly followed.
Language: HTML - Size: 1.84 MB - Last synced at: almost 2 years ago - Pushed at: about 3 years ago - Stars: 0 - Forks: 0
Osman-28/Coffee-Sales-EXCEL-ANALYSIS-
Data Analysis Poroject
Size: 391 KB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0
ankitgmishra/MachineLearning
Continuously deep diving in understanding & advancing my expertise in Machine Learning through ongoing education and hands on experience with practical learning.
Language: Jupyter Notebook - Size: 103 MB - Last synced at: 4 months ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0
SamAndPel/grab-stick
A set of python utilities to automatically exfiltrate system data.
Language: Python - Size: 39.1 KB - Last synced at: almost 2 years ago - Pushed at: about 5 years ago - Stars: 1 - Forks: 0
lamthuyvo/social-media-data-scripts
Language: Python - Size: 2.08 MB - Last synced at: almost 2 years ago - Pushed at: over 2 years ago - Stars: 242 - Forks: 87
mariannapalaia/london_bike_sharing
An end to end project from data gathering to data visualization using pandas library in Python and Tableau.
Language: Python - Size: 1.15 MB - Last synced at: about 2 years ago - Pushed at: about 2 years ago - Stars: 0 - Forks: 0
mariannapalaia/coffee_sales_dashboard_excel
Dynamic Dashboard reporting Coffee Sales over Time, Coffee Sales by Country and Top 5 Customers leveraging dynamic Charts, Timeline and Slicers.
Size: 558 KB - Last synced at: about 2 years ago - Pushed at: about 2 years ago - Stars: 0 - Forks: 0
tarzahedi/imdb-scraper
Scraper for IMDBA top 1000 movies!
Language: Jupyter Notebook - Size: 13.7 KB - Last synced at: about 2 years ago - Pushed at: over 3 years ago - Stars: 1 - Forks: 0
gondsm/bum
BUM (Bayesian User Model): A User Modelling Technique for Learning from Distributed Devices.
Language: Python - Size: 131 KB - Last synced at: about 2 years ago - Pushed at: over 7 years ago - Stars: 1 - Forks: 0
calista-ai/crowdsourcing-app
A Web Application to collect data from pairwise image comparisons via crowdsourcing
Language: JavaScript - Size: 3.17 MB - Last synced at: over 1 year ago - Pushed at: over 2 years ago - Stars: 4 - Forks: 2
Shamoo100/Web-Automation-For-survey-responses
A project on Product Market Research
Language: Jupyter Notebook - Size: 8.71 MB - Last synced at: about 2 years ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0
DialRC/PortalAPI
Portal Tutorial
Language: Python - Size: 143 KB - Last synced at: over 2 years ago - Pushed at: over 7 years ago - Stars: 11 - Forks: 4
kdvalin/ipmi-exporter
Exports IPMI sensor information to a CSV
Language: Python - Size: 16.6 KB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0
amirhossein-bayati/Scientist-Clustering-Index
Using kmeans algorithm for clustering Google scholar professors into three distinct clusters.
Language: Jupyter Notebook - Size: 153 MB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 8 - Forks: 0
chrislicodes/Udacity-Data-Analyst-Nanodegree
Repository for the projects needed to complete the Data Analyst Nanodegree.
Language: Jupyter Notebook - Size: 93.1 MB - Last synced at: over 2 years ago - Pushed at: over 6 years ago - Stars: 34 - Forks: 22
BahramJannesar/CafebazzarWebsiteScraper
Data gathering from https://cafebazaar.ir
Language: Python - Size: 578 KB - Last synced at: over 2 years ago - Pushed at: over 5 years ago - Stars: 5 - Forks: 1
Sadiq-marcelo/twitter-WeRateDogs-data-analysis-project
Investigate Twitter WeRateDogs Data Analysis Project
Language: HTML - Size: 1.66 MB - Last synced at: over 1 year ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0
rhart-rup/Foursquare-Venue-Scraper-2023
Find all venues in a geographic area using the Foursquare API and collect extended Foursquare data on each venue.
Language: Jupyter Notebook - Size: 37.5 MB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 1 - Forks: 0
petermeissner/diffrprojects
Language: R - Size: 4.52 MB - Last synced at: about 1 year ago - Pushed at: almost 5 years ago - Stars: 2 - Forks: 0
Coool/Cacti Fork of Cacti/cacti
Cacti ™ (Latvian and Russian translator contribution)
Language: PHP - Size: 165 MB - Last synced at: over 2 years ago - Pushed at: almost 3 years ago - Stars: 1 - Forks: 0
MohammadAnas5/Scrape-ambition-box
In this website Job seekers can write and share company reviews and interview stories
Language: Jupyter Notebook - Size: 8.79 KB - Last synced at: over 2 years ago - Pushed at: almost 3 years ago - Stars: 0 - Forks: 0
Yara-Aldajjani/Wrangle-and-Analyze-WeRateDogs-Data
This is Udacity's Data Analyst Nanodegree's 5th project; which is Wrangling and Analyzing WeRateDogs Twitter account.
Language: Jupyter Notebook - Size: 3.57 MB - Last synced at: over 2 years ago - Pushed at: almost 5 years ago - Stars: 4 - Forks: 1
Am-I-Going-On-Holiday/CovidDashboardScraper
🧹 Python web scraper for the Coronavirus.data.gov.uk dashboard
Language: Python - Size: 38.1 KB - Last synced at: over 2 years ago - Pushed at: about 3 years ago - Stars: 0 - Forks: 0
subahanii/COVID19-tracker
This is for Indian cases and data gathering from Indian governments site.
Language: HTML - Size: 23.7 MB - Last synced at: over 2 years ago - Pushed at: over 5 years ago - Stars: 5 - Forks: 0
TelRich/Web_Scrapping_with_BeautifulSoup-and-Wptool
Web scraping Webometrics and Wlkilpedia using Python (Beautiful soup and Wptools) to make a list of top 100 Universities in nigeria
Language: HTML - Size: 318 KB - Last synced at: over 2 years ago - Pushed at: about 3 years ago - Stars: 1 - Forks: 0
TomiJames/Wrangle-and-Analyze-Data
This repo contains files that show the different steps in data wrangling - gathering, assessment and cleaning. The use case is to wrangle data for WeRateDogs twitter account.
Language: HTML - Size: 1.72 MB - Last synced at: over 2 years ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 1
panchis7u7/EMG_Data_Parser
GUI Client for recolecting EGM sensor data from a raspberry PI data emitter through serial comunication and parsing it to an available for format.
Language: C++ - Size: 39.1 KB - Last synced at: over 2 years ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0
Factobotics/FlexHex-Rose-AP
ROSE-AP of the FlexHex project tries to make data gathering from Orion Context Broker entities to Influx-db easier.
Language: Python - Size: 23.4 MB - Last synced at: almost 2 years ago - Pushed at: over 4 years ago - Stars: 1 - Forks: 2
ZawszeBaka/face_recognition
Face Detection => Data Gathering => Training => Face Recognition
Language: Python - Size: 636 KB - Last synced at: over 2 years ago - Pushed at: about 5 years ago - Stars: 2 - Forks: 0
openpilot-community/opc-web
Seeking Maintainers. The official codebase for the openpilot community info portal.
Language: JavaScript - Size: 29.8 MB - Last synced at: over 2 years ago - Pushed at: over 6 years ago - Stars: 4 - Forks: 1
yehia55/We-rate-dogs-data-wrangling
It was the project of the udacity data analysis nano degree, my job was to gather data from Twitter API programmatically with python then assess it and discover its quality and tidiness issues then clean these issues with pandas and finally, I did some analysis and visualization to this data to make insights of it
Language: Jupyter Notebook - Size: 1.62 MB - Last synced at: over 1 year ago - Pushed at: over 4 years ago - Stars: 0 - Forks: 0
Hazem0h/Twitter-Data-Wrangling-Udacity
This is a project in the Udacity Data professional Nanodegree, sponsored by fwd. In this project, I gathered data from different sources, assessed the data, and cleaned it. In the end, I created extracted some insights from that clean data, with visualizations.
Language: Jupyter Notebook - Size: 1.7 MB - Last synced at: over 2 years ago - Pushed at: over 4 years ago - Stars: 0 - Forks: 0
Yara-Aldajjani/Analyze_Online_Job_Postings
This repository has the Udacity data wrangling exercise, which helps us practice the wrangling process.
Language: Jupyter Notebook - Size: 13.7 MB - Last synced at: over 2 years ago - Pushed at: almost 5 years ago - Stars: 0 - Forks: 0
abhi18av/acm-icpc-problems-aggregator
Language: Clojure - Size: 20.5 KB - Last synced at: 9 days ago - Pushed at: over 5 years ago - Stars: 0 - Forks: 0
kleinpa/keyboardtime
Foreground application logger for Windows
Language: Python - Size: 209 KB - Last synced at: over 2 years ago - Pushed at: about 8 years ago - Stars: 2 - Forks: 0
wwengm/findar
Financial Datareader
Language: Python - Size: 49.8 KB - Last synced at: about 1 month ago - Pushed at: over 8 years ago - Stars: 2 - Forks: 1
ctroller/rlfantasy
Custom Rocket League Fantasy League Stats Aggregator
Language: Java - Size: 27.3 KB - Last synced at: over 2 years ago - Pushed at: over 7 years ago - Stars: 2 - Forks: 0
Cerberus-ik/rr-live
This is our entry for the 2017 Riot api challange.
Language: Java - Size: 2.63 MB - Last synced at: over 1 year ago - Pushed at: almost 8 years ago - Stars: 1 - Forks: 0
joedarby/AlcoSensing
An Android app for mobile sensing research
Language: Java - Size: 244 KB - Last synced at: over 2 years ago - Pushed at: over 8 years ago - Stars: 3 - Forks: 0
grctest/BOINC_Scripts
Language: Shell - Size: 18.6 KB - Last synced at: about 2 months ago - Pushed at: almost 9 years ago - Stars: 0 - Forks: 3
