GitHub / miltiadiss / CEID_NE4348-Big-Data-Management-Systems
This project implements a real-time data pipeline with Kafka, Spark, and MongoDB. It generates vehicle data using UXSIM, streams it to a Kafka broker, processes it with Spark, and stores raw and processed data in MongoDB. Queries analyze vehicle counts, speeds, and routes over specified periods.
JSON API: http://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miltiadiss%2FCEID_NE4348-Big-Data-Management-Systems
PURL: pkg:github/miltiadiss/CEID_NE4348-Big-Data-Management-Systems
Stars: 1
Forks: 0
Open issues: 0
License: mit
Language: Python
Size: 4.88 MB
Dependencies parsed at: Pending
Created at: over 1 year ago
Updated at: 5 months ago
Pushed at: 5 months ago
Last synced at: 4 months ago
Topics: kafka-consumer, kafka-producer, pymongo, pysaprk, spark-sql, uxsim