GitHub topics: real-time-data-pipeline
aymane-maghouti/Real-Time-Data-Pipeline-Using-Kafka
This project implements a real-time data pipeline using Apache Kafka, Python's psutil library for metric collection, and SQL Server for data storage. The pipeline collects metrics data from the local computer, processes it through Kafka brokers, and loads it into a SQL Server database. Additionally, a real-time dashboard is created using Power BI.
Language: Python - Size: 380 KB - Last synced at: 14 days ago - Pushed at: over 1 year ago - Stars: 11 - Forks: 3
kishlayjeet/Stock-Market-Real-Time-Data-Pipeline-with-Apache-Kafka-and-Cassandra
A end-to-end real-time stock market data pipeline with Python, AWS EC2, Apache Kafka, and Cassandra Data is processed on AWS EC2 with Apache Kafka and stored in a local Cassandra database.
Language: Python - Size: 2.3 MB - Last synced at: 15 days ago - Pushed at: almost 2 years ago - Stars: 25 - Forks: 7
aymane-maghouti/Real-Time-Streaming-Kafka-Debezium-Spark-Streaming
This project demonstrates real-time data streaming and processing architecture using Kafka, Spark Streaming, and Debezium for capturing CDC (Change Data Capture) events. The pipeline collects transaction data, processes it in real time, and updates a dashboard to display real-time analytics for smartphone data.
Language: Java - Size: 9.02 MB - Last synced at: 17 days ago - Pushed at: 6 months ago - Stars: 7 - Forks: 4
velvirt/Data_Collection_and_Storage_-SQL-
A Data Collection and Storage project using SQL from TripleTen
Size: 1000 Bytes - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 0 - Forks: 0
stereosky/community-highlights Fork of quixio/community-highlights
Collecting highlights from the Quix community and social media in the form of interesting questions, comments, challenges, solutions and insights
Language: Python - Size: 2.14 MB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0
quixio/community-highlights
Collecting highlights from the Quix community and social media in the form of interesting questions, comments, challenges, solutions and insights
Language: Python - Size: 2.14 MB - Last synced at: 8 months ago - Pushed at: 8 months ago - Stars: 1 - Forks: 1
thecodemancer/Apache-Beam
🔥👨💻 Build Big data pipelines with Apache Beam in any language and run it via Spark, Flink, GCP (Google Cloud Dataflow).
Language: Jupyter Notebook - Size: 321 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0
