EthicalML/kafka-spark-streaming-zeppelin-docker

One click deploy docker-compose with Kafka, Spark Streaming, Zeppelin UI and Monitoring (Grafana + Kafka Manager)

Size: 1020 KB - Last synced at: 13 days ago - Pushed at: almost 4 years ago - Stars: 120 - Forks: 74

faizpuad/DataEngineeringProject-DocumentStreamingWithData

The core objective of this project is to build an end-to-end data streaming pipeline that processes this dataset in real-time. By leveraging modern data engineering tools and techniques, we aim to connect, buffer, process, store, and visualize streaming data. This allows for better understanding of data flows, handling of large-scale real-time data

Language: Jupyter Notebook - Size: 1.8 MB - Last synced at: 16 days ago - Pushed at: 6 months ago - Stars: 1 - Forks: 1

chandreshsutariya/bigdata---train-analysis

batch processing and realtime tains(railway) data analysis to help Station Masters refreshing each 20 seconds

Language: Python - Size: 4.08 MB - Last synced at: about 1 month ago - Pushed at: 7 months ago - Stars: 0 - Forks: 0

pereldegla/twitter-trend-sentiment-analysis-world-cup-use-case

How to get closer to the audience using Twitter: an use case following the France football team run during the 2022 World Cup

Language: Jupyter Notebook - Size: 28.3 KB - Last synced at: 8 months ago - Pushed at: over 2 years ago - Stars: 1 - Forks: 1

hieuung/Streaming-Kafka

Using various data processing tool for real time data pipeline with Kafka

Language: Python - Size: 4.68 MB - Last synced at: about 2 months ago - Pushed at: 12 months ago - Stars: 0 - Forks: 0

viyadb/viyadb-spark

Data processing ang ingestion backend for ViyaDB based on Spark streaming

Language: Scala - Size: 182 KB - Last synced at: 20 days ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 2

roksolana-d/spark-streaming-examples

Research on legacy and structured streaming with Spark

Language: Scala - Size: 22.5 KB - Last synced at: about 1 year ago - Pushed at: about 6 years ago - Stars: 2 - Forks: 2

arpendu11/graph-based-data-lake

An ETL application which is written in Quarkus, Spark SQL Streaming, Neo4j and various types of Databases and stores. It also covers the devops frameworks like Jenkins CI/CD, docker and Kubernetes.

Language: Java - Size: 56.6 KB - Last synced at: 12 months ago - Pushed at: about 2 years ago - Stars: 7 - Forks: 2

tmcgrath/spark-scala

Spark with Scala example projects

Language: Scala - Size: 253 KB - Last synced at: over 1 year ago - Pushed at: about 6 years ago - Stars: 32 - Forks: 42

beyhangl/StructuredSparkStreaming

Spark Streaming with Kafka using Scala

Language: Scala - Size: 182 KB - Last synced at: over 1 year ago - Pushed at: over 6 years ago - Stars: 0 - Forks: 0

trendyol-data-eng-summer-intern-2019/recom-engine-streaming

Streaming component of the project, which is written with Spark Streaming.

Language: Scala - Size: 15.6 KB - Last synced at: over 1 year ago - Pushed at: over 5 years ago - Stars: 0 - Forks: 1

zqhxuyuan/kafka-book

《Kafka技术内幕》代码

Language: Java - Size: 816 KB - Last synced at: over 1 year ago - Pushed at: over 7 years ago - Stars: 195 - Forks: 71

AbdullahMu/Data-Streaming-Nanodegree-Project_02-Evaluate-Human-Balance-with-Spark-Streaming

Design data streaming architecture and API for a real-life application called the Step Trending Electronic Data Interface (STEDI). It is a working application used to assess fall risk for seniors. When a senior takes a test, they are scored using an index which reflects the likelihood of falling, and potentially sustaining an injury in the course of walking. STEDI uses a Redis datastore for risk score and other data. The Data Science team has completed a working graph for population risk at a STEDI clinic. The problem is the data is not populated yet. You will work with Kafka Connect Redis Source events and Business Events to create a Kafka topic containing anonymized risk scores of seniors in the clinic.

Language: Python - Size: 827 KB - Last synced at: about 2 years ago - Pushed at: almost 4 years ago - Stars: 2 - Forks: 0

rajeshsantha/MonitoredStructuredStreaming

Repository for Spark structured streaming use case implementations.

Language: Scala - Size: 65.4 KB - Last synced at: about 2 years ago - Pushed at: about 5 years ago - Stars: 1 - Forks: 1

dharaneeshvrd/spark-examples

Spark Examples

Language: Python - Size: 35.2 KB - Last synced at: 11 days ago - Pushed at: almost 4 years ago - Stars: 2 - Forks: 5

cevatarmutlu/kafka_spark_streaming

Language: Python - Size: 5.48 MB - Last synced at: over 1 year ago - Pushed at: over 3 years ago - Stars: 3 - Forks: 2

michelheil/BigData

Projects related to Big Data technologies

Language: Java - Size: 2.24 MB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0

deepcloudlabs/dcl700-2021-jun-21

DCL-700: Big Data Essentials

Language: JavaScript - Size: 21.7 MB - Last synced at: about 2 months ago - Pushed at: almost 4 years ago - Stars: 0 - Forks: 0

froblesmartin/BachFinalProject

Project to compare Apache Spark Streaming vs Apache Flink.

Language: Java - Size: 42 KB - Last synced at: about 2 years ago - Pushed at: almost 4 years ago - Stars: 1 - Forks: 1

monyedavid/substance-effects-on-reflexes

substance effects on reflexes

Language: Scala - Size: 5.86 KB - Last synced at: about 2 years ago - Pushed at: almost 4 years ago - Stars: 0 - Forks: 0

ludengke95/spark-streaming-kafka-template

SparkStreaming新手友好向模板，简化SparkStreaming开发

Language: Java - Size: 114 KB - Last synced at: about 2 years ago - Pushed at: almost 3 years ago - Stars: 1 - Forks: 1

fadhilyori/kaspacore

Mata Elang | Data Preprocessing using Scala and Spark

Language: Scala - Size: 9.77 KB - Last synced at: about 2 years ago - Pushed at: over 4 years ago - Stars: 0 - Forks: 0

abhay6694/PySpark-Component

Collection of spark-components functions for big-data processing

Language: Jupyter Notebook - Size: 12.7 KB - Last synced at: about 2 years ago - Pushed at: about 5 years ago - Stars: 0 - Forks: 0

rotemfogel/spark-streaming-app

Spark Streaming Playground

Language: Scala - Size: 521 KB - Last synced at: almost 2 years ago - Pushed at: over 5 years ago - Stars: 0 - Forks: 0

add1993/automatic-document-classification-spark

Streaming news data from the guardian website and classify the news data into different categories like sports, weather, world news, education etc.

Language: Python - Size: 1.88 MB - Last synced at: about 2 years ago - Pushed at: almost 6 years ago - Stars: 0 - Forks: 1

haozhang-x/log-analysis-spark

Structured Streaming Log Analysis

Language: Scala - Size: 72.3 KB - Last synced at: almost 2 years ago - Pushed at: almost 6 years ago - Stars: 2 - Forks: 2

vchoudhari45/spark-kafka-integration

spark-kafka-integration

Language: Scala - Size: 14.6 KB - Last synced at: about 2 years ago - Pushed at: about 6 years ago - Stars: 1 - Forks: 0

nemosharma6/stream-analysis

real time stream processing

Language: Scala - Size: 6.84 KB - Last synced at: about 2 years ago - Pushed at: about 6 years ago - Stars: 0 - Forks: 0

sergei-grigorev/spark-streaming-project

In-Stream final project

Language: Scala - Size: 107 KB - Last synced at: almost 2 years ago - Pushed at: about 7 years ago - Stars: 1 - Forks: 1

memojja/kafka-spark-examples

Learning kafka-spark entegration and streaming data analyze

Language: Java - Size: 42 KB - Last synced at: about 1 month ago - Pushed at: over 7 years ago - Stars: 0 - Forks: 0