sparkstreaming | Topic | Ecosyste.ms: Repos

Topic: "sparkstreaming"

LinMingQiang/sparkstreaming

:boom: :rocket: 封装sparkstreaming动态调节batch time(有数据就执行计算)；:rocket: 支持运行过程中增删topic；:rocket: 封装sparkstreaming 1.6 - kafka 010 用以支持 SSL。

Language: Scala - Size: 258 KB - Last synced at: about 1 year ago - Pushed at: about 4 years ago - Stars: 182 - Forks: 83

amir-rahnama/pyspark-twitter-stream-mining

Real-time Machine Learning with Apache Spark on Twitter Public Stream

Language: Python - Size: 11.2 MB - Last synced at: about 1 year ago - Pushed at: almost 8 years ago - Stars: 68 - Forks: 31

liumingmusic/HadoopLearning

全套大数据基础学习教程，包含最基础的centos、maven。大数据主要包含hdfs、mr、yarn、hbase、kafka、scala、sparkcore、sparkstreaming、sparksql。教程包含所有的源代码演示以及在线文档说明。

Language: Scala - Size: 5.95 MB - Last synced at: over 1 year ago - Pushed at: over 2 years ago - Stars: 52 - Forks: 24

LinMingQiang/spark-utils

:boom: :alien: :hotsprings::rocket:Encapsulated spark 与其他组件的结合api，方便使用，例如 es，hbase，kudu，kafka，mq等

Language: Scala - Size: 168 KB - Last synced at: about 1 year ago - Pushed at: over 5 years ago - Stars: 35 - Forks: 14

lei-zuquan/java_spark

Spark 2.x 案例操作：Scala版本与 Java1.8lambda版代码示例。涵盖Spark核心技术操作SparkCore、SparkSql、SparkStreaming。同时提供了Spark高级性能优化、序列化、广播变量、数据倾斜、算子优化、JVM优化、troubleshooting、数据倾斜解决方案。是多年来根据工作积累整理出来！

Language: Java - Size: 9.59 MB - Last synced at: about 2 years ago - Pushed at: almost 3 years ago - Stars: 22 - Forks: 3

Captain-SpongeBob/MovieRecommendSystem

电影推荐系统，包括基于ALS、LFM的离线推荐、实时推荐，基于Spark

Language: Scala - Size: 237 MB - Last synced at: about 2 years ago - Pushed at: over 4 years ago - Stars: 18 - Forks: 7

jgperrin/net.jgp.books.spark.ch10

Spark in Action, 2e - chapter 10 - Ingestion through structured streaming

Language: Java - Size: 145 KB - Last synced at: 7 days ago - Pushed at: over 3 years ago - Stars: 15 - Forks: 14

keks51/spark-salesforce

spark salesforce connector

Language: Scala - Size: 251 KB - Last synced at: 22 days ago - Pushed at: about 4 years ago - Stars: 5 - Forks: 0

elouardyabderrahim/Development-of-a-Real-Time-Movie-Recommendation-Pipeline

The project aims to design and implement a real-time movie recommendation system using the EK Stack (Elasticsearch and Kibana), Kafka, and a personalized recommendation API to enhance the user experience on Jay-Zz Entertainment's streaming platform.

Size: 32.2 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 3 - Forks: 0

karthikchaganti/Stockafolio-Insight-Project

A Realtime Stock Portfolio Manager built using Apache Distributed Technologies!

Language: Python - Size: 40.7 MB - Last synced at: over 1 year ago - Pushed at: about 8 years ago - Stars: 3 - Forks: 1

swarna0712/San-Fransisco-Crime-Classification-using-PySpark

Big Data Project - SSML - Spark Streaming for Machine Learning

Language: Python - Size: 21.5 KB - Last synced at: almost 2 years ago - Pushed at: over 3 years ago - Stars: 2 - Forks: 1

yucl80/avrodemo

write , append avro to hdfs file

Language: Scala - Size: 14.6 KB - Last synced at: about 1 month ago - Pushed at: about 8 years ago - Stars: 2 - Forks: 1

ZGG2016/data-warehouse

数仓理论和项目（离线、实时）

Language: Java - Size: 146 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 1

jrphub/jrphub.github.io

Blog on github pages with Jekyll

Language: SCSS - Size: 2.4 MB - Last synced at: about 1 year ago - Pushed at: about 2 years ago - Stars: 1 - Forks: 1

ludengke95/spark-streaming-kafka-template

SparkStreaming新手友好向模板，简化SparkStreaming开发

Language: Java - Size: 114 KB - Last synced at: about 2 years ago - Pushed at: almost 3 years ago - Stars: 1 - Forks: 1

paulatumwine/hashtag-stream

Track trending hashtags on Twitter in real time.

Language: Python - Size: 9.77 MB - Last synced at: about 2 years ago - Pushed at: over 4 years ago - Stars: 1 - Forks: 0

MehdiTAZI/Spark-POCS

Full poc on spark 2, Spark RDD, Spark DStream, Spark SQL, Spark Datasets & DataFrames & Spark Structured Streaming [SCALA][SPARK]

Language: Scala - Size: 12.7 KB - Last synced at: about 1 year ago - Pushed at: almost 7 years ago - Stars: 1 - Forks: 2

jakeeqsb/EntreeETLPipeline_spark_streaming

Language: Python - Size: 18 MB - Last synced at: about 2 years ago - Pushed at: about 7 years ago - Stars: 1 - Forks: 0

neema233/Real_Time_Kafka_Spark_Streaming

A Dockerized Kafka system for streaming server metrics and load balancer logs, with data processing using Spark and storage in a relational database and Hadoop.

Language: Java - Size: 142 KB - Last synced at: 9 months ago - Pushed at: 9 months ago - Stars: 0 - Forks: 0

camilobetanieto/BigDataComputing

Big data computing tasks conducted with PySpark. The problems involve MapReduce and Streaming algorithms.

Language: Python - Size: 27.3 KB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 0 - Forks: 0

deepanshu-yadav/stream_processing_project_s23

This project gets data from Spotify API , ingests into kafka for streaming and processes it through spark streaming. All this is done on Azure.

Language: Python - Size: 801 KB - Last synced at: 12 months ago - Pushed at: 12 months ago - Stars: 0 - Forks: 0

vfx-beavers/de-sprint-8

Потоковая обработка данных

Language: Python - Size: 16.6 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

24jmwangi/coingecko-streamapp

a streaming app and a dashboard for visualizing cryptocurrency data fetched from the CoinGecko API. The streaming app retrieves real-time cryptocurrency information using Spark Streaming and stores it in a PostgreSQL database.

Language: Python - Size: 174 KB - Last synced at: 6 months ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

alaahgag/Real-Time-Sales-Data-Analysis-Application

A real-time sales data analysis Application using Spark Structured Streaming, Kafka as a messaging system, PostgreSQL as a storage for processed data, and Superset for creating a dashboard.

Language: Python - Size: 15.3 MB - Last synced at: about 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0