dwyl/phoenix-ecto-append-only-log-example

📝 A step-by-step example/tutorial showing how to build a Phoenix (Elixir) App where all data is immutable (append only). Precursor to Blockchain, IPFS or Solid!

Language: Elixir - Size: 219 KB - Last synced at: 28 days ago - Pushed at: about 6 years ago - Stars: 81 - Forks: 7

sanchitvj/sports_betting_analytics_engine

A data and analytics engineering platform designed for real-time sports betting analytics.

Language: Python - Size: 2.72 MB - Last synced at: 2 months ago - Pushed at: 3 months ago - Stars: 24 - Forks: 4

aymane-maghouti/Big-Data-Project

This project aims to predict smartphone prices using a combination of batch and stream processing techniques in a Big Data environment. The architecture follows the Lambda Architecture pattern, providing both real-time and batch processing capabilities to users.

Language: Python - Size: 960 KB - Last synced at: about 2 months ago - Pushed at: about 1 year ago - Stars: 11 - Forks: 2

Azure/azure-cosmosdb-spark 📦

Apache Spark Connector for Azure Cosmos DB

Size: 192 MB - Last synced at: 7 days ago - Pushed at: 3 months ago - Stars: 203 - Forks: 120

KyleRoss/node-lambda-log

Basic logging mechanism for Node.js Lambda Functions and other AWS services, which produces logs in JSON format for easier reading through Cloudwatch Logs.

Language: JavaScript - Size: 3.89 MB - Last synced at: 7 days ago - Pushed at: about 2 years ago - Stars: 195 - Forks: 16

malleum-inc/canari3

Canari v3 - next gen Maltego framework for rapid remote and local transform development

Language: Python - Size: 2.75 MB - Last synced at: about 1 month ago - Pushed at: about 4 years ago - Stars: 144 - Forks: 33

OryxProject/oryx 📦

Oryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning

Language: Java - Size: 7.12 MB - Last synced at: 3 months ago - Pushed at: almost 4 years ago - Stars: 1,781 - Forks: 404

huwngnosleep/complete_lakehouse_techstack

This project implements an end-to-end techstack for a data platform, for local development.

Language: Python - Size: 116 MB - Last synced at: about 1 month ago - Pushed at: 7 months ago - Stars: 2 - Forks: 0

apssouza22/big-data-pipeline-lambda-arch

A full big data pipeline (Lambda Architecture) with Spark, Kafka, HDFS and Cassandra.

Language: Java - Size: 1.84 MB - Last synced at: 5 months ago - Pushed at: about 2 years ago - Stars: 177 - Forks: 84

NashTech-Labs/Lambda-Arch-Spark

Language: Scala - Size: 23.4 KB - Last synced at: about 11 hours ago - Pushed at: almost 5 years ago - Stars: 75 - Forks: 37

Azure-Samples/streaming-at-scale

How to implement a streaming at scale solution in Azure

Language: C# - Size: 23.6 MB - Last synced at: 8 months ago - Pushed at: 8 months ago - Stars: 234 - Forks: 98

ChristophShyper/lambda-airly

AWS Lambda function/application invoking Airly API to get some basic information about current air quality

Language: HCL - Size: 134 KB - Last synced at: about 12 hours ago - Pushed at: 11 months ago - Stars: 2 - Forks: 1

smsraj2001/STREAM-BATCH-PROCESSING-KAFKA-SPARK

A project which includes simulation of real time queries by kafka and performing stream and batch processing of the simulated queries by spark. Also, this follows lambda architecture, in which kafka is publisher and spark helps in subscribing

Language: Python - Size: 3.93 MB - Last synced at: 4 months ago - Pushed at: about 2 years ago - Stars: 3 - Forks: 0

exbotanical/goldmund-sh-i

Retired personal website. Multi-service system automated with Kubernetes, custom headless CMS, and bespoke CDN pipeline.

Language: JavaScript - Size: 2.21 MB - Last synced at: 17 days ago - Pushed at: almost 4 years ago - Stars: 0 - Forks: 0

leimao/Chicago_Community_Crime_App

Chicago Community Crime Big Data App Implemented Using Lambda Architecture

Language: HTML - Size: 43.4 MB - Last synced at: about 1 year ago - Pushed at: over 6 years ago - Stars: 0 - Forks: 0

michalzeman/laPlatform

BigData concept for Lambda Architecture design pattern, Apache Kafka -> for streaming/data pipeline integration, Apache Spark -> for distributed data processing, RESTful, services -> Akka HTTP

Language: Scala - Size: 261 KB - Last synced at: about 1 year ago - Pushed at: about 4 years ago - Stars: 3 - Forks: 0

hmmhmmhm/serverless-ts-template

Serverless Typescript Template

Language: TypeScript - Size: 808 KB - Last synced at: about 2 months ago - Pushed at: over 2 years ago - Stars: 15 - Forks: 3

traveldevel/iothub-ro

Central documents repository of the iot hub landscape

Size: 11.3 MB - Last synced at: over 1 year ago - Pushed at: over 7 years ago - Stars: 0 - Forks: 0

cultureamp/visual-testing-api 📦

This is a serverless REST Api that stores images and does image comparison. It uses Looks Same algo to compare the images

Language: JavaScript - Size: 4.22 MB - Last synced at: about 1 year ago - Pushed at: over 6 years ago - Stars: 6 - Forks: 3

ivarvermeulen/nuxt-serverless-at-edge

Nuxt.js Serverless SSR using Cloudfront, Lambda@edge and S3

Language: Vue - Size: 2.24 MB - Last synced at: over 1 year ago - Pushed at: over 2 years ago - Stars: 32 - Forks: 7

dgkanatsios/GameAnalyticsEventHubFunctionsCosmosDatalake

Big data reference architecture and implementation for an online multiplayer game

Language: JavaScript - Size: 563 KB - Last synced at: 9 days ago - Pushed at: about 1 year ago - Stars: 3 - Forks: 2

FaresKi/Scala-Project

Lambda architecture built in Scala and Java, made with love (and Kafka & Spark).

Language: Scala - Size: 2.33 MB - Last synced at: almost 2 years ago - Pushed at: over 3 years ago - Stars: 1 - Forks: 0

szbartnik/LambdaArchitectureTest

Implementation of Lambda Architecture (Spark + Cassandra) to test its performance vs. RDBMS (PostgreSQL)

Language: Scala - Size: 147 KB - Last synced at: almost 2 years ago - Pushed at: almost 8 years ago - Stars: 0 - Forks: 0

DmitryPukhov/spark-trade

Processing stock exchange data with Apache Spark

Language: Scala - Size: 677 KB - Last synced at: almost 2 years ago - Pushed at: almost 6 years ago - Stars: 3 - Forks: 2

jancajthaml/MKRFox1200 Fork of skoky/MKRFox1200

Temperature sensor using Arduino MKRFox1200 and Google appengine/Go

Language: HTML - Size: 2.95 MB - Last synced at: almost 2 years ago - Pushed at: over 7 years ago - Stars: 0 - Forks: 0

Lucass97/FlightAnalysis

This project implemented a lambda architecture for analyzing domestic flight data in the US from 2009 to 2020. It used Apache Spark for batch processing, Spark Streaming for real-time analysis, and SVM models to predict flight cancellations and delays, with Docker for cluster management and Grafana for real-time visualization.

Language: Jupyter Notebook - Size: 5.66 MB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 1

phatnguyen080401/Real-Estate-Sale-Analytics

Create data pipeline using Lambda architecture with Spark, Kafka, Airflow and Snowflake

Language: Python - Size: 194 MB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 5 - Forks: 0

ymericson/big-data-streets

Big Data webapp using Chicago street congestion, crashes, red light violations, and speed camera violations

Language: Scala - Size: 117 MB - Last synced at: almost 2 years ago - Pushed at: over 4 years ago - Stars: 22 - Forks: 12

phatnguyen080401/Lambda-architecture

Build lambda architecture

Language: Jupyter Notebook - Size: 1.94 MB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 1 - Forks: 1

jayway/going-serverless-workshop

A workshop for creating serverless applications on AWS

Language: Shell - Size: 345 KB - Last synced at: about 2 years ago - Pushed at: over 7 years ago - Stars: 7 - Forks: 2

Yannnyan/BDES

Big Data Enterprise System, Kafka, Integration, Microservices

Language: JavaScript - Size: 305 KB - Last synced at: about 1 year ago - Pushed at: about 2 years ago - Stars: 0 - Forks: 0

v4vigtory/Big-Data-Movie-Recommendation-App

Repository containing codes written for the capstone project on Movie Recommendation Engine for University of Chicago's Big Data Application Architecture course (MPCS 53014), taught in Fall 2022 by Mike Spertus

Language: CSS - Size: 11.4 MB - Last synced at: about 2 years ago - Pushed at: about 2 years ago - Stars: 1 - Forks: 0

rirolli/twitch-big-data-analysis

Secondo progetto per il corso di Big Data durante A.A. 2020/2021. Progetto che si incentra sulla creazione di una architettura lambda per effettuare analisi streaming e batch su dati estratti dalla piattaforma Twitch.

Language: Python - Size: 19.5 MB - Last synced at: over 2 years ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0

data-mill-cloud/data-mill 📦

A K8s-based infrastructure for analytics

Language: Shell - Size: 2.93 MB - Last synced at: about 2 years ago - Pushed at: over 5 years ago - Stars: 24 - Forks: 5

dorianbg/lambda-architecture-demo

Developing a Lambda Architecture pipeline using Apache Kafka, Spark Structured Streaming, Redshift, S3, Python

Language: Jupyter Notebook - Size: 66.4 KB - Last synced at: over 2 years ago - Pushed at: about 5 years ago - Stars: 22 - Forks: 13

LorenzoAgnolucci/TwitterRealTimeSentimentAnalysis

Lambda architecture implementation using Apache Storm, Hadoop and HBase to perform Twitter real-time sentiment analysis

Language: Java - Size: 2.43 MB - Last synced at: over 2 years ago - Pushed at: about 5 years ago - Stars: 4 - Forks: 0

dadoPuccio/Tweets-Sentiment-Analysis

Sentiment Classifier of Tweets, based on Lambda Architecture.

Language: Java - Size: 46.5 MB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0

nuald/lambda-sample

A Sample of Lambda Architecture Project

Language: Scala - Size: 1.09 MB - Last synced at: 2 months ago - Pushed at: about 4 years ago - Stars: 3 - Forks: 1

LorenzoGianassi/Twitter_Sentiment_Analysis_Lambda_Architecture

Full term Project of the exam of Parallel Computing of University of Florence. Implementation of Twitter Sentiment Analysis using Hadoop, Apache Storm and HBase to obtain parallelization.

Language: Java - Size: 661 KB - Last synced at: over 2 years ago - Pushed at: almost 3 years ago - Stars: 0 - Forks: 0

Xakimov1610/dotnet-Back

Language: C# - Size: 11.6 MB - Last synced at: 5 days ago - Pushed at: about 3 years ago - Stars: 1 - Forks: 0

markush81/fast2smart

From Fast to Smart Data - Lambda Architecture with Apache Spark, Kafka and Cassandra

Language: Java - Size: 923 KB - Last synced at: over 2 years ago - Pushed at: almost 3 years ago - Stars: 4 - Forks: 4

dennyglee/azure-cosmosdb-spark Fork of Azure/azure-cosmosdb-spark

Apache Spark Connector for Azure Cosmos DB

Language: Scala - Size: 191 MB - Last synced at: 6 days ago - Pushed at: about 7 years ago - Stars: 5 - Forks: 1

nelsonfleig/lambda-architecture

Juypiter Notebooks to demonstrate the Lambda Architecture with Kafka Streams and Apache Spark

Language: Jupyter Notebook - Size: 14.6 KB - Last synced at: almost 2 years ago - Pushed at: over 5 years ago - Stars: 0 - Forks: 0

vladimirivkovic/lambda-architecture4crypto-analytics

Language: Python - Size: 66.4 KB - Last synced at: over 1 year ago - Pushed at: almost 5 years ago - Stars: 2 - Forks: 1

daxamin/Microservices-Log-Analysis-Monitoring

Lambda Architecture

Language: Python - Size: 12.6 MB - Last synced at: about 2 years ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0

VedangJoshi/Lambda-Architecture

GTFS feed processing using Lambda architecture (Kafka, Zookeeper, MapReduce and Apache Storm)

Language: Java - Size: 432 KB - Last synced at: 7 months ago - Pushed at: almost 7 years ago - Stars: 1 - Forks: 1

adamzhang822/Big-Data-Divvy

Big Data webapp using Chicago divvy bike stations data

Language: Java - Size: 77.8 MB - Last synced at: about 2 years ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0

Akardian/lambda-architecture-for-distributed-mobile-sensor

Lambda architecture for distributed mobile sensor

Language: Java - Size: 5.14 MB - Last synced at: over 2 years ago - Pushed at: over 3 years ago - Stars: 2 - Forks: 1

massudavide/progettoBD2

secondo progetto BigData

Language: Scala - Size: 5.49 MB - Last synced at: over 2 years ago - Pushed at: almost 4 years ago - Stars: 0 - Forks: 0

RaghavJindal13/AWS-Lambda-Triggers

A collection of AWS lambda functions that can be used in different scenarios for automation purposes.

Size: 39.1 KB - Last synced at: almost 2 years ago - Pushed at: about 4 years ago - Stars: 0 - Forks: 0

saLeox/Lambda-Architecture-BigData-Pipeline-Curation

Curation of big data pipeline, applying lambda architecture, combining batching and streaming processing. Use the Springboot and Spark to implement.

Size: 9.77 KB - Last synced at: about 2 years ago - Pushed at: about 4 years ago - Stars: 0 - Forks: 0

jtanwk/big-data-AOT

Big data application using lambda architecture on Apache stack to compare current weather to historical trends.

Language: Java - Size: 433 KB - Last synced at: over 2 years ago - Pushed at: almost 5 years ago - Stars: 1 - Forks: 0

makohn/lambda-architecture-poc

♨️ A PoC implementation of the λ-Architecture for collecting and analysing tweets

Language: Scala - Size: 28.3 KB - Last synced at: 2 months ago - Pushed at: about 7 years ago - Stars: 2 - Forks: 1

adilkhash/awesome-data-engineering

A curated list of resources to navigate data engineering

Size: 0 Bytes - Last synced at: 23 days ago - Pushed at: about 7 years ago - Stars: 3 - Forks: 0

klaytn-hackathon/JSP

SupportStation: The People's Online Petition

Language: JavaScript - Size: 3.62 MB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 1 - Forks: 1

rashi-sahu/Freya

Handling large amount of data and processing it online

Language: Shell - Size: 187 KB - Last synced at: about 2 years ago - Pushed at: over 6 years ago - Stars: 1 - Forks: 0

nickozoulis/thunderstorm

Investigating the trade-offs of low latency responses over quality when applying machine learning algorithms over lambda architecture.

Language: Java - Size: 4.82 MB - Last synced at: 6 months ago - Pushed at: about 9 years ago - Stars: 3 - Forks: 2

nickozoulis/pooka

Implementing an extensible lambda architecture

Language: Java - Size: 312 KB - Last synced at: 6 months ago - Pushed at: over 8 years ago - Stars: 0 - Forks: 0