GitHub / faizpuad / DataEngineeringProject-DocumentStreamingWithData

The core objective of this project is to build an end-to-end data streaming pipeline that processes this dataset in real-time. By leveraging modern data engineering tools and techniques, we aim to connect, buffer, process, store, and visualize streaming data. This allows for better understanding of data flows, handling of large-scale real-time data

JSON API: http://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/faizpuad%2FDataEngineeringProject-DocumentStreamingWithData

Topics: data-engineering, document-streaming, fastapi, kafka, mon, spark-streaming-kafka, streamlit

• Files
• Readme
• Dependencies