GitHub topics: data-pipeline-monitoring
elementary-data/dbt-data-reliability
dbt package that is part of Elementary, the dbt-native data observability solution for data & analytics engineers. Monitor your data pipelines in minutes. Available as self-hosted or cloud service with premium features.
Language: Python - Size: 7.73 MB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 430 - Forks: 103
DataProphet/pdlog
Logging for pandas
Language: Python - Size: 33.2 KB - Last synced at: 21 days ago - Pushed at: almost 5 years ago - Stars: 8 - Forks: 0
sanketrs/implementation-of-modern-data-engineering-architecture-with-fabric_analytics
Building a next-generation hybrid data pipeline architecture that combines the power of Microsoft Fabric, Azure Cloud, and Power BI. This pipeline is engineered to tackle the challenges of real-time data ingestion, multi-layered processing, and analytics, delivering business-critical insights.
Language: Python - Size: 32.2 KB - Last synced at: 28 days ago - Pushed at: 4 months ago - Stars: 1 - Forks: 0
allamiro/Data-Pipelines
Every thing about designing installing and implementing data pipelines to include kafka zookeeper hadoop If you enjoy my content please consider supporting what I do Thank you.
Language: Jinja - Size: 4.48 MB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 1 - Forks: 0
DrDroidLab/kenobi
Easiest way to monitor asynchronous data pipelines
Language: Python - Size: 2.47 MB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 55 - Forks: 4
shubhamjais40/AWS-Data-Pipeline-Project-Implementing-Data-Validation-Using-Lambda-based-Gluecrawler-v1.0
This Project demonstrates the Technology shift in Automobile Firm to resolve the data engineering challenge of manual data ops. AWS Cloud Services implemented here as: S3 bucket for lake storage incoming batches, Lambda Python Script for automating the validation function call and Glue Crawler to generate relational table with successful testing.
Language: Python - Size: 347 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 0
