GitHub topics: data-engineering-infrastructure
aiden-liu/aiden-liu.github.io
Blog space on data engineering, machine learning, platform engineering.
Size: 170 KB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 0 - Forks: 0
anna-geller/prefect-deployment-patterns
Code examples showing flow deployment to various types of infrastructure
Language: Python - Size: 249 KB - Last synced at: 3 months ago - Pushed at: over 2 years ago - Stars: 107 - Forks: 10
anna-geller/prefect-aws-lambda
Deploy a Prefect flow to serverless AWS Lambda function
Language: Python - Size: 19.5 KB - Last synced at: 5 months ago - Pushed at: almost 3 years ago - Stars: 36 - Forks: 6
anna-geller/dataflow-ops
Project demonstrating how to automate Prefect 2.0 deployments to AWS ECS Fargate
Language: Python - Size: 1.32 MB - Last synced at: 5 months ago - Pushed at: about 2 years ago - Stars: 113 - Forks: 24
anastasiamkh/aws-dataflow-simulator
Python package that simplifies the creation of AWS infrastructure for simulating real-time data streaming and batch processing, ideal for integrating into machine learning projects.
Language: Python - Size: 2.68 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0
mpearmain/data_contracts_sandbox
This is a repo designed to show the workflow of a data contract, with pre-commit hooks and GitHub Actions on how to have the contract power a data platform
Language: Python - Size: 2.64 MB - Last synced at: 9 months ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 0
anna-geller/prefect-getting-started
Get started with Prefect by scheduling your Prefect flows with GitHub Actions
Language: Python - Size: 22.5 KB - Last synced at: 3 months ago - Pushed at: over 2 years ago - Stars: 7 - Forks: 1
bramvdklinkenberg/adf-airflow-data-project
Data engineering project using Azure Data Factory and Apache Airflow
Language: Python - Size: 45.9 KB - Last synced at: over 1 year ago - Pushed at: over 2 years ago - Stars: 2 - Forks: 0
anna-geller/dataflow-ops-aws-eks
Project demonstrating how to automate Prefect 2.0 deployments to AWS EKS
Language: Python - Size: 78.1 KB - Last synced at: 5 months ago - Pushed at: almost 3 years ago - Stars: 8 - Forks: 1
bytes1inger/Beatlytica
This project implements a real-time event streaming pipeline for a music streaming service, inspired by Spotify Wrapped and Billboard charts. The pipeline is powered by Apache Airflow, Apache Kafka, dbt, Docker, GCP, Spark-Streaming, and Terraform.
Language: Python - Size: 86.1 MB - Last synced at: 6 months ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0
