Topic: "airflow"
apache/airflow
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
Language: Python - Size: 385 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 40,585 - Forks: 15,164
argoproj/argo-workflows
Workflow Engine for Kubernetes
Language: Go - Size: 151 MB - Last synced at: about 21 hours ago - Pushed at: about 22 hours ago - Stars: 15,736 - Forks: 3,317
apache/dolphinscheduler
Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code
Language: Java - Size: 209 MB - Last synced at: 8 days ago - Pushed at: 8 days ago - Stars: 13,569 - Forks: 4,819
PipelineAI/pipeline
PipelineAI
Language: Jsonnet - Size: 90.4 MB - Last synced at: 3 days ago - Pushed at: about 1 year ago - Stars: 4,171 - Forks: 970
orchest/orchest
Build data pipelines, the easy way ๐ ๏ธ
Language: TypeScript - Size: 27.2 MB - Last synced at: 21 days ago - Pushed at: about 2 years ago - Stars: 4,124 - Forks: 262
puckel/docker-airflow
Docker Apache Airflow
Language: Shell - Size: 230 KB - Last synced at: 3 months ago - Pushed at: over 2 years ago - Stars: 3,801 - Forks: 545
jghoman/awesome-apache-airflow
Curated list of resources about Apache Airflow
Language: Shell - Size: 550 KB - Last synced at: about 2 months ago - Pushed at: 10 months ago - Stars: 3,775 - Forks: 496
WeBankFinTech/DataSphereStudio
DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.
Language: Java - Size: 243 MB - Last synced at: about 1 month ago - Pushed at: 3 months ago - Stars: 3,166 - Forks: 1,020
elyra-ai/elyra
Elyra extends JupyterLab with an AI centric approach.
Language: Python - Size: 115 MB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 1,930 - Forks: 354
teamclairvoyant/airflow-maintenance-dags
A series of DAGs/Workflows to help maintain the operation of Airflow
Language: Python - Size: 684 KB - Last synced at: 29 days ago - Pushed at: about 1 year ago - Stars: 1,724 - Forks: 397
san089/Udacity-Data-Engineering-Projects
Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.
Language: Python - Size: 2.03 MB - Last synced at: 21 days ago - Pushed at: almost 3 years ago - Stars: 1,620 - Forks: 527
san089/goodreads_etl_pipeline
An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.
Language: Python - Size: 1.31 MB - Last synced at: 28 days ago - Pushed at: over 5 years ago - Stars: 1,378 - Forks: 227
OBenner/data-engineering-interview-questions
More than 2000+ Data engineer interview questions.
Size: 938 KB - Last synced at: 23 days ago - Pushed at: 5 months ago - Stars: 1,341 - Forks: 477
astronomer/dag-factory
Construct Apache Airflow DAGs Declaratively via YAML configuration files
Language: Python - Size: 9.1 MB - Last synced at: 8 days ago - Pushed at: 8 days ago - Stars: 1,308 - Forks: 197
abhishek-ch/around-dataengineering
A Data Engineering & Machine Learning Knowledge Hub
Language: Python - Size: 12.2 MB - Last synced at: 28 days ago - Pushed at: over 1 year ago - Stars: 1,128 - Forks: 225
astronomer/astronomer-cosmos
Run your dbt Core projects as Apache Airflow DAGs and Task Groups with a few lines of code
Language: Python - Size: 18.1 MB - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 969 - Forks: 214
alanchn31/Data-Engineering-Projects
Personal Data Engineering Projects
Language: Jupyter Notebook - Size: 2.92 MB - Last synced at: 27 days ago - Pushed at: over 2 years ago - Stars: 934 - Forks: 207
iusztinpaul/energy-forecasting
๐ ๐ง๐ต๐ฒ ๐๐๐น๐น ๐ฆ๐๐ฎ๐ฐ๐ธ ๐ณ-๐ฆ๐๐ฒ๐ฝ๐ ๐ ๐๐ข๐ฝ๐ ๐๐ฟ๐ฎ๐บ๐ฒ๐๐ผ๐ฟ๐ธ | ๐๐ฒ๐ฎ๐ฟ๐ป ๐ ๐๐ & ๐ ๐๐ข๐ฝ๐ for free by designing, building and deploying an end-to-end ML batch system ~ ๐ด๐ฐ๐ถ๐ณ๐ค๐ฆ ๐ค๐ฐ๐ฅ๐ฆ + 2.5 ๐ฉ๐ฐ๐ถ๐ณ๐ด ๐ฐ๐ง ๐ณ๐ฆ๐ข๐ฅ๐ช๐ฏ๐จ & ๐ท๐ช๐ฅ๐ฆ๐ฐ ๐ฎ๐ข๐ต๐ฆ๐ณ๐ช๐ข๐ญ๐ด
Language: Python - Size: 4.1 MB - Last synced at: 22 days ago - Pushed at: about 1 year ago - Stars: 913 - Forks: 207
damklis/DataEngineeringProject
Example end to end data engineering project.
Language: Python - Size: 1.76 MB - Last synced at: over 1 year ago - Pushed at: over 2 years ago - Stars: 907 - Forks: 191
raystack/optimus
Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.
Language: Go - Size: 12.2 MB - Last synced at: 28 days ago - Pushed at: about 1 year ago - Stars: 748 - Forks: 154
ankurchavda/streamify
A data engineering project with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more!
Language: Python - Size: 93.2 MB - Last synced at: about 18 hours ago - Pushed at: about 3 years ago - Stars: 715 - Forks: 148
airflow-helm/charts
The User-Community Airflow Helm Chart is the standard way to deploy Apache Airflow on Kubernetes with Helm. Originally created in 2017, it has since helped thousands of companies create production-ready deployments of Airflow on Kubernetes.
Language: Shell - Size: 1.1 MB - Last synced at: 2 months ago - Pushed at: 8 months ago - Stars: 689 - Forks: 478
josephmachado/beginner_de_project
Beginner data engineering project - batch edition
Language: HTML - Size: 31.1 MB - Last synced at: 28 days ago - Pushed at: 5 months ago - Stars: 521 - Forks: 165
rjurney/Agile_Data_Code_2
Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition
Language: Jupyter Notebook - Size: 23.2 MB - Last synced at: 8 days ago - Pushed at: about 1 year ago - Stars: 459 - Forks: 310
andreax79/airflow-code-editor
A plugin for Apache Airflow that allows you to edit DAGs in browser
Language: Vue - Size: 13.5 MB - Last synced at: 10 days ago - Pushed at: 10 days ago - Stars: 435 - Forks: 52
fieldryand/goflow
Simple but powerful DAG scheduler and dashboard
Language: Go - Size: 908 KB - Last synced at: 3 months ago - Pushed at: 4 months ago - Stars: 429 - Forks: 34
gocardless/airflow-dbt ๐ฆ
Apache Airflow integration for dbt
Language: Python - Size: 66.4 KB - Last synced at: 19 days ago - Pushed at: about 1 year ago - Stars: 404 - Forks: 62
apache/airflow-client-python
Apache Airflow - OpenApi Client for Python
Language: Python - Size: 1.77 MB - Last synced at: 3 days ago - Pushed at: 7 days ago - Stars: 402 - Forks: 58
hgrif/airflow-tutorial
Airflow basics tutorial
Size: 32.2 KB - Last synced at: over 1 year ago - Pushed at: almost 4 years ago - Stars: 389 - Forks: 82
cubefs/compass
Compass is a task diagnosis platform for bigdata
Language: Java - Size: 5.92 MB - Last synced at: 28 days ago - Pushed at: 7 months ago - Stars: 385 - Forks: 139
villasv/aws-airflow-stack ๐ฆ
Turbine: the bare metals that gets you Airflow
Language: Python - Size: 2.6 MB - Last synced at: 9 days ago - Pushed at: over 3 years ago - Stars: 378 - Forks: 68
astronomer/astro-sdk ๐ฆ
Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.
Language: Python - Size: 7.54 MB - Last synced at: 22 days ago - Pushed at: about 1 month ago - Stars: 369 - Forks: 48
pytorch/torchx
TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.
Language: Python - Size: 34.1 MB - Last synced at: about 5 hours ago - Pushed at: 1 day ago - Stars: 366 - Forks: 132
huseinzol05/Gather-Deployment ๐ฆ
Gathers Python deployment, infrastructure and practices.
Language: Jupyter Notebook - Size: 146 MB - Last synced at: 2 months ago - Pushed at: almost 2 years ago - Stars: 352 - Forks: 99
feluelle/airflow-diagrams
Auto-generated Diagrams from Airflow DAGs. ๐ฎ ๐ช
Language: Python - Size: 1.2 MB - Last synced at: 17 days ago - Pushed at: 17 days ago - Stars: 342 - Forks: 29
teamclairvoyant/airflow-rest-api-plugin
A plugin for Apache Airflow that exposes rest end points for the Command Line Interfaces
Language: Python - Size: 244 KB - Last synced at: 10 days ago - Pushed at: over 4 years ago - Stars: 326 - Forks: 92
airflow-plugins/Example-Airflow-DAGs
Example DAGs using hooks and operators from Airflow Plugins
Language: Python - Size: 54.7 KB - Last synced at: over 1 year ago - Pushed at: almost 7 years ago - Stars: 322 - Forks: 74
GoogleCloudPlatform/airflow-operator ๐ฆ
Kubernetes custom controller and CRDs to managing Airflow
Language: Go - Size: 38.1 MB - Last synced at: 2 months ago - Pushed at: almost 5 years ago - Stars: 299 - Forks: 65
astronomer/airflow-chart
A Helm chart to install Apache Airflow on Kubernetes
Language: Python - Size: 4.52 MB - Last synced at: 21 days ago - Pushed at: 21 days ago - Stars: 283 - Forks: 93
pipeline-tools/gusty
Making DAG construction easier
Language: Python - Size: 470 KB - Last synced at: 22 days ago - Pushed at: about 1 month ago - Stars: 265 - Forks: 12
chandulal/airflow-testing
Airflow Unit Tests and Integration Tests
Language: Python - Size: 465 KB - Last synced at: 7 months ago - Pushed at: over 2 years ago - Stars: 256 - Forks: 45
tkp-archive/paperboy
A web frontend for scheduling Jupyter notebook reports
Language: Python - Size: 12.5 MB - Last synced at: 20 days ago - Pushed at: 7 months ago - Stars: 253 - Forks: 25
epoch8/airflow-exporter
Airflow plugin to export dag and task based metrics to Prometheus.
Language: Python - Size: 359 KB - Last synced at: 18 days ago - Pushed at: 19 days ago - Stars: 252 - Forks: 77
teamclairvoyant/airflow-scheduler-failover-controller
A process that runs in unison with Apache Airflow to control the Scheduler process to ensure High Availability
Language: Python - Size: 116 KB - Last synced at: 10 days ago - Pushed at: almost 3 years ago - Stars: 235 - Forks: 57
astronomer/ask-astro
An end-to-end LLM reference implementation providing a Q&A interface for Airflow and Astronomer
Language: Python - Size: 74.7 MB - Last synced at: 28 days ago - Pushed at: 3 months ago - Stars: 228 - Forks: 51
ris-tlp/audiophile-e2e-pipeline
Pipeline that extracts data from Crinacle's Headphone and InEarMonitor databases and finalizes data for a Metabase Dashboard. The dashboard is then used to support a purchasing decision of which Headphone / IEM to get.
Language: Python - Size: 847 KB - Last synced at: 2 months ago - Pushed at: over 2 years ago - Stars: 228 - Forks: 50
dataplane-app/dataplane
Dataplane is an Airflow inspired unified data platform with additional data mesh and RPA capability to automate, schedule and design data pipelines and workflows. Dataplane is written in Golang with a React front end.
Language: JavaScript - Size: 281 MB - Last synced at: about 2 months ago - Pushed at: 3 months ago - Stars: 226 - Forks: 33
kaxil/airflowctl ๐ฆ
A CLI tool to streamline getting started with Apache Airflowโข and managing multiple Airflow projects
Language: Python - Size: 313 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 219 - Forks: 17
GokuMohandas/data-engineering
Construct a modern data stack and orchestration the workflows to create high quality data for analytics and ML applications.
Language: Jupyter Notebook - Size: 42 KB - Last synced at: 10 days ago - Pushed at: almost 3 years ago - Stars: 216 - Forks: 37
apachecn/airflow-doc-zh
:book: [่ฏ] Airflow ไธญๆๆๆกฃ
Language: CSS - Size: 3.33 MB - Last synced at: 25 days ago - Pushed at: almost 2 years ago - Stars: 213 - Forks: 61
jgoerner/data-science-stack-cookiecutter
๐ณ๐๐คCookiecutter template to launch an awesome dockerized Data Science toolstack (incl. Jupyster, Superset, Postgres, Minio, AirFlow & API Star)
Language: Jupyter Notebook - Size: 795 KB - Last synced at: 3 months ago - Pushed at: almost 2 years ago - Stars: 211 - Forks: 70
coder2j/airflow-docker
Source code of the Apache Airflow Tutorial for Beginners on YouTube Channel Coder2j (https://www.youtube.com/c/coder2j)
Language: Python - Size: 342 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 205 - Forks: 191
andresionek91/airflow-autoscaling-ecs
Airflow Deployment on AWS ECS Fargate Using Cloudformation
Language: HTML - Size: 1.27 MB - Last synced at: 9 days ago - Pushed at: almost 3 years ago - Stars: 205 - Forks: 57
apache/airflow-client-go
Apache Airflow - OpenApi Client for Go
Language: Go - Size: 537 KB - Last synced at: 7 days ago - Pushed at: over 1 year ago - Stars: 203 - Forks: 24
godatadriven/whirl
Fast iterative local development and testing of Apache Airflow workflows
Language: Shell - Size: 146 MB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 200 - Forks: 10
tomasfarias/airflow-dbt-python
A collection of Airflow operators, hooks, and utilities to elevate dbt to a first-class citizen of Airflow.
Language: Python - Size: 7.9 MB - Last synced at: 8 days ago - Pushed at: 8 days ago - Stars: 199 - Forks: 39
mozilla/telemetry-airflow
Airflow configuration for Telemetry
Language: Python - Size: 3.26 MB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 190 - Forks: 95
GoogleCloudPlatform/cortex-data-foundation
Data Foundation - Google Cloud Cortex Framework
Language: Python - Size: 35.4 MB - Last synced at: 28 days ago - Pushed at: about 1 month ago - Stars: 188 - Forks: 104
xnuinside/airflow_in_docker_compose
Apache Airflow in Docker Compose (for both versions 1.10.* and 2.*)
Language: Python - Size: 513 KB - Last synced at: about 1 month ago - Pushed at: over 1 year ago - Stars: 186 - Forks: 78
nareshk1290/Udacity-Data-Engineering
Udacity Data Engineering Nano Degree (DEND)
Language: Jupyter Notebook - Size: 1.73 MB - Last synced at: 2 months ago - Pushed at: over 5 years ago - Stars: 184 - Forks: 168
garystafford/tickit-data-lake-demo
Resources for video demonstrations and blog posts related to DataOps on AWS
Language: Python - Size: 42.1 MB - Last synced at: about 1 month ago - Pushed at: over 3 years ago - Stars: 175 - Forks: 108
dsaidgovsg/airflow-pipeline
An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR
Language: Python - Size: 146 KB - Last synced at: 10 days ago - Pushed at: 23 days ago - Stars: 174 - Forks: 59
Tauffer-Consulting/domino
User friendly and open source platform for workflow creation and monitoring
Language: Python - Size: 97.1 MB - Last synced at: about 2 months ago - Pushed at: 11 months ago - Stars: 173 - Forks: 19
google/starthinker ๐ฆ
Reference framework for building data workflows provided by Google. Accelerates authentication, logging, scheduling, and deployment of solutions using GCP. To borrow a tagline.. "The framework for professionals with deadlines."
Language: Python - Size: 30.7 MB - Last synced at: 11 days ago - Pushed at: about 1 year ago - Stars: 170 - Forks: 51
astronomer/airflow-provider-great-expectations
Great Expectations Airflow operator
Language: Python - Size: 1.84 MB - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 165 - Forks: 57
Toloka/dbt-af
Distributed run of dbt models using Airflow
Language: Python - Size: 3.36 MB - Last synced at: 20 days ago - Pushed at: 21 days ago - Stars: 161 - Forks: 10
GoogleCloudPlatform/public-datasets-pipelines
Cloud-native, data onboarding architecture for Google Cloud Datasets
Language: Python - Size: 6.64 MB - Last synced at: 27 days ago - Pushed at: 4 months ago - Stars: 159 - Forks: 70
nicor88/aws-ecs-airflow
Run Airflow in AWS ECS(Elastic Container Service) using Fargate tasks
Language: HCL - Size: 94.7 KB - Last synced at: about 1 month ago - Pushed at: 8 months ago - Stars: 159 - Forks: 77
jgoerner/beyond-jupyter
๐๐ป๐ All material from the PyCon.DE 2018 Talk "Beyond Jupyter Notebooks - Building your own data science platform with Python & Docker" (incl. Slides, Video, Udemy MOOC & other References)
Size: 6.59 MB - Last synced at: 3 months ago - Pushed at: over 6 years ago - Stars: 156 - Forks: 44
ucbrise/flor
Flow with FlorDB ๐ป
Language: Jupyter Notebook - Size: 61.9 MB - Last synced at: 25 days ago - Pushed at: about 2 months ago - Stars: 155 - Forks: 21
bryzgaloff/airflow-clickhouse-plugin
The most popular ClickHouse plugin for Airflow. ๐ Top-1% downloads on PyPI: https://pypi.org/project/airflow-clickhouse-plugin! Based on mymarilyn/clickhouse-driver.
Language: Python - Size: 236 KB - Last synced at: 27 days ago - Pushed at: about 1 month ago - Stars: 153 - Forks: 41
astronomer/astronomer-providers
Airflow Providers containing Deferrable Operators & Sensors from Astronomer
Language: Python - Size: 2.39 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 149 - Forks: 26
apache/airflow-site
Apache Airflow Website
Language: HTML - Size: 765 MB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 146 - Forks: 358
apache/incubator-liminal ๐ฆ
Apache Liminals goal is to operationalise the machine learning process, allowing data scientists to quickly transition from a successful experiment to an automated pipeline of model training, validation, deployment and inference in production. Liminal provides a Domain Specific Language to build ML workflows on top of Apache Airflow.
Language: Python - Size: 6.37 MB - Last synced at: 4 days ago - Pushed at: 11 months ago - Stars: 143 - Forks: 42
alanchn31/Movalytics-Data-Warehouse
Data pipeline performing ETL to AWS Redshift using Spark, orchestrated with Apache Airflow
Language: Python - Size: 717 KB - Last synced at: 7 months ago - Pushed at: about 5 years ago - Stars: 133 - Forks: 31
iam-mhaseeb/Skytrax-Data-Warehouse ๐ฆ
A full data warehouse infrastructure with ETL pipelines running inside docker on Apache Airflow for data orchestration, AWS Redshift for cloud data warehouse and Metabase to serve the needs of data visualizations such as analytical dashboards.
Language: Python - Size: 1.34 MB - Last synced at: 4 months ago - Pushed at: about 5 years ago - Stars: 133 - Forks: 28
qubole/afctl
afctl helps to manage and deploy Apache Airflow projects faster and smoother.
Language: Python - Size: 76.2 KB - Last synced at: 11 days ago - Pushed at: almost 3 years ago - Stars: 130 - Forks: 7
ninja-van/airflow-boilerplate
A complete development environment setup for working with Airflow
Language: Python - Size: 985 KB - Last synced at: 2 months ago - Pushed at: over 2 years ago - Stars: 128 - Forks: 54
datacamp/viewflow
Viewflow is an Airflow-based framework that allows data scientists to create data models without writing Airflow code.
Language: Python - Size: 893 KB - Last synced at: 20 days ago - Pushed at: almost 4 years ago - Stars: 126 - Forks: 11
star3am/hashiqube
HashiQube - The Ultimate Hands on DevOps Lab running All the HashiCorp Products in a Github Codespace or a Docker Container using Vagrant or Docker Compose
Language: JavaScript - Size: 173 MB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 122 - Forks: 58
quantumblacklabs/kedro-airflow ๐ฆ
Kedro-Airflow makes it easy to deploy Kedro projects to Airflow.
Language: Python - Size: 139 KB - Last synced at: over 1 year ago - Pushed at: about 3 years ago - Stars: 122 - Forks: 16
MBKraus/incremental_training
Repo that relates to the Medium blog 'Keeping your ML model in shape with Kafka, Airflow' andย MLFlow'
Language: Python - Size: 106 MB - Last synced at: over 1 year ago - Pushed at: about 2 years ago - Stars: 118 - Forks: 57
k3ai/k3ai ๐ฆ
A lightweight tool to get an AI Infrastructure Stack up in minutes not days. K3ai will take care of setup K8s for You, deploy the AI tool of your choice and even run your code on it.
Language: Go - Size: 548 KB - Last synced at: over 1 year ago - Pushed at: over 3 years ago - Stars: 118 - Forks: 12
sungchun12/airflow-toolkit
Any Airflow project day 1, you can spin up a local desktop Kubernetes Airflow environment AND one in Google Cloud Composer with tested data pipelines(DAGs) :desktop_computer: >> [ :rocket:, :ship: ]
Language: HCL - Size: 12.3 MB - Last synced at: 3 months ago - Pushed at: over 1 year ago - Stars: 111 - Forks: 31
astronomer/ap-airflow ๐ฆ
Astronomer Core Docker Images
Language: Jinja - Size: 33 MB - Last synced at: about 1 month ago - Pushed at: about 1 year ago - Stars: 107 - Forks: 30
trannhatnguyen2/NYC_Taxi_Data_Pipeline
Nyc_Taxi_Data_Pipeline - DE Project
Language: Python - Size: 6.58 MB - Last synced at: about 1 month ago - Pushed at: 8 months ago - Stars: 106 - Forks: 21
scribd/objinsync
Continuously synchronize directories from remote object store to local filesystem
Language: Go - Size: 123 KB - Last synced at: 26 days ago - Pushed at: 4 months ago - Stars: 104 - Forks: 21
caoergou/airflow-extended-api-plugin
Yet another Airflow plugin using CLI command as RESTful api, supports Airflow v2.X.
Language: Python - Size: 447 KB - Last synced at: 10 months ago - Pushed at: almost 2 years ago - Stars: 103 - Forks: 31
ananthdurai/airflow-training
Airflow training for the crunch conf
Language: Python - Size: 5.24 MB - Last synced at: over 1 year ago - Pushed at: over 6 years ago - Stars: 103 - Forks: 55
feluelle/airflint
Enforce Best Practices for all your Airflow DAGs. โญ
Language: Python - Size: 331 KB - Last synced at: 17 days ago - Pushed at: 17 days ago - Stars: 101 - Forks: 2
blockchain-etl/polygon-etl
ETL (extract, transform and load) tools for ingesting Polygon blockchain data to Google BigQuery and Pub/Sub
Language: Python - Size: 1.46 MB - Last synced at: 26 days ago - Pushed at: 9 months ago - Stars: 100 - Forks: 76
mikeghen/airflow-tutorial
Use Airflow to move data from multiple MySQL databases to BigQuery
Language: PLpgSQL - Size: 879 KB - Last synced at: 2 months ago - Pushed at: almost 5 years ago - Stars: 100 - Forks: 21
jmeisele/ml-ops
Get your MLOps (Level 1) platform started and going fast.
Language: Python - Size: 332 MB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 96 - Forks: 21
mastak/airflow_multi_dagrun
triggering a DAG run multiple times
Language: Python - Size: 65.4 KB - Last synced at: 7 days ago - Pushed at: over 1 year ago - Stars: 88 - Forks: 27
shravan-kuchkula/udacity-data-eng-proj-1
Developed a data pipeline to automate data warehouse ETL by building custom airflow operators that handle the extraction, transformation, validation and loading of data from S3 -> Redshift -> S3
Language: Python - Size: 3.47 MB - Last synced at: over 1 year ago - Pushed at: over 3 years ago - Stars: 88 - Forks: 58
konosp/dbt-airflow-docker-compose
Execution of DBT models using Apache Airflow through Docker Compose
Language: Python - Size: 192 MB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 87 - Forks: 41
marcosmarxm/airflow-testing-ci-workflow
(project & tutorial) dag pipeline tests + ci/cd setup
Language: Python - Size: 496 KB - Last synced at: 3 months ago - Pushed at: over 4 years ago - Stars: 86 - Forks: 10
mendhak/Airflow-MS-Teams-Operator
Airflow operator that can send messages to MS Teams
Language: Python - Size: 357 KB - Last synced at: 2 months ago - Pushed at: 10 months ago - Stars: 84 - Forks: 25
PowerDataHub/terraform-aws-airflow
Terraform module to deploy an Apache Airflow cluster on AWS, backed by RDS PostgreSQL for metadata, S3 for logs and SQS as message broker with CeleryExecutor
Language: HCL - Size: 702 KB - Last synced at: 4 days ago - Pushed at: over 2 years ago - Stars: 84 - Forks: 40
idealista/airflow-role
Ansible role to install Apache Airflow
Language: YAML - Size: 302 KB - Last synced at: 26 days ago - Pushed at: 4 months ago - Stars: 82 - Forks: 52
