apache/airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Language: Python - Size: 382 MB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 40,370 - Forks: 15,104

jghoman/awesome-apache-airflow

Curated list of resources about Apache Airflow

Language: Shell - Size: 550 KB - Last synced at: about 1 month ago - Pushed at: 10 months ago - Stars: 3,775 - Forks: 496

elyra-ai/elyra

Elyra extends JupyterLab with an AI centric approach.

Language: Python - Size: 115 MB - Last synced at: 7 days ago - Pushed at: 15 days ago - Stars: 1,926 - Forks: 353

teamclairvoyant/airflow-maintenance-dags

A series of DAGs/Workflows to help maintain the operation of Airflow

Language: Python - Size: 684 KB - Last synced at: 17 days ago - Pushed at: 12 months ago - Stars: 1,724 - Forks: 397

san089/goodreads_etl_pipeline

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

Language: Python - Size: 1.31 MB - Last synced at: 15 days ago - Pushed at: about 5 years ago - Stars: 1,378 - Forks: 227

astronomer/dag-factory

Construct Apache Airflow DAGs Declaratively via YAML configuration files

Language: Python - Size: 8.72 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 1,303 - Forks: 195

astronomer/astronomer-cosmos

Run your dbt Core projects as Apache Airflow DAGs and Task Groups with a few lines of code

Language: Python - Size: 18.2 MB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 955 - Forks: 214

tuanavu/airflow-tutorial

Apache Airflow tutorial

Language: Jupyter Notebook - Size: 153 KB - Last synced at: about 2 months ago - Pushed at: over 2 years ago - Stars: 946 - Forks: 1,012

couler-proj/couler

Unified Interface for Constructing and Managing Workflows on different workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow.

Language: Python - Size: 3.16 MB - Last synced at: 2 months ago - Pushed at: 8 months ago - Stars: 933 - Forks: 86

astronomer/astronomer

Helm Charts for the Astronomer Platform, Apache Airflow as a Service on Kubernetes

Language: Python - Size: 11.4 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 473 - Forks: 93

andreax79/airflow-code-editor

A plugin for Apache Airflow that allows you to edit DAGs in browser

Language: Vue - Size: 13.4 MB - Last synced at: 10 days ago - Pushed at: 11 days ago - Stars: 431 - Forks: 52

blockchain-etl/ethereum-etl-airflow

Airflow DAGs for exporting, loading, and parsing the Ethereum blockchain data. How to get any Ethereum smart contract into BigQuery https://towardsdatascience.com/how-to-get-any-ethereum-smart-contract-into-bigquery-in-8-mins-bab5db1fdeee

Language: Python - Size: 3.15 MB - Last synced at: 4 days ago - Pushed at: 3 months ago - Stars: 416 - Forks: 199

apache/airflow-client-python

Apache Airflow - OpenApi Client for Python

Language: Python - Size: 1.75 MB - Last synced at: 6 days ago - Pushed at: 12 days ago - Stars: 401 - Forks: 57

astronomer/astro-cli

CLI that makes it easy to create, test and deploy Airflow DAGs to Astronomer

Language: Go - Size: 16.6 MB - Last synced at: 16 days ago - Pushed at: 16 days ago - Stars: 386 - Forks: 81

astronomer/astro-sdk 📦

Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.

Language: Python - Size: 7.54 MB - Last synced at: 10 days ago - Pushed at: 19 days ago - Stars: 369 - Forks: 48

teamclairvoyant/airflow-rest-api-plugin

A plugin for Apache Airflow that exposes rest end points for the Command Line Interfaces

Language: Python - Size: 244 KB - Last synced at: 7 months ago - Pushed at: over 4 years ago - Stars: 325 - Forks: 93

airflow-plugins/Example-Airflow-DAGs

Example DAGs using hooks and operators from Airflow Plugins

Language: Python - Size: 54.7 KB - Last synced at: over 1 year ago - Pushed at: almost 7 years ago - Stars: 322 - Forks: 74

GoogleCloudPlatform/airflow-operator 📦

Kubernetes custom controller and CRDs to managing Airflow

Language: Go - Size: 38.1 MB - Last synced at: about 2 months ago - Pushed at: almost 5 years ago - Stars: 299 - Forks: 65

astronomer/airflow-chart

A Helm chart to install Apache Airflow on Kubernetes

Language: Python - Size: 4.52 MB - Last synced at: 9 days ago - Pushed at: 9 days ago - Stars: 283 - Forks: 93

tkp-archive/paperboy

A web frontend for scheduling Jupyter notebook reports

Language: Python - Size: 12.5 MB - Last synced at: 8 days ago - Pushed at: 6 months ago - Stars: 253 - Forks: 25

airscholar/e2e-data-engineering

An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All components are containerized with Docker for easy deployment and scalability.

Language: Python - Size: 289 KB - Last synced at: 16 days ago - Pushed at: 4 months ago - Stars: 250 - Forks: 123

teamclairvoyant/airflow-scheduler-failover-controller

A process that runs in unison with Apache Airflow to control the Scheduler process to ensure High Availability

Language: Python - Size: 116 KB - Last synced at: over 1 year ago - Pushed at: almost 3 years ago - Stars: 230 - Forks: 63

kaxil/airflowctl 📦

A CLI tool to streamline getting started with Apache Airflow™ and managing multiple Airflow projects

Language: Python - Size: 313 KB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 219 - Forks: 17

apache/airflow-client-go

Apache Airflow - OpenApi Client for Go

Language: Go - Size: 537 KB - Last synced at: 6 days ago - Pushed at: over 1 year ago - Stars: 200 - Forks: 24

xnuinside/airflow_in_docker_compose

Apache Airflow in Docker Compose (for both versions 1.10.* and 2.*)

Language: Python - Size: 513 KB - Last synced at: 19 days ago - Pushed at: over 1 year ago - Stars: 186 - Forks: 78

rolanddb/airflow-on-kubernetes

A guide to running Airflow on Kubernetes

Size: 5.86 KB - Last synced at: 2 months ago - Pushed at: almost 6 years ago - Stars: 172 - Forks: 16

Toloka/dbt-af

Distributed run of dbt models using Airflow

Language: Python - Size: 3.36 MB - Last synced at: 8 days ago - Pushed at: 8 days ago - Stars: 161 - Forks: 10

astronomer/astronomer-providers

Airflow Providers containing Deferrable Operators & Sensors from Astronomer

Language: Python - Size: 2.39 MB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 148 - Forks: 26

airscholar/RedditDataEngineering

This project provides a comprehensive data pipeline solution to extract, transform, and load (ETL) Reddit data into a Redshift data warehouse. The pipeline leverages a combination of tools and services including Apache Airflow, Celery, PostgreSQL, Amazon S3, AWS Glue, Amazon Athena, and Amazon Redshift.

Language: Python - Size: 118 KB - Last synced at: about 2 months ago - Pushed at: over 1 year ago - Stars: 128 - Forks: 65

ninja-van/airflow-boilerplate

A complete development environment setup for working with Airflow

Language: Python - Size: 985 KB - Last synced at: 2 months ago - Pushed at: over 2 years ago - Stars: 128 - Forks: 54

datacamp/viewflow

Viewflow is an Airflow-based framework that allows data scientists to create data models without writing Airflow code.

Language: Python - Size: 893 KB - Last synced at: 8 days ago - Pushed at: almost 4 years ago - Stars: 126 - Forks: 11

Wittline/uber-expenses-tracking

The goal of this project is to track the expenses of Uber Rides and Uber Eats through data Engineering processes using technologies such as Apache Airflow, AWS Redshift and Power BI.

Language: Jupyter Notebook - Size: 28.2 MB - Last synced at: about 2 months ago - Pushed at: almost 3 years ago - Stars: 121 - Forks: 36

gestaogovbr/Ro-dou

Gerador de DAGs no Apache Airflow para fazer clipping do Diário Oficial da União.

Language: Python - Size: 3.96 MB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 108 - Forks: 31

caoergou/airflow-extended-api-plugin

Yet another Airflow plugin using CLI command as RESTful api, supports Airflow v2.X.

Language: Python - Size: 447 KB - Last synced at: 10 months ago - Pushed at: almost 2 years ago - Stars: 103 - Forks: 31

PowerDataHub/terraform-aws-airflow

Terraform module to deploy an Apache Airflow cluster on AWS, backed by RDS PostgreSQL for metadata, S3 for logs and SQS as message broker with CeleryExecutor

Language: HCL - Size: 702 KB - Last synced at: 2 months ago - Pushed at: over 2 years ago - Stars: 84 - Forks: 40

idealista/airflow-role

Ansible role to install Apache Airflow

Language: YAML - Size: 302 KB - Last synced at: 14 days ago - Pushed at: 3 months ago - Stars: 82 - Forks: 52

WordPress/openverse-catalog 📦

Identifies and collects data on cc-licensed content across web crawl data and public apis.

Language: Python - Size: 92.6 MB - Last synced at: 5 months ago - Pushed at: almost 2 years ago - Stars: 59 - Forks: 54

airflow-plugins/airflow_api_plugin

REST-like API exposing Airflow data and operations

Language: Python - Size: 15.6 KB - Last synced at: over 2 years ago - Pushed at: over 6 years ago - Stars: 57 - Forks: 20

IBM/ada

ADA is a microservice created to retrieve analytics metrics from an Airflow database instance.

Language: Python - Size: 20.1 MB - Last synced at: about 1 month ago - Pushed at: about 1 year ago - Stars: 55 - Forks: 9

KentHsu/Udacity-Data-Engineering-Nanodgree

Udacity Data Engineering Nanodegree Program

Language: Jupyter Notebook - Size: 2.12 MB - Last synced at: about 2 months ago - Pushed at: over 4 years ago - Stars: 52 - Forks: 59

behnamyazdan/ecommerce_realtime_data_pipeline

Ecommerce Realtime Data Pipeline (Data Modeling, Workflow Orchestration, Change Data Capture, Analytical Database and Dashboarding)

Language: Python - Size: 2.75 MB - Last synced at: 4 months ago - Pushed at: about 1 year ago - Stars: 51 - Forks: 10

aws-ia/terraform-aws-mwaa

Terraform module for Amazon MWAA(Apache Airflow)

Language: HCL - Size: 3.21 MB - Last synced at: 9 days ago - Pushed at: about 2 months ago - Stars: 48 - Forks: 60

rafaelpierre/pyjaws

PyJaws: A Pythonic Way to Define Databricks Jobs and Workflows

Language: Python - Size: 3.46 MB - Last synced at: 16 days ago - Pushed at: 11 months ago - Stars: 43 - Forks: 3

DanilBaibak/ml-in-production

The practical use-cases of how to make your Machine Learning Pipelines robust and reliable using Apache Airflow.

Language: Python - Size: 143 KB - Last synced at: almost 2 years ago - Pushed at: over 2 years ago - Stars: 43 - Forks: 19

airscholar/SparkingFlow

This project demonstrates how to use Apache Airflow to submit jobs to Apache spark cluster in different programming laguages using Python, Scala and Java as an example.

Language: Java - Size: 94.7 KB - Last synced at: about 2 months ago - Pushed at: about 1 year ago - Stars: 42 - Forks: 27

kadnan/airflow-scraping

Using Apache Airflow to schedule web scrapers

Language: Python - Size: 28.3 KB - Last synced at: 24 days ago - Pushed at: over 6 years ago - Stars: 42 - Forks: 11

airflow-plugins/google_analytics_plugin

Language: Python - Size: 19.5 KB - Last synced at: over 2 years ago - Pushed at: almost 4 years ago - Stars: 41 - Forks: 37

dmlogv/airflow-tutorial

Learning resources for Airflow Tutorial article.

Language: Python - Size: 31.3 KB - Last synced at: over 2 years ago - Pushed at: almost 5 years ago - Stars: 41 - Forks: 14

Mr-xn/CVE-2022-40127

Apache Airflow < 2.4.0 DAG example_bash_operator RCE POC

Size: 7.81 KB - Last synced at: 2 months ago - Pushed at: over 2 years ago - Stars: 40 - Forks: 6

garystafford/aws-airflow-demo

Project files for the post: Running PySpark Applications on Amazon EMR using Apache Airflow: Using the new Amazon Managed Workflows for Apache Airflow (MWAA) on AWS.

Language: Python - Size: 753 KB - Last synced at: almost 2 years ago - Pushed at: almost 3 years ago - Stars: 39 - Forks: 14

michaelosthege/fairflow

Functional Airflow DAG definitions.

Language: Python - Size: 39.1 KB - Last synced at: 22 days ago - Pushed at: almost 8 years ago - Stars: 38 - Forks: 2

kadnan/Airflow-Tutorial

Basic tutorial of using Apache Airflow

Language: Python - Size: 19.5 KB - Last synced at: 24 days ago - Pushed at: over 6 years ago - Stars: 36 - Forks: 13

AdrianRiesco/Masters-Thesis-on-Big-Data

Master's thesis on Big Data

Language: CSS - Size: 266 MB - Last synced at: about 2 months ago - Pushed at: almost 3 years ago - Stars: 34 - Forks: 7

datarobot/airflow-provider-datarobot

DataRobot provider for Apache Airflow

Language: Python - Size: 688 KB - Last synced at: 4 days ago - Pushed at: 5 days ago - Stars: 31 - Forks: 5

vincentnam/docker_datalake

Datalake

Language: JavaScript - Size: 49.3 MB - Last synced at: 24 days ago - Pushed at: 24 days ago - Stars: 31 - Forks: 11

elyra-ai/pipeline-editor

Common pipeline-editor components used in different clients (e.g. Elyra application, Web browser extensions, etc)

Language: TypeScript - Size: 2.11 MB - Last synced at: 6 days ago - Pushed at: 4 months ago - Stars: 31 - Forks: 22

elasticlabs/airflow-jupyter-docker-compose

Orchestration of data science and earth observation models in Apache Airflow, scale-up with Celery Executor, experiment with jupyter notebook using a docker containers composition

Language: Python - Size: 136 KB - Last synced at: about 2 years ago - Pushed at: almost 3 years ago - Stars: 31 - Forks: 8

blockchain-etl/bitcoin-etl-airflow

Airflow DAGs for https://github.com/blockchain-etl/bitcoin-etl

Language: Python - Size: 99.6 KB - Last synced at: 4 days ago - Pushed at: about 3 years ago - Stars: 31 - Forks: 14

airflow-plugins/salesforce_plugin

Move Data From Salesforce -> S3 -> Redshift

Language: Python - Size: 19.5 KB - Last synced at: over 2 years ago - Pushed at: almost 4 years ago - Stars: 31 - Forks: 24

PBWebMedia/airflow-prometheus-exporter

Export Airflow metrics (from mysql) in prometheus format

Language: Go - Size: 53.7 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 29 - Forks: 10

sergio11/lyric_wave_architecture

🎵 LyricWave - Your AI Music Composer 🎶 Compose Unique MP4 Songs Effortlessly! LyricWave uses AI to create personalized music by harmonizing lyrics with captivating melodies and synthetic vocals. Unleash your musical creativity today! 🚀🎶

Language: Python - Size: 29 MB - Last synced at: 2 months ago - Pushed at: 3 months ago - Stars: 29 - Forks: 6

guidok91/spark-movies-etl

Spark data pipeline that processes movie ratings data.

Language: Python - Size: 3.79 MB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 28 - Forks: 12

doitintl/doit-composer-airflow-training 📦

Getting started with Apache Airflow on Cloud Composer

Language: Python - Size: 4.24 MB - Last synced at: 2 days ago - Pushed at: about 3 years ago - Stars: 28 - Forks: 4

airflow-plugins/pandora-plugin

Plugin offering views, operators, sensors, and more developed at Pandora Media.

Language: Python - Size: 34.2 KB - Last synced at: over 2 years ago - Pushed at: about 7 years ago - Stars: 25 - Forks: 6

unruly/terraform-aws-airflow

Terraform module for a PostgreSQL-backed Apache Airflow instance

Language: HCL - Size: 200 KB - Last synced at: 2 months ago - Pushed at: about 7 years ago - Stars: 24 - Forks: 7

astronomer/astro-provider-databricks 📦

Orchestrate your Databricks notebooks in Airflow and execute them as Databricks Workflows

Language: Python - Size: 11.1 MB - Last synced at: 22 days ago - Pushed at: 10 months ago - Stars: 23 - Forks: 12

moritzkoerber/covid-19-data-engineering-pipeline

A Covid-19 data pipeline on AWS featuring PySpark/Glue, Docker, Great Expectations, Airflow, and Redshift, templated in CloudFormation and CDK, deployable via Github Actions.

Language: Python - Size: 1.31 MB - Last synced at: about 1 month ago - Pushed at: over 1 year ago - Stars: 23 - Forks: 5

airscholar/FootballDataEngineering

An end-to-end data engineering pipeline that fetches data from Wikipedia, cleans and transforms it with Apache Airflow and saves it on Azure Data Lake. Other processing takes place on Azure Data Factory, Azure Synapse and Tableau.

Language: Python - Size: 469 KB - Last synced at: about 2 months ago - Pushed at: over 1 year ago - Stars: 23 - Forks: 19

zkan/dtc-data-engineering-zoomcamp-project

DataTalks.Club's Data Engineering Zoomcamp Project

Language: Python - Size: 1.62 MB - Last synced at: 2 months ago - Pushed at: almost 3 years ago - Stars: 23 - Forks: 6

astronomer/airflow-provider-fivetran-async

A new Airflow Provider for Fivetran, maintained by Astronomer and Fivetran

Language: Python - Size: 206 KB - Last synced at: 4 days ago - Pushed at: 5 days ago - Stars: 22 - Forks: 9

zkan/data-pipelines-with-airflow

Skooldio: Data Pipelines with Airflow

Language: Python - Size: 1.92 MB - Last synced at: 15 days ago - Pushed at: 15 days ago - Stars: 22 - Forks: 15

airflow-plugins/redshift_plugin

Language: Python - Size: 31.3 KB - Last synced at: over 2 years ago - Pushed at: almost 5 years ago - Stars: 22 - Forks: 25

airflow-plugins/mongo_plugin

Language: Python - Size: 10.7 KB - Last synced at: over 2 years ago - Pushed at: about 6 years ago - Stars: 22 - Forks: 20

dain55788/ELT-Data-Pipeline

ELT Data Pipeline implementation in Data Warehousing environment

Language: Jupyter Notebook - Size: 1.62 MB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 20 - Forks: 8

astronomer/astro-provider-ray

This provider contains operators, decorators and triggers to send a ray job from an airflow task

Language: Python - Size: 627 KB - Last synced at: 1 day ago - Pushed at: 2 months ago - Stars: 18 - Forks: 4

akarce/e2e-structured-streaming

End-to-end data pipeline that ingests, processes, and stores data. It uses Apache Airflow to schedule scripts that fetch data from an API, sends the data to Kafka, and processes it with Spark before writing to Cassandra. The pipeline, built with Python and Apache Zookeeper, is containerized with Docker for easy deployment and scalability.

Language: Python - Size: 49.4 MB - Last synced at: about 2 months ago - Pushed at: 11 months ago - Stars: 18 - Forks: 8

airscholar/Kubernetes-For-DataEngineering

This repository contains the necessary configuration files and DAGs (Directed Acyclic Graphs) for setting up a robust data engineering environment using Kubernetes and Apache Airflow

Language: Python - Size: 7.81 KB - Last synced at: about 2 months ago - Pushed at: over 1 year ago - Stars: 18 - Forks: 14

comitivaesperanca/data

🎲 Repositório para armazenar todos os componentes referentes a Data Science / Data Engineering do projeto

Language: Jupyter Notebook - Size: 28.5 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 18 - Forks: 0

AlvaroCavalcante/airflow-parse-bench

Stop creating bad DAGs! Use this tool to measure and compare the parse time of your DAGs, identify bottlenecks, and optimize your Airflow environment for better performance.

Language: Python - Size: 192 KB - Last synced at: 27 days ago - Pushed at: 4 months ago - Stars: 17 - Forks: 0

adilkhash/apache-airflow-intro

Language: Python - Size: 9.77 KB - Last synced at: about 1 month ago - Pushed at: about 2 years ago - Stars: 17 - Forks: 4

ipeluffo/airflow-on-kubernetes

Source code for guide to run Apache Airflow on Kubernetes

Language: Python - Size: 7.81 KB - Last synced at: about 6 hours ago - Pushed at: about 5 years ago - Stars: 17 - Forks: 13

airflow-plugins/facebook_ads_plugin

Language: Python - Size: 13.7 KB - Last synced at: over 2 years ago - Pushed at: over 6 years ago - Stars: 17 - Forks: 9

DataEngineering-LATAM/Airflow-StudyClub

Grupo de estudio Apache Airflow organizado por la comunidad Data Engineering Latam

Language: Python - Size: 323 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 16 - Forks: 10

andreax79/airflow-gitlab-webhook

Commit on Gitlab and run an Apache Airflow DAG

Language: Shell - Size: 22.5 KB - Last synced at: 24 days ago - Pushed at: almost 5 years ago - Stars: 16 - Forks: 5

airflow-plugins/mysql_plugin

Language: Python - Size: 17.6 KB - Last synced at: over 2 years ago - Pushed at: about 6 years ago - Stars: 16 - Forks: 25

blockchain-etl/bitcoin-etl-airflow-neo4j

Airflow DAGs for ingesting Bitcoin blockchain data to Neo4j

Language: Python - Size: 438 KB - Last synced at: 4 days ago - Pushed at: over 2 years ago - Stars: 15 - Forks: 6

airflow-plugins/github_plugin

Language: Python - Size: 21.5 KB - Last synced at: over 2 years ago - Pushed at: about 6 years ago - Stars: 15 - Forks: 7

teamclairvoyant/airflow-user-management-plugin

A plugin for Apache Airflow that allows you to manage the users that can login

Language: Python - Size: 20.5 KB - Last synced at: over 2 years ago - Pushed at: over 5 years ago - Stars: 14 - Forks: 3

abhishekbhakat/airflow-mcp-server

MCP Server for Apache Airflow

Language: Python - Size: 4.47 MB - Last synced at: 9 days ago - Pushed at: 9 days ago - Stars: 13 - Forks: 4

Danysan1/osm-wikidata-map-framework

Merge data from OpenStreetMap and Wikidata to map information about entities and points of interest (mirror of https://gitlab.com/openetymologymap/osm-wikidata-map-framework )

Language: TypeScript - Size: 12.2 MB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 13 - Forks: 0

kishlayjeet/Twitter-Data-Pipeline-using-Airflow-and-AWS-S3

An end-to-end Twitter Data Pipeline that extracts data from Twitter and loads it into AWS S3.

Language: Python - Size: 20.5 KB - Last synced at: 2 months ago - Pushed at: almost 2 years ago - Stars: 13 - Forks: 6

archie-cm/IBM-Data-Engineering-Capstone-Project

Business challenge that requires building a data platform for retailer data analytics.

Language: Jupyter Notebook - Size: 5.76 MB - Last synced at: about 2 months ago - Pushed at: over 2 years ago - Stars: 12 - Forks: 7

badal-io/gcp-airflow-foundations

Opinionated framework based on Airflow 2.0 for building pipelines to ingest data into a BigQuery data warehouse

Language: Python - Size: 1.3 MB - Last synced at: 7 days ago - Pushed at: over 2 years ago - Stars: 12 - Forks: 2

lsjsj92/airflow_tutorial

python airflow tutorial and example

Language: Python - Size: 11.7 KB - Last synced at: 2 months ago - Pushed at: about 3 years ago - Stars: 12 - Forks: 6

dushyantkhosla/airflow4ds

Using Apache Airflow to author, run and monitor complex data pipelines.

Language: Jupyter Notebook - Size: 22.5 MB - Last synced at: over 2 years ago - Pushed at: over 6 years ago - Stars: 12 - Forks: 2

airflow-laminar/airflow-config

A Configuration System for Airflow

Language: Python - Size: 1.2 MB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 11 - Forks: 2

airflow-laminar/airflow-supervisor

Airflow utilities for running long-running or always-on jobs with supervisord

Language: Python - Size: 1.32 MB - Last synced at: 10 days ago - Pushed at: 29 days ago - Stars: 11 - Forks: 2

aymane-maghouti/Big-Data-Project

This project aims to predict smartphone prices using a combination of batch and stream processing techniques in a Big Data environment. The architecture follows the Lambda Architecture pattern, providing both real-time and batch processing capabilities to users.

Language: Python - Size: 960 KB - Last synced at: about 2 months ago - Pushed at: about 1 year ago - Stars: 11 - Forks: 2

astronomer/apache-airflow-providers-transfers 📦

Language: Python - Size: 979 KB - Last synced at: 4 months ago - Pushed at: almost 2 years ago - Stars: 11 - Forks: 3

sarit-si/docker-airflow-pdi-01

Setup Airflow & Pentaho (with Carte) in separate Docker containers

Language: Python - Size: 46.9 KB - Last synced at: almost 2 years ago - Pushed at: about 4 years ago - Stars: 11 - Forks: 2