gestaogovbr/FastETL

Airflow plugins for implementing data pipelines. | Plugins do Airflow para implementação de pipelines de dados.

Language: Python - Size: 4.94 MB - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 46 - Forks: 8

airflow-laminar/airflow-common-operators

Common Airflow Operators / Tasks

Language: Python - Size: 485 KB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 4 - Forks: 0

astronomer/airflow-provider-great-expectations

Great Expectations Airflow operator

Language: Python - Size: 1.94 MB - Last synced at: 9 days ago - Pushed at: 9 days ago - Stars: 168 - Forks: 58

astronomer/astronomer-cosmos

Run your dbt Core projects as Apache Airflow DAGs and Task Groups with a few lines of code

Language: Python - Size: 18.2 MB - Last synced at: 9 days ago - Pushed at: 9 days ago - Stars: 993 - Forks: 226

kuldeep27396/airflow-projects-deployed

This project contains a comprehensive collection of Apache Airflow DAGs designed for learning Airflow concepts from basics to advanced levels. The project includes 25 different DAGs covering various operators, patterns, and production scenarios, all deployed and tested using Astronomer Cloud.

Language: Python - Size: 4.17 MB - Last synced at: 12 days ago - Pushed at: 12 days ago - Stars: 1 - Forks: 0

tomasfarias/airflow-dbt-python

A collection of Airflow operators, hooks, and utilities to elevate dbt to a first-class citizen of Airflow.

Language: Python - Size: 8.28 MB - Last synced at: 7 days ago - Pushed at: about 1 month ago - Stars: 202 - Forks: 38

astronomer/astro-provider-anyscale

An Apache Airflow provider for seamless integration with Anyscale.

Language: Python - Size: 219 KB - Last synced at: 20 days ago - Pushed at: 21 days ago - Stars: 5 - Forks: 4

astronomer/astronomer-providers

Airflow Providers containing Deferrable Operators & Sensors from Astronomer

Language: Python - Size: 2.39 MB - Last synced at: 20 days ago - Pushed at: 20 days ago - Stars: 149 - Forks: 26

lynker-analytics/airflow-conda-operator

Airflow task & operator running python code in conda/mamba environments

Language: Python - Size: 36.1 KB - Last synced at: 22 days ago - Pushed at: 22 days ago - Stars: 2 - Forks: 2

tensor365/AIRFLOW-QLIKSENSE-CLIENT-MANAGED

Trigger Qlik Sense Task From an airflow operator

Language: Python - Size: 47.9 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 0 - Forks: 0

sergio11/lyric_wave_architecture

🎵 LyricWave – AI Music Composer (Proof of Concept) 🎶 A personal project exploring automatic generation of unique MP4 songs. LyricWave blends lyrics with AI-generated melodies and synthetic vocals to experiment with new forms of musical expression. A creative testbed to push your ideas into sound. 🚀🎧

Language: Python - Size: 29 MB - Last synced at: about 2 months ago - Pushed at: 5 months ago - Stars: 34 - Forks: 6

san089/Udacity-Data-Engineering-Projects

Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.

Language: Python - Size: 2.03 MB - Last synced at: about 2 months ago - Pushed at: almost 3 years ago - Stars: 1,620 - Forks: 527

tensor365/AIRFLOW-QLIKSENSE-CLOUD

Qlik Sense Cloud Provider for Apache Airflow

Language: Python - Size: 191 KB - Last synced at: 2 months ago - Pushed at: 2 months ago - Stars: 4 - Forks: 2

sangwanamit621/learning-and-experiments

Welcome to the Learning and Experiments Hub—a dynamic repository capturing my journey of exploration and experimentation in the vast world of technology. This space serves as a digital canvas where I document my learning process, experiments, and discoveries.

Language: Jupyter Notebook - Size: 53.7 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0

wherobots/airflow-providers-wherobots

Airflow extensions for communicating with Wherobots Cloud

Language: Python - Size: 998 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 2

WilsonH918/Data_Pipeline_ETL_Pipeline_Web3_Token

Data Pipeline Project

Language: Python - Size: 17.6 KB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 1 - Forks: 0

sdaberdaku/apache-airflow-providers-pysparkonk8s

Python package for the pysparkonk8s Apache Airflow provider.

Language: Python - Size: 101 KB - Last synced at: 30 days ago - Pushed at: 5 months ago - Stars: 2 - Forks: 1

AlvaroCavalcante/airflow-custom-deferrable-dataflow-operator

Start your Dataflow jobs execution directly from the Triggerer without going to the Worker!

Language: Python - Size: 36.1 KB - Last synced at: 22 days ago - Pushed at: 5 months ago - Stars: 0 - Forks: 0

mikeroyal/Apache-Airflow-Guide

Apache Airflow Guide

Language: Python - Size: 279 KB - Last synced at: 11 days ago - Pushed at: about 1 year ago - Stars: 28 - Forks: 12

RADar-AZDelta/lib_airflow

Custom Airflow operators and hooks, that help to upload our raw hospital data to Google Cloud.

Language: Python - Size: 188 KB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 0 - Forks: 0

An4PDM/Aprendendo-a-criar-DAGs

Este repositório tem objetivo de organizar todas as DAGs que estou criando para fim de consultas posteriores.

Language: Python - Size: 10.7 KB - Last synced at: 5 months ago - Pushed at: 7 months ago - Stars: 1 - Forks: 0

sp1thas/airflow-provider-kaggle

[MIRROR OF https://codeberg.org/sp1thas/airflow-provider-kaggle] Airflow operators and hooks for interacting with the Kaggle API

Language: Python - Size: 47.9 KB - Last synced at: 1 day ago - Pushed at: 7 months ago - Stars: 1 - Forks: 0

rnaguilar/airflow_custom_operator

Contribuições de custom operators no airflow

Language: Python - Size: 0 Bytes - Last synced at: 8 months ago - Pushed at: 8 months ago - Stars: 0 - Forks: 0

hightouchio/airflow-provider-hightouch

Airflow operators, hooks, and sensors for interacting with the Hightouch API

Language: Python - Size: 59.6 KB - Last synced at: 21 days ago - Pushed at: 12 months ago - Stars: 17 - Forks: 13

HakeemSalaudeen/CoreSentiments-project

This project is designed to reinforce my understanding of Data Pipeline Orchestration with Apache Airflow. The focus is on implementing a data pipeline that addresses data ingestion, processing, storage, and analysis. The project challenges me to apply my knowledge of Apache Airflow to solve a practical scenario based problem

Language: Python - Size: 7.81 KB - Last synced at: 8 months ago - Pushed at: 8 months ago - Stars: 0 - Forks: 0

emsalcengiz/Etl_processing

I find Apache Airflow very useful for ETL work. Here I transferred data from the source database(mysql) to the target database(postresql) and used the Airflow Bash Operator.

Language: Python - Size: 9.77 KB - Last synced at: 11 months ago - Pushed at: over 4 years ago - Stars: 1 - Forks: 1

Amsterdam/dataservices-airflow 📦

Airflow docker container for Dataservices.

Language: Python - Size: 5.36 MB - Last synced at: 4 months ago - Pushed at: over 1 year ago - Stars: 2 - Forks: 4

polarbeargo/Data-Pipelines-with-Airflow

Language: Python - Size: 1020 KB - Last synced at: over 1 year ago - Pushed at: almost 4 years ago - Stars: 0 - Forks: 0

BaliDataMan/airflow-Tut

Airflow Tutorial to learn practical implementation with basics Concepts, Architecture, Working & Configuration practises.

Language: Shell - Size: 425 KB - Last synced at: over 1 year ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0

hbasria/airflow-provider-baserow

Language: Python - Size: 14.6 KB - Last synced at: over 1 year ago - Pushed at: almost 2 years ago - Stars: 1 - Forks: 0

shiv-rna/Airflow-Basics

Welcome to my Apache Airflow learning journey repository! 🚀 This repository serves as a comprehensive documentation of my exploration and understanding of Apache Airflow, an open-source platform for orchestrating complex workflows.

Language: Python - Size: 65.8 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

DataDrivenGit/Music-Streaming-App-using-AWS-ETL

Implemented Data Warehouse, Data Lake on AWS and Data modeling with Postgres and Apache Cassandra, Also used Apache Airflow to create data pipeline

Language: Jupyter Notebook - Size: 725 KB - Last synced at: 8 days ago - Pushed at: about 5 years ago - Stars: 4 - Forks: 2

aimlnerd/data-pipelines-with-airflow

Airflow DAG tutorial with docker compose local setup

Language: Python - Size: 16.6 KB - Last synced at: over 1 year ago - Pushed at: over 2 years ago - Stars: 3 - Forks: 0

osule/lard

An operator for performing the Extract and Load bit of the ELT data pipeline in an Apache Airflow workflow.

Language: Python - Size: 9.77 KB - Last synced at: 4 months ago - Pushed at: almost 3 years ago - Stars: 1 - Forks: 0

debussy-labs/debussy_concert

Debussy is an opinionated Data Architecture and Engineering framework, enabling data analysts and engineers to build better platforms and pipelines.

Language: Python - Size: 1.29 MB - Last synced at: 7 days ago - Pushed at: over 2 years ago - Stars: 28 - Forks: 5

sagardua297/udacity-data-engineering-nd

Data Pipeline Analytics Platform is an end-to-end generic Big Data pipeline. Involves following tech stack: AWS S3, AWS Redshift, AWS EMR Cluster, Apache Spark, Apache Airflow.

Language: Python - Size: 1.81 MB - Last synced at: almost 2 years ago - Pushed at: over 4 years ago - Stars: 1 - Forks: 0

Cori1109/airflow-provider-hightouch

Airflow operators, hooks, and sensors for interacting with the Hightouch API

Language: Python - Size: 39.1 KB - Last synced at: 27 days ago - Pushed at: over 2 years ago - Stars: 1 - Forks: 0

scriptstar/airflow-weather-station

Get started with Apache Airflow. Check the README for instructions on how to run your first DAGs today. 🚀

Language: Python - Size: 12.5 MB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0

hbasria/airflow-provider-redmine

Language: Python - Size: 14.6 KB - Last synced at: over 1 year ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0

Tinmarian/Apache_Airflow

Repositorio para realizar el curso llamado "Apache Airflow" en Udemy, expuesto por Javier López Tomás (Data Engineer)

Language: Python - Size: 45.9 KB - Last synced at: almost 2 years ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0

andyherko/airflow_docker_sample

Playing around with Airflow

Language: Python - Size: 22.5 KB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0

Faisal-AlDhuwayhi/Data-Pipelines-with-Airflow

Building Data Pipelines for a data warehouse with Airflow and AWS

Language: Python - Size: 85.9 KB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0

hereiamken/Udacity-Data-Engineering

All projects from Udacity Data Engineering Course

Language: Jupyter Notebook - Size: 5.97 MB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 4 - Forks: 0

IgnacioNMiranda/trigger-dag-run-params-demo

Apache Airflow demo project that setup 3 DAGs to explain how to pass parameters from a DAG to a triggered DAG.

Language: Python - Size: 4.88 KB - Last synced at: 5 months ago - Pushed at: over 2 years ago - Stars: 1 - Forks: 0

gustavosantosio/etl_api_twitter

Language: Python - Size: 51.5 MB - Last synced at: 9 days ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 1

addi-kamal/twitter-etl-airflow

Airflow Pipline to extract data from twitter API then transform it and finally load it to google cloud storage bucket.

Language: Python - Size: 5.86 KB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0

AfonsoFeliciano/Forex-Data-Pipeline

POC do curso Apache Airflow: The Hands-On Guide utilizando dados de paridade de moedas financeiras

Language: Shell - Size: 72.3 KB - Last synced at: over 2 years ago - Pushed at: almost 3 years ago - Stars: 0 - Forks: 1

essien1990/ETL_Pipeline_Airflow

Creating pipelines using Python3 and Apache Airflow to load tables into Google Big Query Dataware House

Language: Python - Size: 3.91 KB - Last synced at: 4 months ago - Pushed at: over 3 years ago - Stars: 1 - Forks: 0

narotam333/de-project-1a

How to use variables and runtime config in Apache Airflow

Language: Python - Size: 14.6 KB - Last synced at: about 2 years ago - Pushed at: about 3 years ago - Stars: 0 - Forks: 1

tilakpatidar/airflow-gpg-plugin

Airflow plugin with hooks and operators to work with GPG encryption and decryption.

Language: Python - Size: 26.4 KB - Last synced at: over 2 years ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0

frieds/udacity_data_eng_projects

Udacity data engineering nanodegree projects using song logs dataset

Language: Python - Size: 18.6 KB - Last synced at: over 2 years ago - Pushed at: over 5 years ago - Stars: 1 - Forks: 0

augusto-herrmann/FastETL Fork of economiagovbr/FastETL

Common functionality for implementing data flows in Apache Airflow DAGs.

Language: Python - Size: 4.75 MB - Last synced at: over 2 years ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0

qusay-elewy/data-pipelines-with-airflow

Udacity Data Engineering Nanodegree - Project #5

Language: Python - Size: 19.5 KB - Last synced at: 8 months ago - Pushed at: almost 4 years ago - Stars: 0 - Forks: 1

asdfgeoff/airflow-operators

A collection of useful operators for Apache Airflow

Language: Python - Size: 12.7 KB - Last synced at: over 2 years ago - Pushed at: over 5 years ago - Stars: 1 - Forks: 1

stivenramireza/airflow-service

Airflow service to programmatically author, schedule and monitor workflows.

Language: Python - Size: 26.4 KB - Last synced at: over 2 years ago - Pushed at: over 4 years ago - Stars: 0 - Forks: 0

angadsingh/airflow-hdinsight

HDInsight provider for Airflow

Language: Python - Size: 64.5 KB - Last synced at: 25 days ago - Pushed at: about 1 year ago - Stars: 5 - Forks: 2

buildupchao/airflow-resources

All kinds of airflow operator examples.

Size: 1000 Bytes - Last synced at: over 2 years ago - Pushed at: about 6 years ago - Stars: 0 - Forks: 0