airflow-operators | Topic | Ecosyste.ms: Repos

Topic: "airflow-operators"

san089/Udacity-Data-Engineering-Projects

Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.

Language: Python - Size: 2.03 MB - Last synced at: 19 days ago - Pushed at: almost 3 years ago - Stars: 1,620 - Forks: 527

astronomer/astronomer-cosmos

Run your dbt Core projects as Apache Airflow DAGs and Task Groups with a few lines of code

Language: Python - Size: 18.2 MB - Last synced at: 12 days ago - Pushed at: 12 days ago - Stars: 955 - Forks: 214

tomasfarias/airflow-dbt-python

A collection of Airflow operators, hooks, and utilities to elevate dbt to a first-class citizen of Airflow.

Language: Python - Size: 7.9 MB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 199 - Forks: 39

astronomer/airflow-provider-great-expectations

Great Expectations Airflow operator

Language: Python - Size: 2.18 MB - Last synced at: 15 days ago - Pushed at: 15 days ago - Stars: 164 - Forks: 57

astronomer/astronomer-providers

Airflow Providers containing Deferrable Operators & Sensors from Astronomer

Language: Python - Size: 2.39 MB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 149 - Forks: 26

gestaogovbr/FastETL

Airflow plugins for implementing data pipelines. | Plugins do Airflow para implementação de pipelines de dados.

Language: Python - Size: 4.93 MB - Last synced at: 13 days ago - Pushed at: 13 days ago - Stars: 46 - Forks: 8

sergio11/lyric_wave_architecture

🎵 LyricWave – AI Music Composer (Proof of Concept) 🎶 A personal project exploring automatic generation of unique MP4 songs. LyricWave blends lyrics with AI-generated melodies and synthetic vocals to experiment with new forms of musical expression. A creative testbed to push your ideas into sound. 🚀🎧

Language: Python - Size: 29 MB - Last synced at: 8 days ago - Pushed at: 3 months ago - Stars: 34 - Forks: 6

mikeroyal/Apache-Airflow-Guide

Apache Airflow Guide

Language: Python - Size: 279 KB - Last synced at: about 1 month ago - Pushed at: about 1 year ago - Stars: 28 - Forks: 12

debussy-labs/debussy_concert

Debussy is an opinionated Data Architecture and Engineering framework, enabling data analysts and engineers to build better platforms and pipelines.

Language: Python - Size: 1.29 MB - Last synced at: about 1 month ago - Pushed at: about 2 years ago - Stars: 28 - Forks: 5

hightouchio/airflow-provider-hightouch

Airflow operators, hooks, and sensors for interacting with the Hightouch API

Language: Python - Size: 59.6 KB - Last synced at: 13 days ago - Pushed at: 11 months ago - Stars: 17 - Forks: 11

astronomer/astro-provider-anyscale

An Apache Airflow provider for seamless integration with Anyscale.

Language: Python - Size: 218 KB - Last synced at: 13 days ago - Pushed at: 3 months ago - Stars: 5 - Forks: 3

angadsingh/airflow-hdinsight

HDInsight provider for Airflow

Language: Python - Size: 64.5 KB - Last synced at: 17 days ago - Pushed at: about 1 year ago - Stars: 5 - Forks: 2

airflow-laminar/airflow-common-operators

Common Airflow Operators / Tasks

Language: Python - Size: 336 KB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 4 - Forks: 0

tensor365/AIRFLOW-QLIKSENSE-CLOUD

Qlik Sense Cloud Provider for Apache Airflow

Language: Python - Size: 191 KB - Last synced at: 24 days ago - Pushed at: 24 days ago - Stars: 4 - Forks: 2

hereiamken/Udacity-Data-Engineering

All projects from Udacity Data Engineering Course

Language: Jupyter Notebook - Size: 5.97 MB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 4 - Forks: 0

DataDrivenGit/Music-Streaming-App-using-AWS-ETL

Implemented Data Warehouse, Data Lake on AWS and Data modeling with Postgres and Apache Cassandra, Also used Apache Airflow to create data pipeline

Language: Jupyter Notebook - Size: 725 KB - Last synced at: 7 months ago - Pushed at: almost 5 years ago - Stars: 4 - Forks: 3

aimlnerd/data-pipelines-with-airflow

Airflow DAG tutorial with docker compose local setup

Language: Python - Size: 16.6 KB - Last synced at: over 1 year ago - Pushed at: over 2 years ago - Stars: 3 - Forks: 0

lynker-analytics/airflow-conda-operator

Airflow task & operator running python code in conda/mamba environments

Language: Python - Size: 19.5 KB - Last synced at: 10 days ago - Pushed at: 10 days ago - Stars: 2 - Forks: 2

sdaberdaku/apache-airflow-providers-pysparkonk8s

Python package for the pysparkonk8s Apache Airflow provider.

Language: Python - Size: 101 KB - Last synced at: 11 days ago - Pushed at: 4 months ago - Stars: 2 - Forks: 1

Amsterdam/dataservices-airflow 📦

Airflow docker container for Dataservices.

Language: Python - Size: 5.36 MB - Last synced at: 3 months ago - Pushed at: about 1 year ago - Stars: 2 - Forks: 4

WilsonH918/Data_Pipeline_ETL_Pipeline_Web3_Token

Data Pipeline Project

Language: Python - Size: 17.6 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 1 - Forks: 0

An4PDM/Aprendendo-a-criar-DAGs

Este repositório tem objetivo de organizar todas as DAGs que estou criando para fim de consultas posteriores.

Language: Python - Size: 10.7 KB - Last synced at: 4 months ago - Pushed at: 5 months ago - Stars: 1 - Forks: 0

sp1thas/airflow-provider-kaggle

[MIRROR OF https://codeberg.org/sp1thas/airflow-provider-kaggle] Airflow operators and hooks for interacting with the Kaggle API

Language: Python - Size: 47.9 KB - Last synced at: 6 days ago - Pushed at: 6 months ago - Stars: 1 - Forks: 0

hbasria/airflow-provider-baserow

Language: Python - Size: 14.6 KB - Last synced at: about 1 year ago - Pushed at: almost 2 years ago - Stars: 1 - Forks: 0

Cori1109/airflow-provider-hightouch

Airflow operators, hooks, and sensors for interacting with the Hightouch API

Language: Python - Size: 39.1 KB - Last synced at: 3 months ago - Pushed at: over 2 years ago - Stars: 1 - Forks: 0

IgnacioNMiranda/trigger-dag-run-params-demo

Apache Airflow demo project that setup 3 DAGs to explain how to pass parameters from a DAG to a triggered DAG.

Language: Python - Size: 4.88 KB - Last synced at: 3 months ago - Pushed at: over 2 years ago - Stars: 1 - Forks: 0

osule/lard

An operator for performing the Extract and Load bit of the ELT data pipeline in an Apache Airflow workflow.

Language: Python - Size: 9.77 KB - Last synced at: 3 months ago - Pushed at: almost 3 years ago - Stars: 1 - Forks: 0

essien1990/ETL_Pipeline_Airflow

Creating pipelines using Python3 and Apache Airflow to load tables into Google Big Query Dataware House

Language: Python - Size: 3.91 KB - Last synced at: 3 months ago - Pushed at: over 3 years ago - Stars: 1 - Forks: 0

emsalcengiz/Etl_processing

I find Apache Airflow very useful for ETL work. Here I transferred data from the source database(mysql) to the target database(postresql) and used the Airflow Bash Operator.

Language: Python - Size: 9.77 KB - Last synced at: 10 months ago - Pushed at: about 4 years ago - Stars: 1 - Forks: 1

sagardua297/udacity-data-engineering-nd

Data Pipeline Analytics Platform is an end-to-end generic Big Data pipeline. Involves following tech stack: AWS S3, AWS Redshift, AWS EMR Cluster, Apache Spark, Apache Airflow.

Language: Python - Size: 1.81 MB - Last synced at: over 1 year ago - Pushed at: over 4 years ago - Stars: 1 - Forks: 0

frieds/udacity_data_eng_projects

Udacity data engineering nanodegree projects using song logs dataset

Language: Python - Size: 18.6 KB - Last synced at: about 2 years ago - Pushed at: over 5 years ago - Stars: 1 - Forks: 0

asdfgeoff/airflow-operators

A collection of useful operators for Apache Airflow

Language: Python - Size: 12.7 KB - Last synced at: over 2 years ago - Pushed at: over 5 years ago - Stars: 1 - Forks: 1

tensor365/AIRFLOW-QLIKSENSE-CLIENT-MANAGED

Trigger Qlik Sense Task From an airflow operator

Language: Python - Size: 47.9 KB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 0 - Forks: 0

wherobots/airflow-providers-wherobots

Airflow extensions for communicating with Wherobots Cloud

Language: Python - Size: 998 KB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 0 - Forks: 2

sangwanamit621/learning-and-experiments

Welcome to the Learning and Experiments Hub—a dynamic repository capturing my journey of exploration and experimentation in the vast world of technology. This space serves as a digital canvas where I document my learning process, experiments, and discoveries.

Language: Jupyter Notebook - Size: 53.7 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

AlvaroCavalcante/airflow-custom-deferrable-dataflow-operator

Start your Dataflow jobs execution directly from the Triggerer without going to the Worker!

Language: Python - Size: 36.1 KB - Last synced at: 14 days ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0

RADar-AZDelta/lib_airflow

Custom Airflow operators and hooks, that help to upload our raw hospital data to Google Cloud.

Language: Python - Size: 188 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0

rnaguilar/airflow_custom_operator

Contribuições de custom operators no airflow

Language: Python - Size: 0 Bytes - Last synced at: 6 months ago - Pushed at: 6 months ago - Stars: 0 - Forks: 0

HakeemSalaudeen/CoreSentiments-project

This project is designed to reinforce my understanding of Data Pipeline Orchestration with Apache Airflow. The focus is on implementing a data pipeline that addresses data ingestion, processing, storage, and analysis. The project challenges me to apply my knowledge of Apache Airflow to solve a practical scenario based problem

Language: Python - Size: 7.81 KB - Last synced at: 7 months ago - Pushed at: 7 months ago - Stars: 0 - Forks: 0

shiv-rna/Airflow-Basics

Welcome to my Apache Airflow learning journey repository! 🚀 This repository serves as a comprehensive documentation of my exploration and understanding of Apache Airflow, an open-source platform for orchestrating complex workflows.

Language: Python - Size: 65.8 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

scriptstar/airflow-weather-station

Get started with Apache Airflow. Check the README for instructions on how to run your first DAGs today. 🚀

Language: Python - Size: 12.5 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

hbasria/airflow-provider-redmine

Language: Python - Size: 14.6 KB - Last synced at: about 1 year ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0

andyherko/airflow_docker_sample

Playing around with Airflow

Language: Python - Size: 22.5 KB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0

Tinmarian/Apache_Airflow

Repositorio para realizar el curso llamado "Apache Airflow" en Udemy, expuesto por Javier López Tomás (Data Engineer)

Language: Python - Size: 45.9 KB - Last synced at: almost 2 years ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0

BaliDataMan/airflow-Tut

Airflow Tutorial to learn practical implementation with basics Concepts, Architecture, Working & Configuration practises.

Language: Shell - Size: 425 KB - Last synced at: about 1 year ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0

addi-kamal/twitter-etl-airflow

Airflow Pipline to extract data from twitter API then transform it and finally load it to google cloud storage bucket.

Language: Python - Size: 5.86 KB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0

gustavosantosio/etl_api_twitter

Language: Python - Size: 51.5 MB - Last synced at: 3 days ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 1

Faisal-AlDhuwayhi/Data-Pipelines-with-Airflow

Building Data Pipelines for a data warehouse with Airflow and AWS

Language: Python - Size: 85.9 KB - Last synced at: almost 2 years ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0

AfonsoFeliciano/Forex-Data-Pipeline

POC do curso Apache Airflow: The Hands-On Guide utilizando dados de paridade de moedas financeiras

Language: Shell - Size: 72.3 KB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 1

narotam333/de-project-1a

How to use variables and runtime config in Apache Airflow

Language: Python - Size: 14.6 KB - Last synced at: about 2 years ago - Pushed at: about 3 years ago - Stars: 0 - Forks: 1

tilakpatidar/airflow-gpg-plugin

Airflow plugin with hooks and operators to work with GPG encryption and decryption.

Language: Python - Size: 26.4 KB - Last synced at: over 2 years ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0

augusto-herrmann/FastETL Fork of economiagovbr/FastETL

Common functionality for implementing data flows in Apache Airflow DAGs.

Language: Python - Size: 4.75 MB - Last synced at: about 2 years ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0