Topic: "airflow-docker"
coder2j/airflow-docker
Source code of the Apache Airflow Tutorial for Beginners on YouTube Channel Coder2j (https://www.youtube.com/c/coder2j)
Language: Python - Size: 342 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 205 - Forks: 191
Wittline/uber-expenses-tracking
The goal of this project is to track the expenses of Uber Rides and Uber Eats through data Engineering processes using technologies such as Apache Airflow, AWS Redshift and Power BI.
Language: Jupyter Notebook - Size: 28.2 MB - Last synced at: about 2 months ago - Pushed at: almost 3 years ago - Stars: 121 - Forks: 36
data-burst/airflow-git-sync
Sync DAG changes from Git to Airflow
Size: 126 KB - Last synced at: 26 days ago - Pushed at: 26 days ago - Stars: 57 - Forks: 11
abhioncbr/docker-airflow
Repo for building docker based airflow image. Containers support multiple features like writing logs to local or S3 folder and Initializing GCP while container booting. https://abhioncbr.github.io/docker-airflow/
Language: Python - Size: 1.04 MB - Last synced at: 26 days ago - Pushed at: about 6 years ago - Stars: 32 - Forks: 5
sergio11/lyric_wave_architecture
๐ต LyricWave - Your AI Music Composer ๐ถ Compose Unique MP4 Songs Effortlessly! LyricWave uses AI to create personalized music by harmonizing lyrics with captivating melodies and synthetic vocals. Unleash your musical creativity today! ๐๐ถ
Language: Python - Size: 29 MB - Last synced at: 2 months ago - Pushed at: 3 months ago - Stars: 29 - Forks: 6
mikestack15/orangutan-stem
An open-source project dedicated to constructing robust data pipelines and scalable software infrastructure. We leverage industry-standard tools favored by developers to enhance efficiency and reliability. Uniquely, these pipelines are field-tested on farms across Sumatra, Indonesia, ensuring real-world applicability and resilience.
Language: Python - Size: 151 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 29 - Forks: 2
DatacollectorVN/Airflow-Tutorial
My self-learning about Apache Airflow
Language: Python - Size: 18.6 KB - Last synced at: over 2 years ago - Pushed at: almost 3 years ago - Stars: 29 - Forks: 12
mikeroyal/Apache-Airflow-Guide
Apache Airflow Guide
Language: Python - Size: 279 KB - Last synced at: about 1 month ago - Pushed at: about 1 year ago - Stars: 28 - Forks: 12
anilkulkarni87/airflow-docker
This is my Apache Airflow Local development setup on Windows 10 WSL2/Mac using docker-compose. It will also include some sample DAGs and workflows.
Language: Python - Size: 106 KB - Last synced at: about 1 year ago - Pushed at: over 1 year ago - Stars: 22 - Forks: 10
jksinghpro/docker-airflow
Docker for airflow with mysql as backend
Language: Shell - Size: 9.77 KB - Last synced at: over 2 years ago - Pushed at: over 6 years ago - Stars: 13 - Forks: 8
rahul765/Machine-Learning-Pipelines
From data gathering to model deployment. Complete ML pipeline using Docker, Airflow and Python.
Language: Python - Size: 335 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 12 - Forks: 3
vjgpt/twitter-pipeline
In this project, you will be building a Twitter Scheduler using Apache Airflow on Docker.
Language: Python - Size: 26.4 KB - Last synced at: 2 months ago - Pushed at: almost 5 years ago - Stars: 10 - Forks: 7
knowsuchagency/docker-compose-airflow
An airflow deployment configuration with sane defaults
Language: Python - Size: 47.9 KB - Last synced at: 6 days ago - Pushed at: about 6 years ago - Stars: 10 - Forks: 2
ericdaat/data-stack
A starting point for a data stack using Python, Apache Airflow and Metabase.
Language: Python - Size: 53.7 KB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 9 - Forks: 5
mpavanetti/airflow
This set of code and instructions has the porpouse to instanciate a compiled environment with set of docker images like airflow webserver, airflow scheduler, postgresql, pyspark, Data Pipeline consuming data from weather api , processing with pyspark and storing in postgresql
Language: PHP - Size: 1.62 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 8 - Forks: 3
okzapradhana/etl-flatfile-airflow
Building Data Warehouse on BigQuery which takes flat file as the data sources with Airflow as the Orchestrator
Language: Python - Size: 34.9 MB - Last synced at: over 2 years ago - Pushed at: about 4 years ago - Stars: 8 - Forks: 6
pacuna/airflow-docker
Run Apache Airflow using Docker containers
Language: Python - Size: 6.84 KB - Last synced at: about 2 months ago - Pushed at: almost 7 years ago - Stars: 8 - Forks: 6
judeleonard/Prescriber-ETL-data-pipeline
An End-to-End ETL data pipeline that leverages pyspark parallel processing to process about 25 million rows of data coming from a SaaS application using Apache Airflow as an orchestration tool and various data warehouse technologies and finally using Apache Superset to connect to DWH for generating BI dashboards for weekly reports
Language: Python - Size: 7.42 MB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 7 - Forks: 1
resdevd/docker-compose-airflow
Quick Docker Compose deployment for Apache Airflow version 2.x. Ideal for local development
Language: Shell - Size: 3.91 KB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 7 - Forks: 6
AhmetFurkanDEMIR/airflow-spark-kafka-example
Airflow, Spark and Kafka example
Language: Dockerfile - Size: 532 KB - Last synced at: about 2 months ago - Pushed at: over 1 year ago - Stars: 6 - Forks: 0
Aiscalate/aiscalator
Tools to streamline Jupyter Notebook Prototypes into robust Data Products
Language: Python - Size: 1.53 MB - Last synced at: 19 days ago - Pushed at: over 2 years ago - Stars: 5 - Forks: 2
DunnBC22/Data_Engineering_Projects
This repository includes data engineering projects using Apache Airflow. I hope to add more projects using different technologies soon!
Language: TSQL - Size: 62.7 MB - Last synced at: 10 days ago - Pushed at: 10 days ago - Stars: 4 - Forks: 0
rushabh2390/SharesETL
Shares ETL to scrape historical data of one shares and import in metabase
Language: Python - Size: 70.3 KB - Last synced at: almost 2 years ago - Pushed at: over 2 years ago - Stars: 4 - Forks: 0
sreekesh93/airflow_beginners
to start airflow in local with basic setup
Language: Python - Size: 2.93 KB - Last synced at: over 1 year ago - Pushed at: over 3 years ago - Stars: 4 - Forks: 0
gustschaefer/Twitter-Batch-ETL
ETL que extraรญ diariamente os trending topics do Twitter em diversos paรญses, realiza transformaรงรตes com PySpark e envia os dados para o Amazon S3 utilizando Apache Airflow como orquestrador.
Language: Python - Size: 1.03 MB - Last synced at: 6 months ago - Pushed at: about 4 years ago - Stars: 4 - Forks: 0
saianupkumarp/airflow-docker
Airflow docker boiler plate with LocalExecutor, CeleryExecutor. #TODO Kubernetes
Language: Python - Size: 33.2 KB - Last synced at: over 2 years ago - Pushed at: over 5 years ago - Stars: 4 - Forks: 1
Pirate-Emperor/BigData-Pipeline
BigData Pipeline is a local testing environment for experimenting with various storage solutions (RDB, HDFS), query engines (Trino), schedulers (Airflow), and ETL/ELT tools (DBT). It supports MySQL, Hadoop, Hive, Kudu, and more.
Language: Dockerfile - Size: 7.95 MB - Last synced at: 2 months ago - Pushed at: 7 months ago - Stars: 3 - Forks: 0
abhishekbhakat/Turbine
Simple python script for easy local airflow deployment with docker. Packed with additional components. Will be adding more going forward.
Language: Python - Size: 113 KB - Last synced at: 7 days ago - Pushed at: 8 months ago - Stars: 3 - Forks: 0
Chinaskidev/ETL-Clima-ElSalvador
MLOps, haciendo un ETL sencillo usando Docker y Airflow y Google Cloud
Language: Python - Size: 50.8 KB - Last synced at: about 2 months ago - Pushed at: about 1 year ago - Stars: 3 - Forks: 1
aimlnerd/data-pipelines-with-airflow
Airflow DAG tutorial with docker compose local setup
Language: Python - Size: 16.6 KB - Last synced at: over 1 year ago - Pushed at: over 2 years ago - Stars: 3 - Forks: 0
vcwild/airflow-init
Apache Airflow quick setup orchestrated with Docker Compose
Language: Python - Size: 317 KB - Last synced at: 8 days ago - Pushed at: almost 4 years ago - Stars: 3 - Forks: 0
slatawa/Airflow-Monitoring-ElasticSearch-LogStash-Kibana
In this project we look to set up Airflow monitoring using ElasticSearch-LogStash-Kibana (ELK stack). We will set up the stack using docker images.
Language: Shell - Size: 3.31 MB - Last synced at: 2 months ago - Pushed at: almost 4 years ago - Stars: 3 - Forks: 3
chattarajoy/airflow-kube-helm Fork of BrechtDeVlieger/airflow-kube-helm
Airflow helm chart (KubernetesExecutor)
Language: Shell - Size: 47.9 KB - Last synced at: over 2 years ago - Pushed at: over 5 years ago - Stars: 3 - Forks: 5
Jiansiyu/airflow_docker_compose
This is a simplified version of Apache Airflow manage with docker compose v2, It support running docker image inside DAG file
Language: Python - Size: 240 KB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 2 - Forks: 0
judeleonard/Data-Engineering
This is a repo to keep track of all my projects related to data engineering
Language: Python - Size: 24 MB - Last synced at: 7 months ago - Pushed at: 7 months ago - Stars: 2 - Forks: 2
ALEXUSCR-27/Amazon-Data-ETL
ETL pipeline for Amazon product sales data, using Apache Airflow for data orchestration and Supabase for storage, by containerizing the environment with Docker, the setup is scalable and easily deployable, supporting data-driven decision-making.
Language: Python - Size: 4.49 MB - Last synced at: 2 months ago - Pushed at: 7 months ago - Stars: 2 - Forks: 1
kaushil24/llm-finetune-pipeline
Airflow pipeline to finetune LLM on Kubernetes
Language: Python - Size: 5.8 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 2 - Forks: 0
frankcash/airflow_compose
A simple docker-compose ready repository for using Apache Airflow.
Language: Python - Size: 57.6 KB - Last synced at: 3 months ago - Pushed at: about 2 years ago - Stars: 2 - Forks: 0
vikrantdeshpande09876/Credit_Card_Fraud_Detection
Deploying Classification Model on GCP for detecting anomalous behavior in credit-card transactions. A crucial aspect for credit-card companies to recognize fraudulent activity and not charge customers for items they didn't purchase. A subtle line needs to be drawn and my main focus here is to reduce blocked-transactions of legitimate customers.
Language: Jupyter Notebook - Size: 41.7 MB - Last synced at: 27 days ago - Pushed at: about 2 years ago - Stars: 2 - Forks: 1
dataopstix/modelt
Modelt(mowยทdelt) is a modern data integration solution that connects data to data for advanced analytics.
Language: Shell - Size: 463 KB - Last synced at: 2 months ago - Pushed at: about 3 years ago - Stars: 2 - Forks: 1
thiagoheron1/study_airflow
Repositรณrio criado para estudar Airflow baseado nos cursos do Marc Lamberti
Language: Python - Size: 1.03 MB - Last synced at: over 2 years ago - Pushed at: almost 4 years ago - Stars: 2 - Forks: 0
netflame/docker-airflow
Dockerized Airflow based on Alpine
Language: Python - Size: 6.84 KB - Last synced at: over 2 years ago - Pushed at: about 6 years ago - Stars: 2 - Forks: 0
yuhexiong/airflow-dag-kafka-flink-doris-python
Language: Python - Size: 43 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 1 - Forks: 0
choval/devairflow
Simple local development airflow image
Size: 111 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 1 - Forks: 0
ByteMeDirk/apache-airflow-data-engineer
Apache Airflow For Data Engineers Tutorial
Language: Python - Size: 6.69 MB - Last synced at: 3 months ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 1
stevehoober254/ETL_Data_Pipeline_For_Retail_Store
ETL (Extract, Transform, Load) pipeline to integrate sales data from various sources into a central data warehouse
Language: Python - Size: 5.86 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 0
Serge-Ntumba/Daily_automated_ETL
This project focuses on the seamless integration of data from a CSV file into a PostgreSQL database using Python. Additionally, Airflow is employed to perform a daily extraction of this data, transferring it to an S3 Bucket for further analysis and storage.
Language: Python - Size: 228 KB - Last synced at: about 1 year ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 0
Serge-Ntumba/stream_data_with_kafka_docker_airflow_spark
A complete data pipeline, from data extraction to storage, using a combination of tools for specific purposes: Python for data retrieval from API, Airflow for scheduling task, Kafka for data streaming, Spark for data processing, and Cassandra for data storage.
Language: Python - Size: 298 KB - Last synced at: about 1 year ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 0
CanaanGM/databases-infrastructure
on demand databases deployment, varuios kinds, adding more as i use them!
Language: Python - Size: 4.02 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 0
Gakhramanzode/Airflow
Bash-ัะบัะธะฟั ะฟะพ ัััะฐะฝะพะฒะบะต Airflow
Language: Shell - Size: 14.6 KB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 1 - Forks: 1
anastasiia-p/airflow-ml
Airflow Pipeline for Machine Learning
Language: Python - Size: 9.77 KB - Last synced at: about 2 years ago - Pushed at: about 2 years ago - Stars: 1 - Forks: 0
tier940/docker-repo
Language: Python - Size: 115 KB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 1 - Forks: 0
seanleong339/Twitdockair
An ETL(Extract, Transform, Load) Data pipeline project, using Apache Airflow on Docker
Language: Python - Size: 72.3 KB - Last synced at: over 2 years ago - Pushed at: almost 3 years ago - Stars: 1 - Forks: 0
AakashDeorukhkar/Airflow_NYC_Taxi_Data
Ingesting NYC Yellow Taxi Data to GCP using Airflow running on Docker
Language: Python - Size: 4.08 MB - Last synced at: 10 months ago - Pushed at: almost 3 years ago - Stars: 1 - Forks: 0
julientoucoula17/apache_airflow-with-Docker
Apache Airflow installation with Docker ๐ฌ๏ธ
Language: Python - Size: 10.7 KB - Last synced at: 17 days ago - Pushed at: about 3 years ago - Stars: 1 - Forks: 0
finloop/grafana-airflow-etl
End to end data pipeline with Apache Airflow and Grafana
Language: Python - Size: 41.9 MB - Last synced at: 7 days ago - Pushed at: about 3 years ago - Stars: 1 - Forks: 2
ruanbekker/docker-apache-airflow
Apache Airflow using Docker Compose
Language: Python - Size: 12.7 KB - Last synced at: 3 days ago - Pushed at: almost 4 years ago - Stars: 1 - Forks: 0
BruceDone/airflow-docker
Docker Apache Airflow
Language: Shell - Size: 9.77 KB - Last synced at: 3 months ago - Pushed at: almost 8 years ago - Stars: 1 - Forks: 1
ethanwebber123/Airflow-ETL-ELT
Airflow-ETL-ELT is a robust data pipeline tool that enables efficient extraction, transformation, loading, and orchestration of data workflows. It offers a scalable and customizable solution for managing complex ETL and ELT processes with ease.
Size: 1000 Bytes - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 0 - Forks: 0
navs-svan/nba-dashboard
PowerBI Dashboard using scraped box score data of the NBA Season 2024-25
Language: Python - Size: 12 MB - Last synced at: 6 days ago - Pushed at: 7 days ago - Stars: 0 - Forks: 0
opt-nc/airflow-extended-image
Image Docker Airflow personnalisรฉe avec des providers additionnels pour รฉtendre les fonctionnalitรฉs de base et simplifier le dรฉploiement.
Language: Dockerfile - Size: 12.7 KB - Last synced at: 20 days ago - Pushed at: 20 days ago - Stars: 0 - Forks: 0
rahulsamant37/FraudDetection
Build Realtime Fraud Detection AI from Scratch - End to End Machine Learning Project
Language: Shell - Size: 10.7 KB - Last synced at: 28 days ago - Pushed at: 28 days ago - Stars: 0 - Forks: 0
aakashsyadav1999/Covid-Data-Engineering-Project
This is repo which will guide you about the data engineering project which will Extract, Load and then Transform into the database.
Language: Python - Size: 444 KB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 0 - Forks: 0
micheldpd24/cust_review_analysis
End-to-End Review Analysis Pipeline : Scrape, process, and analyze customer reviews from Trustpilot using Apache Airflow and Python | Interactive Dash Dashboard : Visualize sentiment, trends, and textual insights with a dynamic, Dockerized Dash app.
Language: Jupyter Notebook - Size: 315 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 0 - Forks: 0
enricogoerlitz/explore-airflow
Apache Airflow project for orchestrating ETL workflows in a data warehouse environment. Implements a medallion architecture (Bronze, Silver, Gold layers) with Postgres integration for scalable and modular data processing pipelines.
Language: Python - Size: 1.33 MB - Last synced at: 2 months ago - Pushed at: 2 months ago - Stars: 0 - Forks: 0
micheldpd24/mlops_air_msr
MLOps Pipeline for Music Recommendation - Spotify playlist continuation
Language: Python - Size: 25.5 MB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0
Ramshankar07/Stock-HMM
Streamlit application designed to forecast stock prices using a Hidden Markov Model (HMM) based on historical stock price data. The application enables users to view 1-day and multi-day forecasts for several major tech stocks, including: Apple Inc. (AAPL) Tesla Inc. (TSLA) NVIDIA Corporation (NVDA)
Language: Python - Size: 47.9 KB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 0 - Forks: 0
nasirkadri2601/Live_Cricket_Data_Pipeline
This project captures live cricket data in raw JSON format, cleans and transforms it, and stores it in a centralized data warehouse. The data is then used for analysis, including match outcome predictions, player performance, and team strategy insights, enabling data-driven decisions.
Language: Jupyter Notebook - Size: 25.6 MB - Last synced at: 2 months ago - Pushed at: 6 months ago - Stars: 0 - Forks: 0
RayanGAtech/HR-Roster-Change-Detection-Pipeline-with-Apache-Airflow-and-PostgreSQL
Automated data pipeline tailored for HR systems to process and analyze roster data. The pipeline leverages Apache Airflow to ingest and transform CSV files containing roster information, and PostgreSQL to store, deduplicate, and extract unique data points.
Language: Python - Size: 6.84 KB - Last synced at: 6 months ago - Pushed at: 6 months ago - Stars: 0 - Forks: 0
wlopezm-unal/reddit_project_airflow_aws
This project focuses on implementing an ETL pipeline using Apache Airflow to efficiently extract data from Reddit, transform it as needed, and load it into an AWS S3 bucket. The use of Airflow allows for robust orchestration of the data workflow, ensuring that each step of the ETL process is executed in a reliable and repeatable manner.
Language: Python - Size: 38.1 KB - Last synced at: 2 months ago - Pushed at: 7 months ago - Stars: 0 - Forks: 0
TuanAnh30/ETL_With_AirFlow
Simulate loading log data in JSON files into a database using an ETL process with Airflow to automate execution.
Language: Python - Size: 25.4 MB - Last synced at: 8 months ago - Pushed at: 8 months ago - Stars: 0 - Forks: 0
jasontrandev/ETL-Data-Pipeline-Airflow-Project1
Build an ETL Data Pipelines that uses Airflow DAGs to extract employeesโ data from PostgreSQL Schemas, load it into AWS Data Lake, then transform it with Python script, and finally load it into Snowflake Data warehouse using SCD type 2.
Language: Python - Size: 3.74 MB - Last synced at: 9 months ago - Pushed at: 9 months ago - Stars: 0 - Forks: 0
joshua-dada-mayowa/Data-Engineer-ETL-Project
ETL Project using Airflow, DBT, Bigquery,Postgres,Docker and Docker Compose
Language: Python - Size: 41.3 MB - Last synced at: 10 months ago - Pushed at: 10 months ago - Stars: 0 - Forks: 0
mashalk1/airflow-using-docker-compose
A configuration file to run Apache Airflow 2.0 via Docker. The attached file is a Docker Compose file which is much easier to run. All you have to do is install the file, go to that directory and run "docker-compose up" and the rest will done by docker itself.
Size: 3.91 KB - Last synced at: 11 days ago - Pushed at: 10 months ago - Stars: 0 - Forks: 0
igorlangoni/online_retail_data_pipeline
An end-to-end pipeline that ingests raw data from CSV files through Airflow DAGS into BigQuery. From there, it uses dbt to normalize and clean the data and afterwards to make the transformations and come up with relevan reports.
Language: Python - Size: 15.4 MB - Last synced at: 10 months ago - Pushed at: 10 months ago - Stars: 0 - Forks: 0
RafaelQSantos-RQS/windturbine-airflow-project
Este projeto foi desenvolvido no curso de Airflow da Udemy, lecionado pelo professor Fernando Amaral.
Language: Python - Size: 186 KB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 0 - Forks: 0
djamier/docker-airflow
Orchestrate data pipeline using airflow
Language: Python - Size: 43.9 KB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 0 - Forks: 0
aohus/etl-pipeline-with-modeling
Docker๋ฅผ ์ฌ์ฉํ์ฌ Hadoop ์ํ๊ณ์ ๊ตฌ์ฑ ์์์ ๊ธฐํ ํ์ ์๋น์ค๋ฅผ ์ปจํ ์ด๋ํํ์ฌ ๊ฐ๋ ฅํ ๋ฐ์ดํฐ ์์ง๋์ด๋ง ํ๊ฒฝ์ ์ค์ ํ๋ ๋ฐฉ๋ฒ์ ๋ณด์ฌ์ค๋๋ค. ์ค์ ์๋ Hadoop (HDFS, YARN), Apache Hive, PostgreSQL ๋ฐ Apache Airflow๊ฐ ํฌํจ๋๋ฉฐ, ์ด๋ค ๋ชจ๋๊ฐ ์ํํ๊ฒ ์๋ํ๋๋ก ๊ตฌ์ฑ๋์ด ์์ต๋๋ค.
Language: Shell - Size: 20.4 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0
shiv-rna/Airflow-Basics
Welcome to my Apache Airflow learning journey repository! ๐ This repository serves as a comprehensive documentation of my exploration and understanding of Apache Airflow, an open-source platform for orchestrating complex workflows.
Language: Python - Size: 65.8 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
AlexanderM-T/AirFlow-Training
En este repositorio se encuentran algunos dags realizados siguiendo el entrenamiento de data engineer con el fin de aprender y practicar airflow
Language: Python - Size: 18.6 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
dorokhin/apache_airflow_docker
Apache airflow packed in docker compose
Language: Python - Size: 6.84 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
IanDanielM/BuyRentScrap-LuxAirflow
Automating Data Scrapers With Python and Airflow
Language: Python - Size: 5.86 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
Sheitak/datalake-jljq
Data Lake project for ingest and transform financial data and dashboard BI proposal
Language: Python - Size: 41 KB - Last synced at: 1 day ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
Booss3my/Datascraper
Automated Indeed Job Offer Scraper: Airflow Orchestrated and Scheduled, Data Loaded into PostgreSQL Database
Language: Python - Size: 1.45 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
martinsejas/FlightPricePrediction
An end-to-end MLOps Pipeline, from data ingestion to inference. Powered by Azure, FastAPI, Apache Airflow, Streamlit and Sklearn.
Language: Jupyter Notebook - Size: 3.87 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
SoukainaBoufousse/ETL_Sales_Data
This repository demonstrates a data engineering ETL (Extract, Transform, Load) pipeline for processing sales data. It utilizes Docker and Apache Airflow for orchestration and PostgreSQL for data storage. The pipeline extracts, transforms, and loads sales data.
Language: Python - Size: 385 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
niyotham/Data-engineer-end-to-end-project-airflow-dbt-soda-bigquery
An end to end data engineering project for loading data into bigquery with airflow and perform transformations using dbt and data quality check with soday
Language: Python - Size: 7.03 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
ndangalasinh/Toronto-Apartment-Evaluation-Analysis
Data Engineering End-to-End Project
Language: Python - Size: 2.87 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
prasadanilmore/ETL-JustEat-TestCase
This documentation provides an overview of the tasks completed in this project. The project comprises three key tasks: ETL (Extract, Transform, Load), API development, and Data Orchestration using Airflow. Each task is detailed below along with explanations of design choices and considerations.
Language: Python - Size: 79.1 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
sunghj1118/airflow
In this repository I study Airflow and related topics/implementations.
Language: Python - Size: 539 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
Penosh22/Market-news-and-forex-exchange-ETL-pipeline
ETL pipeline to extract webscraped forex data and google news library data on daily basis and storing it in a postgres database for market analysis and insights
Language: Python - Size: 1.95 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
avocadojj/marketing
Making pipeline for bank marketing campaign
Language: Jupyter Notebook - Size: 212 MB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 2
lafirm/gold-price-data-engg
Get Current Gold Rate Periodically and Store the data using Airflow (Taskflow API) using Docker
Language: Python - Size: 203 KB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0
andyherko/airflow_docker_sample
Playing around with Airflow
Language: Python - Size: 22.5 KB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0
judeleonard/e-commerce_activity_tracking
This is an ELT data pipeline setup to track the activities of an e-commerce website based on orders, reviews, deliveries and shipment date. This project utilized technologies like Airflow, AWS RDS-Postgres, Python etc.
Language: Python - Size: 596 KB - Last synced at: over 1 year ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0
AqilaFadia/Data_Engineer-Portfolio
This is My Example Project Data Engineer
Language: Python - Size: 2.93 KB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0
scalactic/airflow-example
airflow-example in docker
Language: Python - Size: 7.81 KB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0
jesslossa/airflow-docker-localexecutor
Implements Airflow through docker on localhost (LocalExecutor), using docker-desktop.
Language: Python - Size: 171 KB - Last synced at: about 1 year ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0
muhk01/airflow_bmkg_data
Airflow ETL for Earthquake Data
Language: Python - Size: 235 KB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0
rikiafifuddin/airflow-docker
Airflow Docker with pip instalation using requirement.txt and using DAG from cloned repository
Language: Dockerfile - Size: 16.6 KB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0
