GitHub topics: airflow-docker | Ecosyste.ms: Repos

ethanwebber123/Airflow-ETL-ELT

Airflow-ETL-ELT is a robust data pipeline tool that enables efficient extraction, transformation, loading, and orchestration of data workflows. It offers a scalable and customizable solution for managing complex ETL and ELT processes with ease.

Size: 1000 Bytes - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 0 - Forks: 0

rahulsamant37/FraudDetection

Build Realtime Fraud Detection AI from Scratch - End to End Machine Learning Project

Language: Shell - Size: 10.7 KB - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 0 - Forks: 0

aakashsyadav1999/Covid-Data-Engineering-Project

This is repo which will guide you about the data engineering project which will Extract, Load and then Transform into the database.

Language: Python - Size: 444 KB - Last synced at: 11 days ago - Pushed at: 11 days ago - Stars: 0 - Forks: 0

opt-nc/airflow-extended-image

Image Docker Airflow personnalisée avec des providers additionnels pour étendre les fonctionnalités de base et simplifier le déploiement.

Language: Dockerfile - Size: 16.6 KB - Last synced at: 17 days ago - Pushed at: 17 days ago - Stars: 0 - Forks: 0

DunnBC22/Data_Engineering_Projects

This repository includes data engineering projects using Apache Airflow. I hope to add more projects using different technologies soon!

Language: TSQL - Size: 60.5 MB - Last synced at: 17 days ago - Pushed at: 17 days ago - Stars: 4 - Forks: 0

yuhexiong/airflow-dag-kafka-flink-doris-python

Language: Python - Size: 43 KB - Last synced at: 20 days ago - Pushed at: 20 days ago - Stars: 1 - Forks: 0

Wittline/uber-expenses-tracking

The goal of this project is to track the expenses of Uber Rides and Uber Eats through data Engineering processes using technologies such as Apache Airflow, AWS Redshift and Power BI.

Language: Jupyter Notebook - Size: 28.2 MB - Last synced at: about 1 month ago - Pushed at: almost 3 years ago - Stars: 121 - Forks: 36

micheldpd24/cust_review_analysis

End-to-End Review Analysis Pipeline : Scrape, process, and analyze customer reviews from Trustpilot using Apache Airflow and Python | Interactive Dash Dashboard : Visualize sentiment, trends, and textual insights with a dynamic, Dockerized Dash app.

Language: Jupyter Notebook - Size: 315 KB - Last synced at: 25 days ago - Pushed at: 26 days ago - Stars: 0 - Forks: 0

enricogoerlitz/explore-airflow

Apache Airflow project for orchestrating ETL workflows in a data warehouse environment. Implements a medallion architecture (Bronze, Silver, Gold layers) with Postgres integration for scalable and modular data processing pipelines.

Language: Python - Size: 1.33 MB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 0 - Forks: 0

micheldpd24/mlops_air_msr

MLOps Pipeline for Music Recommendation - Spotify playlist continuation

Language: Python - Size: 25.5 MB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 0 - Forks: 0

sergio11/lyric_wave_architecture

🎵 LyricWave - Your AI Music Composer 🎶 Compose Unique MP4 Songs Effortlessly! LyricWave uses AI to create personalized music by harmonizing lyrics with captivating melodies and synthetic vocals. Unleash your musical creativity today! 🚀🎶

Language: Python - Size: 29 MB - Last synced at: about 1 month ago - Pushed at: 2 months ago - Stars: 29 - Forks: 6

Pirate-Emperor/BigData-Pipeline

BigData Pipeline is a local testing environment for experimenting with various storage solutions (RDB, HDFS), query engines (Trino), schedulers (Airflow), and ETL/ELT tools (DBT). It supports MySQL, Hadoop, Hive, Kudu, and more.

Language: Dockerfile - Size: 7.95 MB - Last synced at: about 1 month ago - Pushed at: 6 months ago - Stars: 3 - Forks: 0

AhmetFurkanDEMIR/airflow-spark-kafka-example

Airflow, Spark and Kafka example

Language: Dockerfile - Size: 532 KB - Last synced at: 29 days ago - Pushed at: over 1 year ago - Stars: 6 - Forks: 0

mikeroyal/Apache-Airflow-Guide

Apache Airflow Guide

Language: Python - Size: 279 KB - Last synced at: 7 days ago - Pushed at: about 1 year ago - Stars: 28 - Forks: 12

choval/devairflow

Simple local development airflow image

Size: 111 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 1 - Forks: 0

Ramshankar07/Stock-HMM

Streamlit application designed to forecast stock prices using a Hidden Markov Model (HMM) based on historical stock price data. The application enables users to view 1-day and multi-day forecasts for several major tech stocks, including: Apple Inc. (AAPL) Tesla Inc. (TSLA) NVIDIA Corporation (NVDA)

Language: Python - Size: 47.9 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0

nasirkadri2601/Live_Cricket_Data_Pipeline

This project captures live cricket data in raw JSON format, cleans and transforms it, and stores it in a centralized data warehouse. The data is then used for analysis, including match outcome predictions, player performance, and team strategy insights, enabling data-driven decisions.

Language: Jupyter Notebook - Size: 25.6 MB - Last synced at: about 1 month ago - Pushed at: 5 months ago - Stars: 0 - Forks: 0

gustschaefer/Twitter-Batch-ETL

ETL que extraí diariamente os trending topics do Twitter em diversos países, realiza transformações com PySpark e envia os dados para o Amazon S3 utilizando Apache Airflow como orquestrador.

Language: Python - Size: 1.03 MB - Last synced at: 5 months ago - Pushed at: about 4 years ago - Stars: 4 - Forks: 0

RayanGAtech/HR-Roster-Change-Detection-Pipeline-with-Apache-Airflow-and-PostgreSQL

Automated data pipeline tailored for HR systems to process and analyze roster data. The pipeline leverages Apache Airflow to ingest and transform CSV files containing roster information, and PostgreSQL to store, deduplicate, and extract unique data points.

Language: Python - Size: 6.84 KB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 0 - Forks: 0

slatawa/Airflow-Monitoring-ElasticSearch-LogStash-Kibana

In this project we look to set up Airflow monitoring using ElasticSearch-LogStash-Kibana (ELK stack). We will set up the stack using docker images.

Language: Shell - Size: 3.31 MB - Last synced at: about 1 month ago - Pushed at: almost 4 years ago - Stars: 3 - Forks: 3

finloop/grafana-airflow-etl

End to end data pipeline with Apache Airflow and Grafana

Language: Python - Size: 41.9 MB - Last synced at: 1 day ago - Pushed at: about 3 years ago - Stars: 1 - Forks: 2

judeleonard/Data-Engineering

This is a repo to keep track of all my projects related to data engineering

Language: Python - Size: 24 MB - Last synced at: 6 months ago - Pushed at: 6 months ago - Stars: 2 - Forks: 2

ALEXUSCR-27/Amazon-Data-ETL

ETL pipeline for Amazon product sales data, using Apache Airflow for data orchestration and Supabase for storage, by containerizing the environment with Docker, the setup is scalable and easily deployable, supporting data-driven decision-making.

Language: Python - Size: 4.49 MB - Last synced at: about 1 month ago - Pushed at: 6 months ago - Stars: 2 - Forks: 1

wlopezm-unal/reddit_project_airflow_aws

This project focuses on implementing an ETL pipeline using Apache Airflow to efficiently extract data from Reddit, transform it as needed, and load it into an AWS S3 bucket. The use of Airflow allows for robust orchestration of the data workflow, ensuring that each step of the ETL process is executed in a reliable and repeatable manner.

Language: Python - Size: 38.1 KB - Last synced at: about 1 month ago - Pushed at: 7 months ago - Stars: 0 - Forks: 0

TuanAnh30/ETL_With_AirFlow

Simulate loading log data in JSON files into a database using an ETL process with Airflow to automate execution.

Language: Python - Size: 25.4 MB - Last synced at: 7 months ago - Pushed at: 7 months ago - Stars: 0 - Forks: 0

jasontrandev/ETL-Data-Pipeline-Airflow-Project1

Build an ETL Data Pipelines that uses Airflow DAGs to extract employees’ data from PostgreSQL Schemas, load it into AWS Data Lake, then transform it with Python script, and finally load it into Snowflake Data warehouse using SCD type 2.

Language: Python - Size: 3.74 MB - Last synced at: 8 months ago - Pushed at: 8 months ago - Stars: 0 - Forks: 0

data-burst/airflow-git-sync

Sync DAG changes from Git to Airflow

Size: 122 KB - Last synced at: 8 months ago - Pushed at: 8 months ago - Stars: 43 - Forks: 7

mashalk1/airflow-using-docker-compose

A configuration file to run Apache Airflow 2.0 via Docker. The attached file is a Docker Compose file which is much easier to run. All you have to do is install the file, go to that directory and run "docker-compose up" and the rest will done by docker itself.

Size: 3.91 KB - Last synced at: about 1 month ago - Pushed at: 9 months ago - Stars: 0 - Forks: 0

joshua-dada-mayowa/Data-Engineer-ETL-Project

ETL Project using Airflow, DBT, Bigquery,Postgres,Docker and Docker Compose

Language: Python - Size: 41.3 MB - Last synced at: 9 months ago - Pushed at: 9 months ago - Stars: 0 - Forks: 0

ByteMeDirk/apache-airflow-data-engineer

Apache Airflow For Data Engineers Tutorial

Language: Python - Size: 6.69 MB - Last synced at: 2 months ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 1

igorlangoni/online_retail_data_pipeline

An end-to-end pipeline that ingests raw data from CSV files through Airflow DAGS into BigQuery. From there, it uses dbt to normalize and clean the data and afterwards to make the transformations and come up with relevan reports.

Language: Python - Size: 15.4 MB - Last synced at: 10 months ago - Pushed at: 10 months ago - Stars: 0 - Forks: 0

RafaelQSantos-RQS/windturbine-airflow-project

Este projeto foi desenvolvido no curso de Airflow da Udemy, lecionado pelo professor Fernando Amaral.

Language: Python - Size: 186 KB - Last synced at: 10 months ago - Pushed at: 10 months ago - Stars: 0 - Forks: 0

djamier/docker-airflow

Orchestrate data pipeline using airflow

Language: Python - Size: 43.9 KB - Last synced at: 10 months ago - Pushed at: 10 months ago - Stars: 0 - Forks: 0

aohus/etl-pipeline-with-modeling

Docker를 사용하여 Hadoop 생태계의 구성 요소와 기타 필수 서비스를 컨테이너화하여 강력한 데이터 엔지니어링 환경을 설정하는 방법을 보여줍니다. 설정에는 Hadoop (HDFS, YARN), Apache Hive, PostgreSQL 및 Apache Airflow가 포함되며, 이들 모두가 원활하게 작동하도록 구성되어 있습니다.

Language: Shell - Size: 20.4 MB - Last synced at: 12 months ago - Pushed at: 12 months ago - Stars: 0 - Forks: 0

abhioncbr/docker-airflow

Repo for building docker based airflow image. Containers support multiple features like writing logs to local or S3 folder and Initializing GCP while container booting. https://abhioncbr.github.io/docker-airflow/

Language: Python - Size: 1.04 MB - Last synced at: about 1 month ago - Pushed at: almost 6 years ago - Stars: 32 - Forks: 5

mikestack15/orangutan-stem

An open-source project dedicated to constructing robust data pipelines and scalable software infrastructure. We leverage industry-standard tools favored by developers to enhance efficiency and reliability. Uniquely, these pipelines are field-tested on farms across Sumatra, Indonesia, ensuring real-world applicability and resilience.

Language: Python - Size: 151 MB - Last synced at: 12 months ago - Pushed at: 12 months ago - Stars: 29 - Forks: 2

kaushil24/llm-finetune-pipeline

Airflow pipeline to finetune LLM on Kubernetes

Language: Python - Size: 5.8 MB - Last synced at: 12 months ago - Pushed at: 12 months ago - Stars: 2 - Forks: 0

Chinaskidev/ETL-Clima-ElSalvador

MLOps, haciendo un ETL sencillo usando Docker y Airflow y Google Cloud

Language: Python - Size: 50.8 KB - Last synced at: about 1 month ago - Pushed at: about 1 year ago - Stars: 3 - Forks: 1

abhishekbhakat/Turbine

Simple python script for easy local airflow deployment with docker. Packed with additional components. Will be adding more going forward.

Language: Python - Size: 113 KB - Last synced at: 1 day ago - Pushed at: 7 months ago - Stars: 3 - Forks: 0

vjgpt/twitter-pipeline

In this project, you will be building a Twitter Scheduler using Apache Airflow on Docker.

Language: Python - Size: 26.4 KB - Last synced at: about 1 month ago - Pushed at: over 4 years ago - Stars: 10 - Forks: 7

anilkulkarni87/airflow-docker

This is my Apache Airflow Local development setup on Windows 10 WSL2/Mac using docker-compose. It will also include some sample DAGs and workflows.

Language: Python - Size: 106 KB - Last synced at: about 1 year ago - Pushed at: over 1 year ago - Stars: 22 - Forks: 10

BaliDataMan/airflow-Tut

Airflow Tutorial to learn practical implementation with basics Concepts, Architecture, Working & Configuration practises.

Language: Shell - Size: 425 KB - Last synced at: about 1 year ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0

jesslossa/airflow-docker-localexecutor

Implements Airflow through docker on localhost (LocalExecutor), using docker-desktop.

Language: Python - Size: 171 KB - Last synced at: about 1 year ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0

stevehoober254/ETL_Data_Pipeline_For_Retail_Store

ETL (Extract, Transform, Load) pipeline to integrate sales data from various sources into a central data warehouse

Language: Python - Size: 5.86 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 0

shiv-rna/Airflow-Basics

Welcome to my Apache Airflow learning journey repository! 🚀 This repository serves as a comprehensive documentation of my exploration and understanding of Apache Airflow, an open-source platform for orchestrating complex workflows.

Language: Python - Size: 65.8 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

coder2j/airflow-docker

Source code of the Apache Airflow Tutorial for Beginners on YouTube Channel Coder2j (https://www.youtube.com/c/coder2j)

Language: Python - Size: 342 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 205 - Forks: 191

BruceDone/airflow-docker

Docker Apache Airflow

Language: Shell - Size: 9.77 KB - Last synced at: about 2 months ago - Pushed at: almost 8 years ago - Stars: 1 - Forks: 1

AlexanderM-T/AirFlow-Training

En este repositorio se encuentran algunos dags realizados siguiendo el entrenamiento de data engineer con el fin de aprender y practicar airflow

Language: Python - Size: 18.6 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

dorokhin/apache_airflow_docker

Apache airflow packed in docker compose

Language: Python - Size: 6.84 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

IanDanielM/BuyRentScrap-LuxAirflow

Automating Data Scrapers With Python and Airflow

Language: Python - Size: 5.86 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

Serge-Ntumba/stream_data_with_kafka_docker_airflow_spark

A complete data pipeline, from data extraction to storage, using a combination of tools for specific purposes: Python for data retrieval from API, Airflow for scheduling task, Kafka for data streaming, Spark for data processing, and Cassandra for data storage.

Language: Python - Size: 298 KB - Last synced at: about 1 year ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 0

Serge-Ntumba/Daily_automated_ETL

This project focuses on the seamless integration of data from a CSV file into a PostgreSQL database using Python. Additionally, Airflow is employed to perform a daily extraction of this data, transferring it to an S3 Bucket for further analysis and storage.

Language: Python - Size: 228 KB - Last synced at: about 1 year ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 0

knowsuchagency/docker-compose-airflow

An airflow deployment configuration with sane defaults

Language: Python - Size: 47.9 KB - Last synced at: about 19 hours ago - Pushed at: almost 6 years ago - Stars: 10 - Forks: 2

AakashDeorukhkar/Airflow_NYC_Taxi_Data

Ingesting NYC Yellow Taxi Data to GCP using Airflow running on Docker

Language: Python - Size: 4.08 MB - Last synced at: 10 months ago - Pushed at: almost 3 years ago - Stars: 1 - Forks: 0

aimlnerd/data-pipelines-with-airflow

Airflow DAG tutorial with docker compose local setup

Language: Python - Size: 16.6 KB - Last synced at: over 1 year ago - Pushed at: over 2 years ago - Stars: 3 - Forks: 0

Booss3my/Datascraper

Automated Indeed Job Offer Scraper: Airflow Orchestrated and Scheduled, Data Loaded into PostgreSQL Database

Language: Python - Size: 1.45 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

namcxn/airflow-chart

Helm chart airlfow with kubernetes executor provisioned on Kubernetes (GKE, EKS, AKS, ... )

Size: 0 Bytes - Last synced at: over 1 year ago - Pushed at: about 5 years ago - Stars: 0 - Forks: 0

Sheitak/datalake-jljq

Data Lake project for ingest and transform financial data and dashboard BI proposal

Language: Python - Size: 41 KB - Last synced at: 2 months ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

martinsejas/FlightPricePrediction

An end-to-end MLOps Pipeline, from data ingestion to inference. Powered by Azure, FastAPI, Apache Airflow, Streamlit and Sklearn.

Language: Jupyter Notebook - Size: 3.87 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

Penosh22/Market-news-and-forex-exchange-ETL-pipeline

ETL pipeline to extract webscraped forex data and google news library data on daily basis and storing it in a postgres database for market analysis and insights

Language: Python - Size: 1.95 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

CanaanGM/databases-infrastructure

on demand databases deployment, varuios kinds, adding more as i use them!

Language: Python - Size: 4.02 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 0

ranguiquan/airflow-compose

A template for Airflow docker operator with docker compose

Language: Python - Size: 12.7 KB - Last synced at: over 1 year ago - Pushed at: about 2 years ago - Stars: 0 - Forks: 0

SoukainaBoufousse/ETL_Sales_Data

This repository demonstrates a data engineering ETL (Extract, Transform, Load) pipeline for processing sales data. It utilizes Docker and Apache Airflow for orchestration and PostgreSQL for data storage. The pipeline extracts, transforms, and loads sales data.

Language: Python - Size: 385 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

niyotham/Data-engineer-end-to-end-project-airflow-dbt-soda-bigquery

An end to end data engineering project for loading data into bigquery with airflow and perform transformations using dbt and data quality check with soday

Language: Python - Size: 7.03 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

mpavanetti/airflow

This set of code and instructions has the porpouse to instanciate a compiled environment with set of docker images like airflow webserver, airflow scheduler, postgresql, pyspark, Data Pipeline consuming data from weather api , processing with pyspark and storing in postgresql

Language: PHP - Size: 1.62 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 8 - Forks: 3

sreekesh93/airflow_beginners

to start airflow in local with basic setup

Language: Python - Size: 2.93 KB - Last synced at: over 1 year ago - Pushed at: over 3 years ago - Stars: 4 - Forks: 0

ndangalasinh/Toronto-Apartment-Evaluation-Analysis

Data Engineering End-to-End Project

Language: Python - Size: 2.87 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

prasadanilmore/ETL-JustEat-TestCase

This documentation provides an overview of the tasks completed in this project. The project comprises three key tasks: ETL (Extract, Transform, Load), API development, and Data Orchestration using Airflow. Each task is detailed below along with explanations of design choices and considerations.

Language: Python - Size: 79.1 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

sunghj1118/airflow

In this repository I study Airflow and related topics/implementations.

Language: Python - Size: 539 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

rahul765/Machine-Learning-Pipelines

From data gathering to model deployment. Complete ML pipeline using Docker, Airflow and Python.

Language: Python - Size: 335 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 12 - Forks: 3

BigCheeze45/airflow-docker

Ready to go Apache Airflow stack for docker

Language: Shell - Size: 33.2 KB - Last synced at: over 1 year ago - Pushed at: over 4 years ago - Stars: 0 - Forks: 0

dataopstix/modelt

Modelt(mow·delt) is a modern data integration solution that connects data to data for advanced analytics.

Language: Shell - Size: 463 KB - Last synced at: about 2 months ago - Pushed at: almost 3 years ago - Stars: 2 - Forks: 1

scalactic/airflow-example

airflow-example in docker

Language: Python - Size: 7.81 KB - Last synced at: over 1 year ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0

judeleonard/e-commerce_activity_tracking

This is an ELT data pipeline setup to track the activities of an e-commerce website based on orders, reviews, deliveries and shipment date. This project utilized technologies like Airflow, AWS RDS-Postgres, Python etc.

Language: Python - Size: 596 KB - Last synced at: over 1 year ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0

shlin168/airflow-local-playground

docker compose files and some example DAGs for playing and learning airflow

Language: Python - Size: 148 KB - Last synced at: over 1 year ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0

rikiafifuddin/airflow-docker

Airflow Docker with pip instalation using requirement.txt and using DAG from cloned repository

Language: Dockerfile - Size: 16.6 KB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0

vedraiyani/airflow-win

Minimal setup for apache airflow docker

Language: Python - Size: 2.93 KB - Last synced at: almost 2 years ago - Pushed at: almost 3 years ago - Stars: 0 - Forks: 0

avocadojj/marketing

Making pipeline for bank marketing campaign

Language: Jupyter Notebook - Size: 212 MB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 2

lafirm/gold-price-data-engg

Get Current Gold Rate Periodically and Store the data using Airflow (Taskflow API) using Docker

Language: Python - Size: 203 KB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0

andyherko/airflow_docker_sample

Playing around with Airflow

Language: Python - Size: 22.5 KB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0

AqilaFadia/Data_Engineer-Portfolio

This is My Example Project Data Engineer

Language: Python - Size: 2.93 KB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0

muhk01/airflow_bmkg_data

Airflow ETL for Earthquake Data

Language: Python - Size: 235 KB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0

Gakhramanzode/Airflow

Bash-скрипт по установке Airflow

Language: Shell - Size: 14.6 KB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 1 - Forks: 1

anastasiia-p/airflow-ml

Airflow Pipeline for Machine Learning

Language: Python - Size: 9.77 KB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 1 - Forks: 0

judeleonard/Prescriber-ETL-data-pipeline

An End-to-End ETL data pipeline that leverages pyspark parallel processing to process about 25 million rows of data coming from a SaaS application using Apache Airflow as an orchestration tool and various data warehouse technologies and finally using Apache Superset to connect to DWH for generating BI dashboards for weekly reports

Language: Python - Size: 7.42 MB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 7 - Forks: 1

julientoucoula17/apache_airflow-with-Docker

Apache Airflow installation with Docker 🌬️

Language: Python - Size: 10.7 KB - Last synced at: 3 months ago - Pushed at: about 3 years ago - Stars: 1 - Forks: 0

rushabh2390/SharesETL

Shares ETL to scrape historical data of one shares and import in metabase

Language: Python - Size: 70.3 KB - Last synced at: almost 2 years ago - Pushed at: over 2 years ago - Stars: 4 - Forks: 0

vikrantdeshpande09876/Credit_Card_Fraud_Detection

Deploying Classification Model on GCP for detecting anomalous behavior in credit-card transactions. A crucial aspect for credit-card companies to recognize fraudulent activity and not charge customers for items they didn't purchase. A subtle line needs to be drawn and my main focus here is to reduce blocked-transactions of legitimate customers.

Language: Jupyter Notebook - Size: 41.7 MB - Last synced at: about 8 hours ago - Pushed at: about 2 years ago - Stars: 2 - Forks: 1