Ecosyste.ms: Repos

An open API service providing repository metadata for many open source software ecosystems.

GitHub / tosun-si / teams-league-airflow-elt

The goal of this article is showing a real world use case for ELT batch pipeline, with Cloud Storage, BigQuery, Apache Airflow and Cloud Composer : The Extract part is managed in Cloud Storage The Load part is managed from Cloud Storage to BigQuery The Transform part is managed by a BigQuery SQL query Everything is orchestrated by Airflow

JSON API: https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tosun-si%2Fteams-league-airflow-elt

Stars: 0
Forks: 0
Open Issues: 0

License: None
Language: Python
Repo Size: 311 KB
Dependencies: 205

Created: about 1 year ago
Updated: 10 months ago
Last pushed: 10 months ago
Last synced: 10 months ago

Topics: airflow, batch, bigquery, cloud-composer, cloud-storage, dag, elt, orchestration, pipeline, python

Files
    Loading...
    Readme
    Loading...
    Dependencies
    Pipfile pypi
    dags_setup/setup.py pypi
    Pipfile.lock pypi