Ecosyste.ms: Repos
An open API service providing repository metadata for many open source software ecosystems.
GitHub / mathewsrc / ETL-Chicago-Cafe-Permits
This ETL (Extract, Transform, Load) project employs several Python libraries, including Airflow, Soda, Polars, YData Profiling, DuckDB, Requests, Loguru, and Google Cloud to streamline the extraction, transformation, and loading of CSV datasets from the U.S. government's data repository at https://catalog.data.gov.
JSON API: https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mathewsrc%2FETL-Chicago-Cafe-Permits
Stars: 3
Forks: 0
Open Issues: 0
License: mit
Language: HTML
Repo Size: 42.3 MB
Dependencies:
23
Created: 8 months ago
Updated: 2 months ago
Last pushed: 6 months ago
Last synced: 24 days ago
Commit Stats
Commits: 98
Authors: 2
Mean commits per author: 49.0
Development Distribution Score: 0.327
More commit stats: https://commits.ecosyste.ms/hosts/GitHub/repositories/mathewsrc/ETL-Chicago-Cafe-Permits
Topics: airflow, astro, astro-python-sdk, bigquery, continuous-integration, data-quality-checks, data-visualization, docker, duckdb, extract-transform-load, github-actions, looker-studio, polars, python, soda, ydata-profiling
Files
Dependencies
- actions/checkout v3 composite
- actions/setup-python v3 composite
- quay.io/astronomer/astro-runtime 9.2.0 build
- airflow-provider-duckdb ==0.2.0
- apache-airflow-providers-apache-spark ==4.2.0
- apache-airflow-providers-common-sql ==1.8.0
- apache-airflow-providers-docker ==3.8.0
- apache-airflow-providers-http ==4.6.0
- apache-airflow-providers-sqlite ==3.5.0
- astro-sdk-python ==1.7.0
- beautifulsoup4 ==4.12.2
- furl ==2.1.3
- loguru ==0.7.2
- pendulum ==2.1.2
- polars ==0.19.12
- pytest ==7.4.2
- pytest-cov ==4.1.0
- python-dotenv ==0.21.0
- requests ==2.31.0
- ruff ==0.0.292
- soda-core-bigquery ==3.0.45
- soda-core-duckdb ==3.0.45
- ydata-profiling ==4.6.1