Ecosyste.ms: Repos
An open API service providing repository metadata for many open source software ecosystems.
GitHub topics: data-engineering-workflows
waqarg2001/Youtube-Data-Pipeline-AWS
Leveraging AWS Cloud Services, an ETL pipeline transforms YouTube video statistics data. Data is downloaded from Kaggle, uploaded to an S3 bucket, and cataloged using AWS Glue for querying with Athena. AWS Lambda and Glue converts to Parquet format and stores it in a cleansed S3 bucket. AWS QuickSight then visualizes the materialised data.
Language: Python - Size: 2.89 MB - Last synced: about 2 hours ago - Pushed: 1 day ago - Stars: 0 - Forks: 0
cybergeekgyan/Data-Engineering-Portfolio
Data Engineering portfolio projects, resources used to study data tools...
Language: Jupyter Notebook - Size: 2.92 MB - Last synced: about 1 month ago - Pushed: 2 months ago - Stars: 1 - Forks: 0
socialpoint-labs/sqlbucket 📦
Lightweight library to write, orchestrate and test your SQL ETL. Writing ETL with data integrity in mind.
Language: Python - Size: 463 KB - Last synced: 29 days ago - Pushed: 5 months ago - Stars: 71 - Forks: 7
borbert/Data_Engineering_Nanodegree
This repository is the collection point for all of the projects completed during the Udacity Data Engineering Nano Degree program.
Language: Jupyter Notebook - Size: 44.8 MB - Last synced: 7 months ago - Pushed: about 3 years ago - Stars: 0 - Forks: 1
marianajo/beam-examples
Examples that I use to learn and show Apache Beam
Language: Python - Size: 14.6 KB - Last synced: 11 months ago - Pushed: over 5 years ago - Stars: 2 - Forks: 0
junipertcy/networkie
A first course for data engineering workflows
Language: Python - Size: 2.35 MB - Last synced: over 1 year ago - Pushed: about 6 years ago - Stars: 1 - Forks: 52
