alanchn31/Data-Engineering-Projects

Personal Data Engineering Projects

Language: Jupyter Notebook - Size: 2.92 MB - Last synced at: about 5 hours ago - Pushed at: over 2 years ago - Stars: 934 - Forks: 206

tokern/piicatcher

Scan databases and data warehouses for PII data. Tag tables and columns in data catalogs like Amundsen and Datahub

Language: Python - Size: 1.38 MB - Last synced at: about 4 hours ago - Pushed at: over 1 year ago - Stars: 310 - Forks: 99

aws/amazon-redshift-python-driver

Redshift Python Connector. It supports Python Database API Specification v2.0.

Language: Python - Size: 902 KB - Last synced at: 2 days ago - Pushed at: 10 days ago - Stars: 212 - Forks: 76

alanchn31/Movalytics-Data-Warehouse

Data pipeline performing ETL to AWS Redshift using Spark, orchestrated with Apache Airflow

Language: Python - Size: 717 KB - Last synced at: 5 months ago - Pushed at: almost 5 years ago - Stars: 133 - Forks: 31

Wittline/uber-expenses-tracking

The goal of this project is to track the expenses of Uber Rides and Uber Eats through data Engineering processes using technologies such as Apache Airflow, AWS Redshift and Power BI.

Language: Jupyter Notebook - Size: 28.2 MB - Last synced at: about 1 month ago - Pushed at: almost 3 years ago - Stars: 121 - Forks: 36

shravan-kuchkula/udacity-data-eng-proj-1

Developed a data pipeline to automate data warehouse ETL by building custom airflow operators that handle the extraction, transformation, validation and loading of data from S3 -> Redshift -> S3

Language: Python - Size: 3.47 MB - Last synced at: over 1 year ago - Pushed at: over 3 years ago - Stars: 88 - Forks: 58

aws-solutions/clickstream-analytics-on-aws

Clickstream Analytics on AWS source code

Language: TypeScript - Size: 72.4 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 77 - Forks: 26

KentHsu/Udacity-Data-Engineering-Nanodgree

Udacity Data Engineering Nanodegree Program

Language: Jupyter Notebook - Size: 2.12 MB - Last synced at: about 1 month ago - Pushed at: about 4 years ago - Stars: 52 - Forks: 59

heroku-examples/analytics-with-kafka-redshift-metabase

An example system that captures a large stream of product usage data, or events, and provides both real-time data visualization and SQL-based data analytics.

Language: JavaScript - Size: 9.4 MB - Last synced at: 16 days ago - Pushed at: over 2 years ago - Stars: 26 - Forks: 11

moritzkoerber/covid-19-data-engineering-pipeline

A Covid-19 data pipeline on AWS featuring PySpark/Glue, Docker, Great Expectations, Airflow, and Redshift, templated in CloudFormation and CDK, deployable via Github Actions.

Language: Python - Size: 1.31 MB - Last synced at: 19 days ago - Pushed at: over 1 year ago - Stars: 23 - Forks: 5

jackmleitch/StravaDataPipline

:arrows_counterclockwise: :running: EtLT of my own Strava data using the Strava API, MySQL, Python, S3, Redshift, and Airflow

Language: Python - Size: 1.16 MB - Last synced at: about 2 years ago - Pushed at: almost 3 years ago - Stars: 21 - Forks: 2

LoveNui/DataEngineering-Capstone-Project

Language: Jupyter Notebook - Size: 11.7 MB - Last synced at: about 1 month ago - Pushed at: almost 2 years ago - Stars: 17 - Forks: 1

ismaildawoodjee/aws-data-pipeline

A batch processing data pipeline, using AWS resources (S3, EMR, Redshift, EC2, IAM), provisioned via Terraform, and orchestrated from locally hosted Airflow containers. The end product is a Superset dashboard and a Postgres database, hosted on an EC2 instance at this address (powered down):

Language: Python - Size: 4.77 MB - Last synced at: about 1 year ago - Pushed at: about 3 years ago - Stars: 17 - Forks: 6

tmheo/spring-data-jpa-redshift-sample

spring boot data jpa integration with aws redshift sample

Language: Java - Size: 1.66 MB - Last synced at: about 1 year ago - Pushed at: over 9 years ago - Stars: 15 - Forks: 9

kishlayjeet/Zomato-Twitter-Sentiment-Analysis-Data-Pipeline

This project provides valuable customer sentiment insights for Zomato by tracking and analyzing tweets related to their brand and services.

Language: Python - Size: 22.5 KB - Last synced at: about 1 month ago - Pushed at: over 1 year ago - Stars: 13 - Forks: 2

AWS-Big-Data-Projects/Analysing-Census-Data-using-aws

Use aws-emr and aws-redshift to analyse dataset of adult census of USA

Size: 638 KB - Last synced at: 5 days ago - Pushed at: over 4 years ago - Stars: 13 - Forks: 0

vsouza/spark-kinesis-redshift

Example project for consuming AWS Kinesis streamming and save data on Amazon Redshift using Apache Spark

Language: Python - Size: 89.8 KB - Last synced at: 3 days ago - Pushed at: almost 7 years ago - Stars: 11 - Forks: 6

kishaningithub/rdapp

rdapp - Redshift Data API Postgres Proxy

Language: Go - Size: 501 KB - Last synced at: about 1 month ago - Pushed at: almost 2 years ago - Stars: 10 - Forks: 0

lenguyenthedat/aws-redshift-to-rds

A simple command-line tool to copy tables from Amazon Redshift to Amazon RDS (PostgreSQL).

Language: Haskell - Size: 16.6 KB - Last synced at: about 1 month ago - Pushed at: about 8 years ago - Stars: 10 - Forks: 5

essraahmed/Data-Warehouse-With-Redshift

Data Warehouse with AWS Redshift and Visualizing data using Power BI

Language: Jupyter Notebook - Size: 618 KB - Last synced at: about 1 year ago - Pushed at: over 2 years ago - Stars: 8 - Forks: 1

taise/Spectrometer 📦

AWS Redshift monitoring web console

Language: Slim - Size: 587 KB - Last synced at: about 1 year ago - Pushed at: over 4 years ago - Stars: 7 - Forks: 2

twistedFantasy/aws

The goal of this repository is to provide good and clear examples of Amazon CLI commands together with Amazon CDK to easily create any AWS services and resources

Language: Python - Size: 43.9 KB - Last synced at: 8 days ago - Pushed at: over 5 years ago - Stars: 7 - Forks: 0

FedericoSerini/DEND-Project-3-Data-Warehouse-AWS

Project 3 - Data Engineering Nanodegree

Language: Python - Size: 62.5 KB - Last synced at: 24 days ago - Pushed at: about 6 years ago - Stars: 4 - Forks: 10

FedericoSerini/DEND-Project-5-Data-Pipelines

Project 5 - Data Engineering Nanodegree

Language: Python - Size: 4.88 KB - Last synced at: 24 days ago - Pushed at: almost 6 years ago - Stars: 3 - Forks: 4

lregnier/slick-amazon-redshift

A quick example of how to load data from Amazon S3 into Amazon Redshift using Redshift's COPY command through Slick

Language: Scala - Size: 6.84 KB - Last synced at: about 1 year ago - Pushed at: over 6 years ago - Stars: 3 - Forks: 1

eduardofb/redshift-create-manifest

Redshift script to create a MANIFEST file recursively

Language: Python - Size: 16.6 KB - Last synced at: about 1 year ago - Pushed at: almost 8 years ago - Stars: 3 - Forks: 1

mikecerton/The-Retail-ELT-Pipeline-End-To-End

This project designs and implements an ETL pipeline using Apache Airflow (Docker Compose) to ingest, process, and store retail data. AWS S3 acts as the data lake, AWS Redshift as the data warehouse, and Looker Studio for visualization. [Data Engineer]

Language: Python - Size: 1.07 MB - Last synced at: 19 days ago - Pushed at: 19 days ago - Stars: 2 - Forks: 0

Hanan-Nawaz/FlightTragedyAnalysis

Flight Tragedy Analysis is a comprehensive data analysis project focused on examining aviation accidents and incidents from 1905 to 2009. This project provides users with valuable insights into historical plane crashes and their associated data.

Language: Jupyter Notebook - Size: 2.06 MB - Last synced at: about 2 months ago - Pushed at: over 1 year ago - Stars: 2 - Forks: 0

giulic3/data-engineering-nanodegree

Projects realized for the Data Engineering Nanodegree offered by Udacity https://www.udacity.com/course/data-engineer-nanodegree--nd027

Language: Jupyter Notebook - Size: 6.43 MB - Last synced at: 2 months ago - Pushed at: over 2 years ago - Stars: 2 - Forks: 1

polo2444172276/Udacity-Data-Engineering-Nanodegree

Completed Udacity's data engineering nano degree. Went through a series of exercises and projects to learn and practice the trendy big data management tools.

Language: PLpgSQL - Size: 28.7 MB - Last synced at: over 1 year ago - Pushed at: almost 3 years ago - Stars: 2 - Forks: 3

polarbeargo/Udacity-nd027-Data-Warehouse

Language: Jupyter Notebook - Size: 392 KB - Last synced at: about 1 year ago - Pushed at: almost 4 years ago - Stars: 2 - Forks: 3

meejahnsnutshell/AWS_ML_Crypto

A Java API that gathers historical cryptocurrency pricing data (via CryptoCompare API) & makes predictions (via AWS Machine Learning API)

Language: Java - Size: 212 KB - Last synced at: 3 months ago - Pushed at: about 7 years ago - Stars: 2 - Forks: 0

PopoPenguin/AWS_ML_Crypto

A Java API that gathers historical cryptocurrency pricing data (via CryptoCompare API) & makes predictions (via AWS Machine Learning API)

Language: Java - Size: 162 KB - Last synced at: about 1 year ago - Pushed at: over 7 years ago - Stars: 2 - Forks: 0

eduardofb/redshift-remove-duplicates

Remove duplicates entries from a Redshift cluster

Language: Python - Size: 15.6 KB - Last synced at: about 1 year ago - Pushed at: about 8 years ago - Stars: 2 - Forks: 0

aws-samples/zero-etl-architecture-patterns

Zero-ETL integrations - Enable near real-time analytics on petabytes of transactional data

Language: Python - Size: 144 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 1 - Forks: 0

trantuanngoc/us_immigration_data_engineering

US immigration data engineering : ETL pipeline, data modeling and warehousing of US immigration data

Language: HCL - Size: 3.58 MB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 1 - Forks: 0

MaxineXiong/Cloud-Data-Warehousing-with-AWS-Redshift

This project builds a cloud-based ETL pipeline for Sparkify to move data to a cloud data warehouse. It extracts song and user activity data from AWS S3, stages it in Redshift, and transforms it into a star-schema data model with fact and dimension tables, enabling efficient querying to answer business questions.

Language: Jupyter Notebook - Size: 20.5 KB - Last synced at: about 2 months ago - Pushed at: 8 months ago - Stars: 1 - Forks: 0

evanmathew/Reddit_ETL_DE

This project demonstrates a complete data pipeline for extracting, transforming, and loading (ETL) Reddit data into an Amazon Redshift data warehouse. The pipeline uses various AWS services and tools including Apache Airflow, PostgreSQL, AWS S3, AWS Glue, AWS Athena, and Amazon Redshift. The project is orchestrated using Docker and Apache Airflow

Language: Python - Size: 137 KB - Last synced at: 3 months ago - Pushed at: 10 months ago - Stars: 1 - Forks: 0

DimaKuriptya/RedditETL

This project provides a comprehensive data pipeline solution to extract, transform, and load (ETL) Reddit data into a Redshift data warehouse. The pipeline leverages a combination of tools and services including Apache Airflow, Celery, PostgreSQL, Amazon S3, AWS Glue, Amazon Athena, and Amazon Redshift.

Language: Python - Size: 14.6 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 0

Huyen-P/DE_DWH_AWS_S3_RedShift

building etl pipelines to migrate music json data/ metadata files (semi-structured data) into a relational database stored in AWS Redshift cluster

Language: Python - Size: 20.4 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 0

danrbueno/airflow_aws_justwatch_pipeline

Data pipeline using Airflow, GraphQL, AWS S3, AWS Glue Jobs and AWS Redshift

Language: Python - Size: 11.5 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 0

abhinavkumariem/Python-AWS-Redshift

load local files to AWS Redshift using Python and Unleash Insights with Power BI

Language: Jupyter Notebook - Size: 1.91 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 0

SRVivek1/pyspark-rdd-dataframe-examples

PySpark RDD and DataFrame Examples

Language: Python - Size: 113 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 0

kingyiusuen/udacity-data-engineering-nanodegree

Projects for Udacity's Data Engineering Nanodegree

Language: Jupyter Notebook - Size: 1.17 MB - Last synced at: 2 months ago - Pushed at: almost 2 years ago - Stars: 1 - Forks: 0

markoshlima/positional-file-process

This project is based for legacy applications that works with positional files to process data. The objetive is read these positional files when they arrives in AWS S3, and then send to a dataware-house like AWS Redshift, and finally read the results with a Business Intelligence tool as AWS QuickSight.

Size: 873 KB - Last synced at: over 1 year ago - Pushed at: about 3 years ago - Stars: 1 - Forks: 0

lkellermann/sparkify-dw

Udacity Data Engineering Nanodegree Project #3.

Language: Python - Size: 14 MB - Last synced at: about 1 year ago - Pushed at: about 4 years ago - Stars: 1 - Forks: 0

sagardua297/udacity-data-engineering-nd

Data Pipeline Analytics Platform is an end-to-end generic Big Data pipeline. Involves following tech stack: AWS S3, AWS Redshift, AWS EMR Cluster, Apache Spark, Apache Airflow.

Language: Python - Size: 1.81 MB - Last synced at: over 1 year ago - Pushed at: over 4 years ago - Stars: 1 - Forks: 0

Mark-McAdam/Data-Engineering-Batch

Takes product reviews and performs natural language processing to provide sentiment analysis. The new insight gets combined with matching product information in the central database to provide a clearer picture of user behavior.

Language: Python - Size: 963 KB - Last synced at: about 2 years ago - Pushed at: over 4 years ago - Stars: 1 - Forks: 0

andre-marcos-perez/ifood-arch-readme

The application is the documentation of my solution for the iFood data architect test.

Size: 454 KB - Last synced at: about 1 month ago - Pushed at: over 4 years ago - Stars: 1 - Forks: 0

scriptbuzz/aws-datalake-poc-video

AWS hosted enterprise Data Lake with both batch and realtime data pipelines.

Size: 349 KB - Last synced at: about 2 years ago - Pushed at: almost 5 years ago - Stars: 1 - Forks: 0

SimplifyData/Cloud-Data-Warehouse-with-Redshift-AWS

Cloud Data Warehouse of Sparkify Data using Redshift

Language: Python - Size: 1.2 MB - Last synced at: about 2 months ago - Pushed at: almost 5 years ago - Stars: 1 - Forks: 1

rohitsingh4334/udacity-data-engineering-nanodegree

udacity nanodegree course projects.

Language: Jupyter Notebook - Size: 80 MB - Last synced at: about 2 years ago - Pushed at: almost 5 years ago - Stars: 1 - Forks: 0

unknownv2/redshift-fake-driver

A JDBC driver that emulates AWS Redshift specific commands

Language: Scala - Size: 168 KB - Last synced at: 11 months ago - Pushed at: about 6 years ago - Stars: 1 - Forks: 0

marcy-terui/catlass

Cloud Automation as Code with Cloud Automator

Language: Ruby - Size: 9.77 KB - Last synced at: 2 months ago - Pushed at: over 7 years ago - Stars: 1 - Forks: 0

paoliniluis/shift-to-spectrum 📦

An automated SQL script generator to migrate AWS Redshift schemas (or tables) to AWS Redshift Spectrum

Language: JavaScript - Size: 8.79 KB - Last synced at: about 2 years ago - Pushed at: over 7 years ago - Stars: 1 - Forks: 0

GuledIM/Super-Cafe-ETL-AWS

In this group project simulating a real-world setting, we built a scalable ETL pipeline to process daily CSV transactions into a centralized PostgreSQL database. We used Docker, Grafana for visualization, and later implemented AWS cloud services to deploy a scalable, cloud-based ETL system.

Language: Python - Size: 2.93 MB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 0 - Forks: 0

exasol/redshift-virtual-schema

Virtual Schema for connecting Redshift as a data source to Exasol

Language: Java - Size: 74.2 KB - Last synced at: 12 days ago - Pushed at: 12 days ago - Stars: 0 - Forks: 1

jibbs1703/Tickit-Data-Pipeline

This repository demonstrates the creation of a robust data pipeline using an Orchestrator, on-prem and cloud resources. It collects data from on-premises SQL and NoSQL database and loads it into a SQL database in the cloud.

Language: Python - Size: 50.8 KB - Last synced at: 20 days ago - Pushed at: 20 days ago - Stars: 0 - Forks: 0

Naga-Manohar-Y/Food-Delivery-Analysis-in-Real-Time

This project builds a real-time food delivery analytics pipeline using AWS Kinesis, PySpark, Redshift, and QuickSight, with automated deployments via CodeBuild.

Language: Python - Size: 970 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 0 - Forks: 0

jibbs1703/AWSResourceManager

This repository contains the python modules and packages make up the AWS Resource Manager, a custom python package/wheel designed to simplify the management of AWS services through custom-written use cases and utilities. This repository serves to reinforce my knowledge on building python packages and wheels.

Language: Python - Size: 35.2 KB - Last synced at: 2 months ago - Pushed at: 2 months ago - Stars: 0 - Forks: 0

BhawnaMehbubani/Kafka-Spark-Redshift-Streaming-Data-Ingestion-Project

This project is a real-time data pipeline designed for ingesting, processing, and storing telecom call records. It integrates Apache Kafka, Apache Spark Streaming, and AWS Redshift to handle large volumes of streaming data in near real-time. The pipeline is containerized with Docker Compose, enabling easy deployment, scalability, and modularity.

Language: Python - Size: 952 KB - Last synced at: about 2 months ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0

jibbs1703/Weather-Gas-ETL-Pipeline

This repository contains a in ETL pipeline for collecting, transforming and storing hourly weather and atmospheric gas data. The pipeline leverages Docker containerization, AWS cloud infrastructure resources and is orchestrated using Apache-Airflow.

Language: Python - Size: 17.6 KB - Last synced at: about 2 months ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0

VvEK-Hiremath/AWS-RedshiftCode

Learning Redshift

Language: Shell - Size: 8.79 KB - Last synced at: 7 months ago - Pushed at: 7 months ago - Stars: 0 - Forks: 0

VvEK-Hiremath/Airlines-Data-Pipeline-Project-AWS

Implementing data pipeline using AWS services for airlines data

Language: Python - Size: 195 KB - Last synced at: 7 months ago - Pushed at: 7 months ago - Stars: 0 - Forks: 0

dondogecl/cool_data_pipeline

Data pipeline from RDBMS to AWS

Language: Python - Size: 41 KB - Last synced at: 8 months ago - Pushed at: 8 months ago - Stars: 0 - Forks: 0

desininja/Airline-Data-Ingestion-Pipeline

ETL pipeline using AWS services.

Language: Python - Size: 4.33 MB - Last synced at: 3 months ago - Pushed at: 8 months ago - Stars: 0 - Forks: 0

rashmishreev/atm-analytics-bigaata-aws

Analyze over 2.5 million ATM transaction records from Spar Nord Bank to optimize ATM usage patterns and enhance customer service using AWS Services and Big Data Analytics.

Language: Jupyter Notebook - Size: 42.4 MB - Last synced at: about 2 months ago - Pushed at: 9 months ago - Stars: 0 - Forks: 0

NitinPrasad5/Flight_Data-Analysis

This project implements a data pipeline using Amazon Web Services (AWS) to process and analyze a Flight Dataset. The pipeline collects raw data, processes it, stores the processed data in a data warehouse, and performs analysis using SQL queries. The analysis results are visualized dynamically using Power BI dashboards.

Language: Python - Size: 4.09 MB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 0 - Forks: 0

DivineSamOfficial/Banking-Data-Warehouse-Pipeline

Banking Data Warehouse Pipeline

Language: Python - Size: 52.1 MB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 0 - Forks: 0

mihirkudale/youtube-analysis-data-engineering-project

This project aims to securely manage, streamline, and perform analysis on the structured and semi-structured YouTube videos data based on the video categories and the trending metrics.

Language: Python - Size: 114 KB - Last synced at: about 2 months ago - Pushed at: 12 months ago - Stars: 0 - Forks: 0

shrikantnaidu/Data-Warehousing-with-AWS

Data Warehousing with AWS

Language: Jupyter Notebook - Size: 12.7 KB - Last synced at: 3 months ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

MartinKalema/mysql-kafka-s3-redshift-data-pipeline

ETL pipeline

Language: Python - Size: 2.29 MB - Last synced at: 3 months ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

eljandoubi/Airflow-data-pipeline

Airflow data pipeline

Language: Python - Size: 5.86 KB - Last synced at: about 2 months ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

eljandoubi/AWS-Data-Warehouse

Build an ETL pipeline for a database hosted on AWS Redshift.

Language: Python - Size: 11.7 KB - Last synced at: about 2 months ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

imverma/CineETL_Movie_Insights_Data_Pipeline

A data pipeline that conducts ETL processes to AWS Redshift, utilizing Spark and coordinated by Apache Airflow.

Language: Python - Size: 20.5 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

dhvani-k/CineETL_Movie_Insights_Data_Pipeline

A data pipeline that conducts ETL processes to AWS Redshift, utilizing Spark and coordinated by Apache Airflow.

Language: Python - Size: 20.5 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

martins-jean/Event-Driven-Serverless-ETL-in-AWS

Leveraged AWS services to automate the consolidation of toll plaza transactions in a data warehouse.

Language: Python - Size: 96.7 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

milamarcan/data_pipeline_apache_airflow

Move data from AWS S3 to Redshift using Apache Airflow

Language: Python - Size: 30.3 KB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0

SaadAhmedWaqar/Data-Warehousing-Redshift

A Data Warehousing project for retail sales using dimension modelling best practices with SCD type 2 on AWS Redshift. Utilizing AWS Lambda, Glue Workflows and Python Shell jobs to create and automate an ELT pipeline where batch data coming into S3 is loaded onto Redshift and necessary transformations are performed to meet requirements.

Language: Python - Size: 411 KB - Last synced at: almost 2 years ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0

paulinho-16/MEIC-AID

Todo o conteúdo produzido para a unidade curricular AID (Análise e Integração de Dados), para o curso em Engenharia Informática e Computação na FEUP

Language: Python - Size: 25.4 MB - Last synced at: almost 2 years ago - Pushed at: about 2 years ago - Stars: 0 - Forks: 0

FundingCircle/blueshift Fork of influitive/blueshift

Amazon Redshift adapter for Sequel

Language: Ruby - Size: 80.1 KB - Last synced at: about 1 year ago - Pushed at: about 2 years ago - Stars: 0 - Forks: 0

mmosad19419/Sparkify-Cloud-Data-Warehouse-with-AWS-Cloud-Redshift

Sparkify Cloud Data Warehouse with AWS Cloud Redshift for Sparkify music streaming app company

Language: Python - Size: 15.6 KB - Last synced at: about 2 years ago - Pushed at: about 2 years ago - Stars: 0 - Forks: 0

bautidea/python_iac_pipeline

Data pipeline using python infrastructure as a code (IaC), between Amazon S3 Bucket and Amazon Redshift

Language: Jupyter Notebook - Size: 20.1 MB - Last synced at: almost 2 years ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0

Sanjay9921/AWS-Projects

Collection of AWS mini projects and projects

Language: Jupyter Notebook - Size: 14.6 KB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0

epomatti/aws-redshift

AWS Redshift

Language: HCL - Size: 106 KB - Last synced at: 2 months ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0

pgplarosa/Data-Architecture-for-Tracking-COVID-Statistics-and-Sentiments

Data Engineering Final Project - June 23, 2022

Language: Python - Size: 4.09 MB - Last synced at: about 2 years ago - Pushed at: almost 3 years ago - Stars: 0 - Forks: 0

santiagortiiz/Platzi-AWS-Redshift

Platzi. School of Amazon Web Services. Redshift for Big Data management.

Size: 101 MB - Last synced at: 3 months ago - Pushed at: almost 3 years ago - Stars: 0 - Forks: 0

ManjinderSingh3/ETL-Operations-using-AWS-Glue-and-Redshift

Used AWS Glue to perform ETL operations and load resultant data to AWS Redshift. In the second phase used AWS CloudWatch rules and LAMBDA to automatically run GLUE Jobs

Language: Python - Size: 604 KB - Last synced at: about 2 years ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0

cmuth001/file-upload-to-s3-save-in-redshift

A simple application to upload a csv file to AWS s3 and save in Apachie Redshift Cluster

Language: Python - Size: 13.3 MB - Last synced at: about 2 years ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0

Prajna-Bahuguna/Redshift-Terraform

Configuring Redshift cluster using Terraform.

Language: HCL - Size: 11.7 KB - Last synced at: over 1 year ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 3

RATHOD-SHUBHAM/Amazon-Cloud

List of amazing AWS Services that can be utilized.🚀

Language: Jupyter Notebook - Size: 14.9 MB - Last synced at: 2 months ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0

paul-data-ai/Cloud-Data-Warehouse-Using-S3-and-Redshift-

Udacity Data Engineering Nanodegree Project 3

Language: Python - Size: 5.86 KB - Last synced at: about 2 years ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0

polarbeargo/Data-Pipelines-with-Airflow

Language: Python - Size: 1020 KB - Last synced at: about 1 year ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0

najuzilu/CDW-AWSRedshift

Building a cloud data warehouse with AWS Redshift.

Language: Python - Size: 334 KB - Last synced at: about 2 months ago - Pushed at: almost 4 years ago - Stars: 0 - Forks: 0

jsparadacelis/SparkifyRedshift

This project contains files to create a Data Warehouse using Amazon Redshift which is a columnar database based on PostgreSQL.

Language: Python - Size: 203 KB - Last synced at: almost 2 years ago - Pushed at: about 4 years ago - Stars: 0 - Forks: 0

jomavera/dataPipeline

ETL pipeline with AWS Redshift orchestrated with Airflow

Language: Python - Size: 275 KB - Last synced at: about 2 years ago - Pushed at: about 4 years ago - Stars: 0 - Forks: 0

bhavinidata/DatawareHouse-AWS-Redshift

An implementation of a Data Warehouse leveraging AWS RedShift. This project builds an ETL pipeline for the database hosted on AWS Redshift that extracts their data from multiple JSON files residing in S3 buckets, stages them in Redshift, and transforms data into a set of dimensional tables for their analytics team to continue finding insights in what songs their users are listening to.

Language: Jupyter Notebook - Size: 51.8 KB - Last synced at: about 2 years ago - Pushed at: about 4 years ago - Stars: 0 - Forks: 0

jomavera/DWHawsRedshift

Data warehouse with AWS Redshift

Language: Python - Size: 202 KB - Last synced at: about 2 years ago - Pushed at: about 4 years ago - Stars: 0 - Forks: 0

omarfessi/automation-data-pipeline-Airflow-Redshift

Data pipelines created and monitored using Airflow to feed data into Redshift

Language: Python - Size: 43 KB - Last synced at: about 2 years ago - Pushed at: about 4 years ago - Stars: 0 - Forks: 0

Gares95/Data-Warehouse_AWS-Redshift

Building an ETL pipeline for a database hosted on Redshift. Project based on Udacity's template.

Language: Jupyter Notebook - Size: 34.2 KB - Last synced at: 3 months ago - Pushed at: over 4 years ago - Stars: 0 - Forks: 0