GitHub topics: redshift | Ecosyste.ms: Repos

thexhr/ctempd

Small daemon to set screen temperature automatically

Language: C - Size: 42 KB - Last synced at: 25 days ago - Pushed at: almost 2 years ago - Stars: 2 - Forks: 0

mikeacosta/airflow-data-pipeline

ETL Workflow Automation with Apache Airflow

Language: Python - Size: 131 KB - Last synced at: 2 months ago - Pushed at: about 5 years ago - Stars: 2 - Forks: 0

bhavanachitragar/zillow-data-analytics

A Python script extracts data from Zillow and stores it in an initial S3 bucket. Then, Lambda functions handle the flow: copying the data to a processing bucket and transforming it from JSON to CSV format. The final CSV data resides in another S3 bucket, ready to be loaded into Amazon Redshift for in-depth analysis. QuickSight for visualizations

Language: Python - Size: 66.4 KB - Last synced at: about 2 months ago - Pushed at: 11 months ago - Stars: 0 - Forks: 0

confluentinc/demo-change-data-capture

This demo shows how to capture data changes from relational databases (Oracle and PostgreSQL) and stream them to Confluent Cloud, use ksqlDB for real-time stream processing, send enriched data to cloud data warehouses (Snowflake and Amazon Redshift).

Language: HCL - Size: 8.57 MB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 10 - Forks: 2

gfelot/DEND-DataWarehouse

Learn to build a Data Warehouse on AWS with an Redshift cluster from S3 data - An Udacity Data Engineer Nano Degree Project

Language: Python - Size: 118 KB - Last synced at: 11 months ago - Pushed at: almost 6 years ago - Stars: 2 - Forks: 1

mrzzy/providence

Apply Data Engineering to Personal Finance

Language: HTML - Size: 4.77 MB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 5 - Forks: 0

TerrorSquad/ansible-post-installation

Ansible playbook used for installation of essential development and general use software

Language: Shell - Size: 2.81 MB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 13 - Forks: 1

thesaas-company/xray

Xray: Simplify database structure extraction for MySQL, PostgreSQL, BigQuery, Redshift, MsSQL and Snowflake in Go

Language: Go - Size: 24.8 MB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 3 - Forks: 1

dzrekenathan/Yahoo-finances-data-event

A data engineering training project to build an end-to-end pipline for a real-time processing of data

Language: Jupyter Notebook - Size: 86.9 KB - Last synced at: about 1 month ago - Pushed at: 11 months ago - Stars: 0 - Forks: 0

success4lyf/AWS-Data-Warehouse

An implementation of ETL pipeline in the cloud using AWS Resources such as CloudFormation, S3 Bucket, Lambda, Redshift and others.

Language: Python - Size: 13.8 MB - Last synced at: 11 months ago - Pushed at: about 2 years ago - Stars: 2 - Forks: 0

faiyaz106/Football-League-Analytics

This is repository is related to building end to end data pipeline for football leagues analysis and predictive modeling.

Language: Python - Size: 8.91 MB - Last synced at: 12 months ago - Pushed at: 12 months ago - Stars: 2 - Forks: 1

ironSource/node-pg-metadata 📦

quickly get the metadata of a postresql or redshift db

Language: JavaScript - Size: 11.7 KB - Last synced at: about 1 month ago - Pushed at: about 2 years ago - Stars: 4 - Forks: 3

kor-financial-market-project/kfm-dashboard

💸한국 금융시장과 소비에 대한 이해💸_한국은행 데이터의 시각화

Language: Python - Size: 3.63 MB - Last synced at: 11 months ago - Pushed at: 12 months ago - Stars: 0 - Forks: 0

mashiike/redshift-data-sql-driver

go sql driver for Redshift-Data API

Language: Go - Size: 63.5 KB - Last synced at: 1 day ago - Pushed at: about 1 year ago - Stars: 8 - Forks: 2

pandark/dotfiles 📦

This repository moved. Please use the new location https://gitlab.com/pandark/dotfiles

Size: 0 Bytes - Last synced at: 12 months ago - Pushed at: 12 months ago - Stars: 2 - Forks: 0

essraahmed/Data-Warehouse-With-Redshift

Data Warehouse with AWS Redshift and Visualizing data using Power BI

Language: Jupyter Notebook - Size: 618 KB - Last synced at: 12 months ago - Pushed at: over 2 years ago - Stars: 8 - Forks: 1

essraahmed/Data-Pipeline-with-Airflow

Data Pipeline with Apache Airflow

Language: Python - Size: 444 KB - Last synced at: 12 months ago - Pushed at: over 2 years ago - Stars: 1 - Forks: 1

elinefarias/formacao-datawarehouse-Redshift-BigQuery-SnowFlake

Repositório destinado a estudos sobre a plataforma snowflake

Size: 9.77 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

GoogleCloudPlatform/datacatalog-connectors-rdbms 📦

Sample code with integration between Data Catalog and RDBMS data sources.

Language: Python - Size: 532 KB - Last synced at: about 11 hours ago - Pushed at: over 3 years ago - Stars: 72 - Forks: 50

TommasoRonconi/galapy

Spectral modelling tool for galaxies in python

Language: Python - Size: 198 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 5 - Forks: 1

jasoncorneliog/ssh-config-bastion

This project allows a user to connect to a database resource (Redshift) behind a bastion host from localhost.

Language: Shell - Size: 69.3 KB - Last synced at: about 1 year ago - Pushed at: over 4 years ago - Stars: 0 - Forks: 0

yarox/data-eng-nano-project3

Code for the third project of the Udacity Data Engineering Nanodegree

Language: Jupyter Notebook - Size: 157 KB - Last synced at: about 1 year ago - Pushed at: almost 5 years ago - Stars: 0 - Forks: 0

yarox/data-eng-nano-project5

Code for the fifth project of the Udacity Data Engineering Nanodegree

Language: Python - Size: 188 KB - Last synced at: about 1 year ago - Pushed at: over 4 years ago - Stars: 1 - Forks: 0

ymktmk/lambda-s3-copy-redshift

LambdaでS3のJSONデータをRedshiftに取り込むサンプル

Language: Python - Size: 1000 Bytes - Last synced at: about 1 year ago - Pushed at: about 3 years ago - Stars: 0 - Forks: 0

shipyardapp/amazonredshift-blueprints

Simplified blueprints for building data pipelines with Amazon Redshift.

Language: Python - Size: 39.1 KB - Last synced at: about 1 year ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0

Rdkang/chinguBrightnessChanger

a brightness and color temperature GUI with vim keybindings

Language: Python - Size: 36.1 KB - Last synced at: about 1 year ago - Pushed at: over 3 years ago - Stars: 1 - Forks: 0

eduardofb/redshift-create-manifest

Redshift script to create a MANIFEST file recursively

Language: Python - Size: 16.6 KB - Last synced at: about 1 year ago - Pushed at: almost 8 years ago - Stars: 3 - Forks: 1

JialunC/Udacity-Data-Engineering

I took the curriculum from https://github.com/Flor91/Data-engineering-nanodegree, on top of that I Dockerize it to avoid interfering with your local environment

Language: Jupyter Notebook - Size: 873 KB - Last synced at: about 1 year ago - Pushed at: about 4 years ago - Stars: 3 - Forks: 1

SimpleFinance/radshift 📦

Custom authentication proxy for AWS Redshift (unmaintained)

Language: Go - Size: 302 KB - Last synced at: about 1 year ago - Pushed at: almost 8 years ago - Stars: 2 - Forks: 2

conduitio-labs/conduit-connector-redshift

Conduit connector for Amazon Redshift

Language: Go - Size: 196 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 0

jaredfiacco2/AWS-E-Scooter-Tracker

Use AWS Lambda to Pull E-Scooter and E-Bike Location Data, store in S3 & Redshift using Data Vault Data Model, Server to Google Data Studio Dashboard

Language: PLpgSQL - Size: 29.2 MB - Last synced at: 9 months ago - Pushed at: almost 3 years ago - Stars: 12 - Forks: 1

vuanhtuan1012/data-warehouse

Building an ETL pipeline that extracts data from S3, stages them in Redshift.

Language: Jupyter Notebook - Size: 429 KB - Last synced at: about 2 months ago - Pushed at: almost 4 years ago - Stars: 2 - Forks: 0

practo/tipoca-stream

Near real time cloud native data pipeline in AWS (CDC+Sink). Hosts code for RedshiftSink. RDS to RedshiftSink Pipeline with masking and reloading support.

Language: Go - Size: 16.3 MB - Last synced at: 11 months ago - Pushed at: about 1 year ago - Stars: 47 - Forks: 5

KyleGough/dotfiles

Personal dotfiles for: zsh, vim, i3, neofetch, urxvt, redshift, libinput gestures, and dmenu.

Language: Shell - Size: 202 KB - Last synced at: about 1 year ago - Pushed at: about 2 years ago - Stars: 1 - Forks: 0

tensult/multidb-jdbc-driver

Helps connect to multiple databases and also supports query routing.

Language: Java - Size: 18.6 KB - Last synced at: about 1 year ago - Pushed at: over 2 years ago - Stars: 2 - Forks: 1

SlicingDice/slicingdice-javascript

Official JavaScript client for SlicingDice, Data Warehouse and Analytics Database as a Service.

Language: JavaScript - Size: 474 KB - Last synced at: about 1 year ago - Pushed at: over 6 years ago - Stars: 11 - Forks: 2

ZeroTwoDataRW/Ecommerce-DE-On-AWS

An end-to-end AWS data engineering pipeline that orchestrates data ingestion, processing, and storage using Kinesis, S3, Redshift, Lambda, DynamoDB

Language: Python - Size: 81.1 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 0

ritikdhame/Automated-Telecom-Customer-Churn-Analysis

AWS Glue & Airflow automate weekly churn analysis pipeline (S3, Redshift) for telco (1M+ customers) with actionable Tableau dashboard (user segmentation, churn reasons, geographical distribution).

Language: Python - Size: 2.15 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

annehutter/grid-model

Code to compute ionization field from density fields and source catalogue (CIFOG)

Language: C - Size: 259 MB - Last synced at: about 2 months ago - Pushed at: over 2 years ago - Stars: 10 - Forks: 4

Patreon/euphrates 📦

The great data river from mysql to redshift 💦🌊

Language: Java - Size: 63.5 KB - Last synced at: about 1 year ago - Pushed at: almost 6 years ago - Stars: 10 - Forks: 1

anthonywong611/SF-Transactions-Modeling

Store San Francisco transaction data into SFTP and pull into a Redshift data warehouse.

Language: Jupyter Notebook - Size: 7.85 MB - Last synced at: about 1 year ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0

prayagnshah/End-to-End-Pipeline

Zillow Data Pipeline: Extracts data from Zillow, transfers it through AWS services, and performs analytics. Utilizes Python scripts, AWS Lambda, S3, Amazon RedShift, and QuickSight. Explore docs/images for architecture visuals.

Language: Python - Size: 727 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

leletan/maiev

The world of warcraft heroine, a part of the Watchers

Language: Scala - Size: 53.7 KB - Last synced at: about 1 year ago - Pushed at: almost 7 years ago - Stars: 2 - Forks: 0

tangphucnhan/Airflow-Copy-S3-To-Redshift

Develop an Airflow pipeline to transfer data from S3 bucket to Redshift warehouse.

Language: Python - Size: 1000 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

varver/Redshift-migration

Migrate Redshift Data to Another Redshft Cluster or Redshift in another AWS Account .

Language: Python - Size: 11.7 KB - Last synced at: about 1 year ago - Pushed at: almost 8 years ago - Stars: 0 - Forks: 0

shivamsharma/SqlShift Fork of himanshpal/SqlShift

🚚 Mysql to Redshift data transfer using Apache Spark.

Language: Scala - Size: 2.11 MB - Last synced at: about 1 year ago - Pushed at: over 6 years ago - Stars: 0 - Forks: 0

MilhosOU/DBT_BigQuery_TableSizeWatcher

Monitors and estimates sizes of specified tables and columns in a large database cluster.

Size: 3.91 KB - Last synced at: about 1 year ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 0

aws-samples/amazon-redshift-data-lifecycle-manager

This code sample focuses on automating movement (unload) and management (delete/vacuum/analyze) of data in Amazon Redshift tables with temporal attributes through a simple configuration while maintaining movement and management choice, priority, logging, auditing and checkpointing.

Language: Python - Size: 919 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 6 - Forks: 5

mhaywardhill/Redshift-DWH-ETL

Udacity Data Engineering Nanodegree Project 3

Language: Jupyter Notebook - Size: 15.6 KB - Last synced at: about 1 year ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0

dmanjunath/node-redshift

A simple collection of tools to help you get started with Amazon Redshift from node.js

Language: JavaScript - Size: 525 KB - Last synced at: 24 days ago - Pushed at: over 2 years ago - Stars: 69 - Forks: 48

data-integrations/redshifttos3-action 📦

Unloads the result of a query for redshift to one or more files on Amazon Simple Storage Service (Amazon S3).

Language: Java - Size: 55.7 KB - Last synced at: about 1 year ago - Pushed at: over 2 years ago - Stars: 1 - Forks: 1

garystafford/tickit-data-lake-demo

Resources for video demonstrations and blog posts related to DataOps on AWS

Language: Python - Size: 42.1 MB - Last synced at: about 1 year ago - Pushed at: over 3 years ago - Stars: 155 - Forks: 89

joyceannie/US-immigrations-data-warehouse

A data warehouse to perform analytics on the immigration trends in the US.

Language: Jupyter Notebook - Size: 10.7 MB - Last synced at: about 1 month ago - Pushed at: about 2 years ago - Stars: 2 - Forks: 1

bpjena/data-apps

data-native applications

Language: Python - Size: 12.1 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 3 - Forks: 1

ramitsurana/redshift-scripts

Sample scripts and SQL commands for RedShift

Language: PLpgSQL - Size: 41 KB - Last synced at: 2 months ago - Pushed at: over 6 years ago - Stars: 7 - Forks: 6

andrelbd1/poc_api_flask_celery

API poc using Flask and Celery for asynchronous processing

Language: Python - Size: 283 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

NitinSPatil15/Project-5-Data-Pipelines-in-Airflow

An automated ETL pipeline for songs data for monitoring end to end data workflows having data warehouse in AWS Redshift using Apache Airflow

Language: Python - Size: 279 KB - Last synced at: about 1 year ago - Pushed at: almost 5 years ago - Stars: 0 - Forks: 2

ankur715/AWS_Redshift_Postgresql

redshift cluster analysis with postgresql database

Language: Python - Size: 7.33 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 0

daniel-lyon/zfinder

Redshift Finding Algorithm

Language: Python - Size: 114 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 0

alphro/twilioETL

Create PostgreSQL ETL pipeline using Twilio API into redshift database.

Language: Python - Size: 8.79 KB - Last synced at: about 1 year ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0

exasol/redshift-virtual-schema

Virtual Schema for connecting Redshift as a data source to Exasol

Language: Java - Size: 63.5 KB - Last synced at: about 2 months ago - Pushed at: 2 months ago - Stars: 0 - Forks: 1

koustreak/Sondesh

Metadata Comparison Toolkit. As of now, V-1.0.0 only consists Comparison of two DDL file ( .sql ) or two DDL statement. You can also parse sql files .

Language: Python - Size: 285 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 10 - Forks: 0

jacobceles/multi-db-querier

Web App which uses SQL to query Redshift, MySQL, and MongoDB.

Language: Python - Size: 13.2 MB - Last synced at: over 1 year ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 1

topgyalgurung/AWS_Redshift-

Size: 1000 Bytes - Last synced at: over 1 year ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0

SideswipeeZ/houdini-shader-converter

Houdini Python Shelf tool for Converting Shaders within Houdini using JSON

Language: Python - Size: 85.9 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 38 - Forks: 9

smpotts/data-api-explorer

Using the Redshift Data API to execute commands against a Redshift cluster.

Language: Python - Size: 2.93 KB - Last synced at: 30 days ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0

mongodb-partners/Atlas_to_Redshift

Migrate data from MongoDB Atlas to AWS Redshift

Language: Python - Size: 208 KB - Last synced at: about 1 year ago - Pushed at: about 2 years ago - Stars: 9 - Forks: 3

tianlinhe/data_warehouse_RedShift

Data Engineer (Udacity): Project 3 Data Warehouse (DWH) with RedShift on Amazon Web Service (AWS)

Language: Jupyter Notebook - Size: 124 KB - Last synced at: over 1 year ago - Pushed at: over 4 years ago - Stars: 0 - Forks: 1

DataDrivenGit/Yelp_Data_Warehouse

Implemented Data Warehouse using YELP data using EC2 instance from S3 to Redshift

Language: Python - Size: 5.86 KB - Last synced at: over 1 year ago - Pushed at: almost 5 years ago - Stars: 1 - Forks: 0

NorberMV/data-engineering

All micro-challenges from data-engineering Coderhouse Course.

Language: Jupyter Notebook - Size: 557 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

snowplow/snowplow-web-data-model 📦

SQL data model for working with Snowplow web data. Supports Redshift and Looker. Snowflake and BigQuery coming soon

Language: LookML - Size: 218 KB - Last synced at: about 1 year ago - Pushed at: over 4 years ago - Stars: 61 - Forks: 31

kartikra/dattasa

python project for dattasa package in pypi

Language: Python - Size: 64.5 KB - Last synced at: 8 days ago - Pushed at: about 7 years ago - Stars: 4 - Forks: 0

amrelauoty/Sparkify-ETL-Redshift

Language: Jupyter Notebook - Size: 14.6 KB - Last synced at: about 1 month ago - Pushed at: over 2 years ago - Stars: 1 - Forks: 0

fujiwara/Rin

Rin is a Redshift data Importer by SQS messaging.

Language: Go - Size: 149 KB - Last synced at: 7 months ago - Pushed at: over 2 years ago - Stars: 26 - Forks: 6

JeffreySunjay4044/dbmigration

Stream data between two databases . Supports both ddl and dml statements. Built on top of kafka and debezium in python

Language: Python - Size: 138 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 1

pratiksha2712/ETL-Pipeline

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

Language: Python - Size: 1.47 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

eglaubauf/egRedshiftTools

Tools for Redshift in SideFx Houdini

Language: Python - Size: 1.79 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 36 - Forks: 7

wrongusernameyetagain/this_wont_work

A somewhat disorganised repo for ideas and tips I may have with open sourced material.

Size: 3.49 MB - Last synced at: over 1 year ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0

clarityai-eng/jsonschema2ddl Fork of better/jsonschema2db

Generate tables dynamically from a JSON Schema and insert data

Language: Python - Size: 15.5 MB - Last synced at: about 1 year ago - Pushed at: about 3 years ago - Stars: 7 - Forks: 0

Tanay0510/Data-Pipeline-with-Airflow

Built Data Pipelines with Airflow. Created custom operators to perform tasks such as staging the data, filling the data warehouse, and running checks on the data as the final step

Language: Python - Size: 18.6 KB - Last synced at: over 1 year ago - Pushed at: almost 4 years ago - Stars: 1 - Forks: 0

Tanay0510/Cloud-Data-Warehouse

Built an ETL pipeline that extracts their data from S3, stages them in Redshift, and transforms data into a set of dimensional tables. Load data from S3 to staging tables on Redshift and execute SQL statements that create the analytics tables from these staging tables.

Language: Python - Size: 7.81 KB - Last synced at: over 1 year ago - Pushed at: almost 4 years ago - Stars: 1 - Forks: 0

datafe/api-market

An APIs platform provides an API market and authorized APIs management for users. This project uses Node.js + React + Vite to implement a system based on Alibaba Cloud DataWorks DataService. (基于阿里云DataWorks数据服务建立一个API的市场，同时用户也能在上面申请API。)

Language: TypeScript - Size: 1.26 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 3 - Forks: 0

groupon/luigi-warehouse

A luigi powered analytics / warehouse stack

Language: Python - Size: 62.5 KB - Last synced at: about 1 year ago - Pushed at: about 8 years ago - Stars: 87 - Forks: 18

shahinyusifli/dw-credit-risk

A system has been set up for analyzing credit risk, involving a data warehouse and pipeline. The tools used for this solution include Prefect for workflow management, Redshift for the data warehouse, and an S3 bucket for storage.

Language: Python - Size: 1.33 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

Dina-Hosny/Sparkify---Data-Pipelines-with-Airflow

Sparkify - Data Pipelines with Airflow - Udacity Data Engineering Expert Track.

Language: Python - Size: 22.5 KB - Last synced at: about 2 months ago - Pushed at: over 2 years ago - Stars: 2 - Forks: 0

alwayswannaly/s3-redshift-bridge

Sync S3 and Redshift data

Language: Python - Size: 39.1 KB - Last synced at: over 1 year ago - Pushed at: about 2 years ago - Stars: 1 - Forks: 1

osule/lard

An operator for performing the Extract and Load bit of the ELT data pipeline in an Apache Airflow workflow.

Language: Python - Size: 9.77 KB - Last synced at: 29 days ago - Pushed at: almost 3 years ago - Stars: 1 - Forks: 0

abideenml/RealTime-StarRatingPrediction-with-AWSKinesis

This repository contains an End to End Real time 🕰️ Machine Learning Pipeline to predict star ⭐️ rating of product reviews. This project uses AWS Sagemaker, Kinesis, Lambda, S3, Redshift, Athena, and Step functions. Deployment of multiple models for AB testing and Bandit testing is also included.

Language: Jupyter Notebook - Size: 15.1 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0