GitHub topics: redshift

Repositories

DataKitchen/dataops-testgen

DataOps Data Quality TestGen is part of DataKitchen's Open Source Data Observability. DataOps TestGen delivers simple, fast data quality test generation and execution by data profiling, new dataset hygiene review, AI generation of data quality validation tests, ongoing testing of data refreshes, & continuous anomaly monitoring

Language: Python - Size: 5.23 MB - Last synced at: about 2 hours ago - Pushed at: about 3 hours ago - Stars: 55 - Forks: 3

Askaniy/TrueColorTools

GUI application for calculating human-visible colors of celestial bodies from their photometric data

Language: Python - Size: 103 MB - Last synced at: about 6 hours ago - Pushed at: about 7 hours ago - Stars: 20 - Forks: 6

getredash/redash

Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.

Language: Python - Size: 27.2 MB - Last synced at: about 7 hours ago - Pushed at: about 11 hours ago - Stars: 27,254 - Forks: 4,459

donnemartin/awesome-aws

A curated list of awesome Amazon Web Services (AWS) libraries, open source repos, guides, blogs, and other resources. Featuring the Fiery Meter of AWSome.

Language: Python - Size: 784 KB - Last synced at: about 7 hours ago - Pushed at: about 1 year ago - Stars: 12,904 - Forks: 1,758

apache/doris

Apache Doris is an easy-to-use, high performance and unified analytics database.

Language: Java - Size: 1020 MB - Last synced at: about 15 hours ago - Pushed at: about 17 hours ago - Stars: 13,611 - Forks: 3,409

elementary-data/elementary

The dbt-native data observability solution for data & analytics engineers. Monitor your data pipelines in minutes. Available as self-hosted or cloud service with premium features.

Language: HTML - Size: 205 MB - Last synced at: 1 day ago - Pushed at: 3 days ago - Stars: 2,055 - Forks: 183

growthbook/growthbook

Open Source Feature Flagging and A/B Testing Platform

Language: TypeScript - Size: 107 MB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 6,528 - Forks: 552

fluentmigrator/fluentmigrator

Fluent migrations framework for .NET

Language: C# - Size: 129 MB - Last synced at: about 11 hours ago - Pushed at: 1 day ago - Stars: 3,359 - Forks: 677

airbytehq/airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.

Language: Python - Size: 664 MB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 18,015 - Forks: 4,493

DataKitchen/data-observability-installer

Installer for DataKitchen's Open Source Data Observability Products. Data breaks. Servers break. Your toolchain breaks. Ensure your team is the first to know and the first to solve with visibility across and down your data estate. Save time with simple, fast data quality test generation and execution. Trust your data, tools, and systems end to end.

Language: Python - Size: 268 KB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 114 - Forks: 10

jitsucom/jitsu

Jitsu is an open-source Segment alternative. Fully-scriptable data ingestion engine for modern data teams. Set-up a real-time data pipeline in minutes, not days

Language: TypeScript - Size: 42.6 MB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 4,286 - Forks: 312

aws/aws-sdk-pandas

pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).

Language: Python - Size: 15.9 MB - Last synced at: 1 day ago - Pushed at: 4 days ago - Stars: 4,013 - Forks: 706

rudderlabs/rudder-server

Privacy and Security focused Segment-alternative, in Golang and React

Language: Go - Size: 308 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 4,173 - Forks: 329

artie-labs/transfer

Database replication platform that leverages change data capture. Stream production data from databases to your data warehouse (Snowflake, BigQuery, Redshift, Databricks) in real-time.

Language: Go - Size: 3.75 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 649 - Forks: 32

tobymao/sqlglot

Python SQL Parser and Transpiler

Language: Python - Size: 485 MB - Last synced at: 4 days ago - Pushed at: 6 days ago - Stars: 7,602 - Forks: 840

ToucanToco/weaverbird

A visual data pipeline builder with various backends

Language: TypeScript - Size: 45.1 MB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 101 - Forks: 17

dbeaver/dbeaver

Free universal database tool and SQL client

Language: Java - Size: 184 MB - Last synced at: 4 days ago - Pushed at: 6 days ago - Stars: 43,282 - Forks: 3,681

Multiwoven/multiwoven

🔥🔥🔥 Open source composable CDP - alternative to hightouch and census.

Language: Ruby - Size: 6.17 MB - Last synced at: 4 days ago - Pushed at: 6 days ago - Stars: 1,601 - Forks: 71

datacoves/dbt-coves

CLI tool for dbt users to simplify creation of staging models (yml and sql) files

Language: Python - Size: 2.57 MB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 262 - Forks: 16

starlake-ai/starlake

Declarative text based tool for data analysts and engineers to extract, load, transform and orchestrate their data pipelines.

Language: Scala - Size: 170 MB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 104 - Forks: 23

dbrest-io/dbREST

Spin up a REST API for any Major Database

Language: Go - Size: 499 KB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 35 - Forks: 3

ploomber/jupysql Fork of catherinedevlin/ipython-sql

Better SQL in Jupyter. 📊

Language: Python - Size: 12.9 MB - Last synced at: 5 days ago - Pushed at: about 1 month ago - Stars: 770 - Forks: 78

tuva-health/tuva

Main repo including core data model, data marts, data quality tests, and terminology sets.

Language: Shell - Size: 43.9 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 249 - Forks: 83

lephare-photoz/lephare

LePHARE is a code for calculating photometric redshifts.

Language: C++ - Size: 101 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 6 - Forks: 1

tuva-health/demo

A starter dbt project and synthetic claims dataset for trying out the Tuva Project.

Size: 3.53 MB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 21 - Forks: 28

Wisser/Jailer

Database Subsetting and Relational Data Browsing Tool.

Language: Java - Size: 417 MB - Last synced at: 4 days ago - Pushed at: 7 days ago - Stars: 2,985 - Forks: 133

plotly/falcon 📦

Free, open-source SQL client for Windows and Mac 🦅

Language: JavaScript - Size: 13.7 MB - Last synced at: 6 days ago - Pushed at: about 2 years ago - Stars: 5,126 - Forks: 278

kestra-io/plugin-jdbc

Language: Java - Size: 2.4 MB - Last synced at: 9 days ago - Pushed at: 9 days ago - Stars: 16 - Forks: 17

k1LoW/tbls

tbls is a CI-Friendly tool to document a database, written in Go.

Language: Go - Size: 125 MB - Last synced at: 8 days ago - Pushed at: 9 days ago - Stars: 3,751 - Forks: 174

Chemrat/redshift-qt

redshift-gtk rewrite with C++/Qt

Language: C++ - Size: 37.1 KB - Last synced at: 2 days ago - Pushed at: 9 days ago - Stars: 30 - Forks: 6

srinadh99/AstroFormer

Photometry Guided Cross Attention Transformers for Astronomical Image Processing

Language: Jupyter Notebook - Size: 22.2 MB - Last synced at: 10 days ago - Pushed at: 10 days ago - Stars: 0 - Forks: 0

kartikey2807/Validation-Scripts

Validation scripts to check for consistency between L5 and L3 schemas.

Language: Python - Size: 142 KB - Last synced at: 10 days ago - Pushed at: 10 days ago - Stars: 0 - Forks: 0

fdmsantos/terraform-aws-kinesis-firehose

Dynamic Terraform module, which creates a Kinesis Firehose Stream and others resources like Cloudwatch, IAM Roles and Security Groups that integrate with Kinesis Firehose. Supports all destinations and all Kinesis Firehose Features.

Language: HCL - Size: 260 KB - Last synced at: 10 days ago - Pushed at: 2 months ago - Stars: 13 - Forks: 8

CartoDB/analytics-toolbox-core

A set of UDFs and Procedures to extend BigQuery, Snowflake, Redshift, Postgres and Databricks with Spatial Analytics capabilities

Language: JavaScript - Size: 12.6 MB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 192 - Forks: 44

eglaubauf/egMatLib

Material Library for Houdini Mantra, Redshift and Arnold

Language: Python - Size: 233 MB - Last synced at: 11 days ago - Pushed at: 11 days ago - Stars: 98 - Forks: 19

synmetrix/synmetrix

Synmetrix – production-ready open source semantic layer on Cube

Language: JavaScript - Size: 4.38 MB - Last synced at: 7 days ago - Pushed at: 3 months ago - Stars: 546 - Forks: 31

elgohr/go-localstack

Go Wrapper for using localstack

Language: Go - Size: 16.9 MB - Last synced at: 11 days ago - Pushed at: 11 days ago - Stars: 82 - Forks: 19

montara-io/dbt-command-center

Never sift through endless dbt™ logs again. dbt Command Center is a free, open-source, local web application that provides a user-friendly interface to monitor and manage dbt runs.

Language: TypeScript - Size: 3.55 MB - Last synced at: 11 days ago - Pushed at: 11 days ago - Stars: 28 - Forks: 0

gatanegro/Astro--COM

Astronomy in COM ( Collatz octave sequence Model) and UOFT theory framework

Language: Python - Size: 57.6 KB - Last synced at: 11 days ago - Pushed at: 11 days ago - Stars: 0 - Forks: 0

san089/goodreads_etl_pipeline

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

Language: Python - Size: 1.31 MB - Last synced at: 12 days ago - Pushed at: about 5 years ago - Stars: 1,365 - Forks: 224

kjchrz03/sql_data_requests

A sample of play data requests showcasing my intermediate SQL data skills.

Size: 13.7 KB - Last synced at: 13 days ago - Pushed at: 13 days ago - Stars: 0 - Forks: 0

data-dot-all/dataall

A modern data marketplace that makes collaboration among diverse users (like business, analysts and engineers) easier, increasing efficiency and agility in data projects on AWS.

Language: Python - Size: 97.9 MB - Last synced at: 14 days ago - Pushed at: 14 days ago - Stars: 240 - Forks: 82

runprism/prism

Prism is the easiest way to develop, orchestrate, and execute data pipelines in Python.

Language: Python - Size: 2.42 MB - Last synced at: 5 days ago - Pushed at: 5 months ago - Stars: 85 - Forks: 2

jonls/redshift

Redshift adjusts the color temperature of your screen according to your surroundings. This may help your eyes hurt less if you are working in front of the screen at night.

Language: C - Size: 2.05 MB - Last synced at: 17 days ago - Pushed at: 11 months ago - Stars: 5,991 - Forks: 426

awslabs/aws-orbit-workbench 📦

A Data Platform built for AWS, powered by Kubernetes.

Language: Python - Size: 53.7 MB - Last synced at: 9 days ago - Pushed at: almost 2 years ago - Stars: 148 - Forks: 92

moia-oss/bastion-host-forward

CDK Construct for creating a bastion host to forward a connection to several AWS data services inside a private subnet from your local machine

Language: TypeScript - Size: 2.48 MB - Last synced at: 17 days ago - Pushed at: 17 days ago - Stars: 31 - Forks: 6

josephmachado/beginner_de_project

Beginner data engineering project - batch edition

Language: HTML - Size: 31.1 MB - Last synced at: 17 days ago - Pushed at: 3 months ago - Stars: 513 - Forks: 158

Correia-jpv/fucking-awesome-aws

A curated list of awesome Amazon Web Services (AWS) libraries, open source repos, guides, blogs, and other resources. Featuring the Fiery Meter of AWSome. With repository stars⭐ and forks🍴

Language: Python - Size: 763 KB - Last synced at: 5 days ago - Pushed at: 18 days ago - Stars: 12 - Forks: 1

mauriho/texture-connector

Texture Connector is a tool for Autodesk Maya that automates the assignment and connection of textures to materials.

Language: Python - Size: 184 KB - Last synced at: 19 days ago - Pushed at: 19 days ago - Stars: 3 - Forks: 0

DunHouGo/renderEngine

Custom API wrapper for popular render engines in Cinema 4D.

Language: Python - Size: 848 KB - Last synced at: 19 days ago - Pushed at: 19 days ago - Stars: 13 - Forks: 5

capitalone/locopy

locopy: Loading/Unloading to Redshift and Snowflake using Python.

Language: Python - Size: 9.72 MB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 107 - Forks: 49

ScalefreeCOM/datavault4dbt

Scalefree's dbt package for a Data Vault 2.0 implementation congruent to the original Data Vault 2.0 definition by Dan Linstedt including the Staging Area, DV2.0 main entities, PITs and Snapshot Tables.

Language: PLSQL - Size: 31.8 MB - Last synced at: 8 days ago - Pushed at: 8 days ago - Stars: 152 - Forks: 29

apache/doris-streamloader

Stream Loader for Apache Doris

Language: Go - Size: 39.1 KB - Last synced at: 6 days ago - Pushed at: 25 days ago - Stars: 24 - Forks: 18

datacoves/docs

Datacoves public documentation

Language: Python - Size: 47.5 MB - Last synced at: 1 day ago - Pushed at: 22 days ago - Stars: 3 - Forks: 2

dresende/node-orm2

Object Relational Mapping

Language: JavaScript - Size: 2.32 MB - Last synced at: 21 days ago - Pushed at: about 2 months ago - Stars: 3,061 - Forks: 374

philips-software/gautocloud-connectors 📦

goautocloud connectors for Cloud foundry services

Language: Go - Size: 1.33 MB - Last synced at: 8 days ago - Pushed at: 23 days ago - Stars: 3 - Forks: 1

cuebook/CueObserve

Timeseries Anomaly detection and Root Cause Analysis on data in SQL data warehouses and databases

Language: Python - Size: 6.15 MB - Last synced at: 22 days ago - Pushed at: about 3 years ago - Stars: 229 - Forks: 24

k1LoW/setup-tbls

GitHub Action for tbls

Language: Makefile - Size: 8.79 KB - Last synced at: 15 days ago - Pushed at: 6 months ago - Stars: 14 - Forks: 1

snowplow/snowplow-rdb-loader

Stores Snowplow enriched events in Redshift, Snowflake and Databricks

Language: Scala - Size: 6.48 MB - Last synced at: 16 days ago - Pushed at: 27 days ago - Stars: 31 - Forks: 16

terraform-aws-modules/terraform-aws-redshift

Terraform module to create AWS Redshift resources 🇺🇦

Language: HCL - Size: 186 KB - Last synced at: 19 days ago - Pushed at: about 1 month ago - Stars: 87 - Forks: 160

starlake-ai/jsqltranspiler

Rewrite BigQuery, Redshift, Snowflake and Databricks queries into DuckDB compatible SQL (with deep transformation of functions, data types and format characters) using Java.

Language: Java - Size: 12.4 MB - Last synced at: 28 days ago - Pushed at: about 1 month ago - Stars: 49 - Forks: 4

xnuinside/simple-ddl-parser

Simple DDL Parser to parse SQL (HQL, TSQL, AWS Redshift, BigQuery, Snowflake and other dialects) ddl files to json/python dict with full information about columns: types, defaults, primary keys, etc. & table properties, types, domains, etc.

Language: Python - Size: 2.04 MB - Last synced at: 25 days ago - Pushed at: 7 months ago - Stars: 195 - Forks: 42

HakeemSalaudeen/salesproject-batch-processing-on-AWS

This repository contains Terraform code to deploy a serverless batch processing architecture on AWS, designed to replace an on-premises system with a scalable, reliable, and maintainable cloud solution.

Language: HCL - Size: 107 KB - Last synced at: 30 days ago - Pushed at: 30 days ago - Stars: 0 - Forks: 0

grai-io/grai-core

Language: Python - Size: 121 MB - Last synced at: 18 days ago - Pushed at: 29 days ago - Stars: 303 - Forks: 21

lynnlangit/Hello-AWS-Data-Services

AWS Data/MLServices sample code & notes for my LinkedIn Learning courses

Language: Jupyter Notebook - Size: 96 MB - Last synced at: 7 days ago - Pushed at: about 1 year ago - Stars: 177 - Forks: 99

a-cosmology-group/acg

Cosmology Group repo for its public webpage, reviews & publications on tensions in ΛCDM, redshift and the CMB

Language: HTML - Size: 169 MB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 6 - Forks: 4

scratchdata/scratchdata

Scratch is a swiss army knife for big data.

Language: Go - Size: 13.6 MB - Last synced at: 23 days ago - Pushed at: 10 months ago - Stars: 1,113 - Forks: 54

aws-samples/query-databases-with-natural-language

This project uses the open-source model Mistral Small, deployed in Amazon SageMaker or invoked via API on Amazon Bedrock, to enable users to chat with their database using natural language, without writing any code or SQL query.

Language: Jupyter Notebook - Size: 707 KB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 21 - Forks: 4

ltguillaume/redshift-tray

A no-frills GUI for the excellent Redshift, with some optional OS hotkeys

Language: AutoHotkey - Size: 1.18 MB - Last synced at: about 1 month ago - Pushed at: 8 months ago - Stars: 61 - Forks: 1

kirillbobyrev/dotfiles

eternal quest of crafting an environment that works for me

Language: Lua - Size: 594 KB - Last synced at: 16 days ago - Pushed at: about 1 month ago - Stars: 4 - Forks: 1

tuva-health/medicare_cclf_connector

This connector is a dbt project that maps Medicare CCLF claims data to the Tuva Input Layer.

Size: 1.02 MB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 13 - Forks: 16

apache/doris-sdk

SDK for Apache Doris

Language: Thrift - Size: 35.2 KB - Last synced at: 6 days ago - Pushed at: about 1 year ago - Stars: 8 - Forks: 7

svc98/AWS_Food_Delivery_Streaming

Engineered a scalable real-time CI/CD pipeline for JSON order data using PySpark, enabling immediate insights.

Language: Python - Size: 1.24 MB - Last synced at: 4 days ago - Pushed at: 9 months ago - Stars: 1 - Forks: 0

svc98/AWS_Airline_Batch

Built an AWS batch processing pipeline for flight data using Glue ETL and Redshift to deliver historical insights.

Language: Python - Size: 4.27 MB - Last synced at: 4 days ago - Pushed at: 9 months ago - Stars: 1 - Forks: 0

unytics/catalog_builder

Data Catalogs Made Easy

Language: Python - Size: 2.64 MB - Last synced at: 7 days ago - Pushed at: about 1 month ago - Stars: 22 - Forks: 2

rdagumampan/yuniql

Free and open source schema versioning and database migration made natively with .NET/6. NEW THIS MAY 2022! v1.3.15 released!

Language: C# - Size: 4.97 MB - Last synced at: 17 days ago - Pushed at: 9 months ago - Stars: 419 - Forks: 64

sauravtanwar786/Data-Engineering-With-AWS

Build an ETL pipeline and data lakehouse solution

Language: Python - Size: 4.69 MB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 0 - Forks: 0

kishaningithub/rdapp

rdapp - Redshift Data API Postgres Proxy

Language: Go - Size: 501 KB - Last synced at: 18 days ago - Pushed at: almost 2 years ago - Stars: 10 - Forks: 0

solaranamnesis/edwin-hubble

Public Domain Works of Edwin Hubble (1889-1953).

Size: 5.86 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 0 - Forks: 0

madand/runit-services

Runit service scripts

Language: Shell - Size: 44.9 KB - Last synced at: 29 days ago - Pushed at: over 2 years ago - Stars: 94 - Forks: 12

top-guns/etl-gun

᠊ᡃ࡚ࠢ࠘ ⸝່ࠡࠣ᠊᠊ࠢ࠘᠊᠊ࠢ࠘᠊᠊ࠢ࠘᠊᠊ࠢ࠘᠊᠊° ̿̿ ̿̿💥 ETL toolkit which supports generators, RxJS streams, IxJS iterables, nodejs streams, error handling, business rules and many more

Language: TypeScript - Size: 2.05 MB - Last synced at: 9 days ago - Pushed at: over 1 year ago - Stars: 20 - Forks: 1

shinichi-takii/ddlparse

DDL parase and Convert to BigQuery JSON schema and DDL statements

Language: Python - Size: 88.9 KB - Last synced at: 25 days ago - Pushed at: over 1 year ago - Stars: 88 - Forks: 29

qwertyiop/dotfiles

dotfilles

Language: Shell - Size: 4.4 MB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 0 - Forks: 0

racarvajal/ML_prediction_pipeline_run

Example code to run prediction pipeline from Carvajal et al. 2023

Language: Jupyter Notebook - Size: 36 MB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 0 - Forks: 0

philips-software/rsdl 📦

Redshift/PostgreSQL real-time TSV downloader

Language: Go - Size: 520 KB - Last synced at: 8 days ago - Pushed at: about 2 months ago - Stars: 4 - Forks: 0

DadaNanjesha/Redshift-ETL-Project

The project covers the complete data pipeline—from importing data from an RDS source to HDFS using Sqoop, processing data with Spark, to executing analytical queries on an AWS Redshift cluster.

Language: Jupyter Notebook - Size: 833 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 1 - Forks: 0

Oceania2018/EntityFrameworkCore.BootKit

EntityFrameworkCore Start Kit

Language: C# - Size: 107 KB - Last synced at: 6 days ago - Pushed at: about 2 months ago - Stars: 37 - Forks: 10

ShottzHT/Healthcare-Analysis

Analyze healthcare data to identify key trends, risk factors, and actionable insights using Tableau dashboards and Python preprocessing. Enhance healthcare decision-making with interactive visualizations and data-driven approaches.

Size: 1.95 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 0 - Forks: 0

zack0061/End-to-End-Data-Pipeline

📈 A scalable, production-ready data pipeline for real-time streaming & batch processing, integrating Kafka, Spark, Airflow, AWS, Kubernetes, and MLflow. Supports end-to-end data ingestion, transformation, storage, monitoring, and AI/ML serving with CI/CD automation using Terraform & GitHub Actions.

Language: Python - Size: 2.6 MB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 0 - Forks: 0

rubnsbarbosa/airflow-data-pipeline

Airflow DAG pipeline which extract data from s3 and load into Redshift

Language: Python - Size: 16.6 KB - Last synced at: 6 days ago - Pushed at: 2 months ago - Stars: 0 - Forks: 0

BhawnaMehbubani/Process-and-Ingest-only-quality-movies-in-Redshift-Dara-Warehouse

This repository contains a production-grade ETL (Extract, Transform, Load) pipeline built with AWS Glue and Amazon Redshift. The pipeline processes a raw IMDb movie dataset stored in Amazon S3, applies data quality validation, dynamically routes data based on validation results, and loads it into Amazon Redshift for advanced analytic

Language: Python - Size: 202 KB - Last synced at: about 1 month ago - Pushed at: 3 months ago - Stars: 1 - Forks: 0