Ecosyste.ms: Repos

An open API service providing repository metadata for many open source software ecosystems.

GitHub topics: etl

Repositories

rainbowatcher/simple

Make data work easier, saving your working time

Language: TypeScript - Size: 967 KB - Last synced: about 2 hours ago - Pushed: about 3 hours ago - Stars: 0 - Forks: 0

ajthinking/data-story

A visual process builder

Language: TypeScript - Size: 126 MB - Last synced: about 2 hours ago - Pushed: about 3 hours ago - Stars: 177 - Forks: 16

sul-dlss/libsys-airflow

Airflow DAGS for migrating and managing ILS data into FOLIO along with other LibSys workflows

Language: Python - Size: 2.84 MB - Last synced: about 4 hours ago - Pushed: about 23 hours ago - Stars: 5 - Forks: 0

zhukovanan/Wildberries

Интеграция API Wildberries партнеры с помощью R

Language: R - Size: 2.93 KB - Last synced: about 7 hours ago - Pushed: over 3 years ago - Stars: 2 - Forks: 0

ariacom/Seal-Report

Database Reporting Tool and Tasks (.Net)

Language: C# - Size: 58.4 MB - Last synced: about 8 hours ago - Pushed: about 9 hours ago - Stars: 1,357 - Forks: 458

evdubs/oic-options-chains

ETL for OIC Options Chains

Language: Racket - Size: 52.7 KB - Last synced: about 8 hours ago - Pushed: about 8 hours ago - Stars: 26 - Forks: 8

xyflow/awesome-node-based-uis

A curated list with resources about node-based UIs

Size: 144 KB - Last synced: about 4 hours ago - Pushed: 3 months ago - Stars: 1,902 - Forks: 100

turbot/steampipe

Zero-ETL, infinite possibilities. Live query APIs, code & more with SQL. No DB required.

Language: Go - Size: 21.6 MB - Last synced: about 5 hours ago - Pushed: 1 day ago - Stars: 6,391 - Forks: 245

astronomer/astro-sdk

Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.

Language: Python - Size: 7.19 MB - Last synced: about 15 hours ago - Pushed: about 16 hours ago - Stars: 317 - Forks: 39

mage-ai/mage-ai

🧙 Build, run, and manage data pipelines for integrating and transforming data.

Language: Python - Size: 170 MB - Last synced: 14 days ago - Pushed: 14 days ago - Stars: 6,940 - Forks: 616

aryn-ai/sycamore

🍁 Sycamore is an LLM-powered search and analytics platform for unstructured data.

Language: Python - Size: 52.5 MB - Last synced: about 3 hours ago - Pushed: about 16 hours ago - Stars: 160 - Forks: 15

Sage-Bionetworks/Genie

Validation and processing of GENIE files

Language: Python - Size: 2.76 MB - Last synced: about 19 hours ago - Pushed: 1 day ago - Stars: 12 - Forks: 8

jinjernot/leliel

Build dynamic product pages by fetching data from HP's API.

Language: HTML - Size: 361 KB - Last synced: about 17 hours ago - Pushed: about 19 hours ago - Stars: 0 - Forks: 0

datacoves/docs

Datacoves public documentation

Size: 32.3 MB - Last synced: about 20 hours ago - Pushed: 1 day ago - Stars: 2 - Forks: 0

bgarrak/covid-19-afterwardz

This project manipulates and transports NYT Covid-19 data to a SSMS server for use in a Tableau project: Tracking Covid-19 Across Time and County

Language: R - Size: 57.6 KB - Last synced: about 21 hours ago - Pushed: almost 3 years ago - Stars: 0 - Forks: 0

CityOfPhiladelphia/databridge-etl-tools

Command line ETL tools for Databridge

Language: Python - Size: 505 KB - Last synced: about 21 hours ago - Pushed: about 22 hours ago - Stars: 0 - Forks: 0

quadratichq/quadratic

Quadratic | Data Science Spreadsheet with Python & SQL

Language: TypeScript - Size: 1.13 GB - Last synced: about 15 hours ago - Pushed: about 19 hours ago - Stars: 2,717 - Forks: 128

Aliviarahma/ETL-Banksim-Artificial

ETL of Banksim Artificial

Language: Jupyter Notebook - Size: 23.4 KB - Last synced: about 22 hours ago - Pushed: about 23 hours ago - Stars: 0 - Forks: 0

dataforgelabs/dataforge-core

DataForge helps data teams write functional transformation pipelines by leveraging software engineering principals

Language: PLpgSQL - Size: 288 KB - Last synced: about 21 hours ago - Pushed: about 22 hours ago - Stars: 5 - Forks: 0

nucleuscloud/neosync

Open source data anonymization and synthetic data orchestration for developers. Create high fidelity synthetic data and sync it across your environments.

Language: TypeScript - Size: 56.2 MB - Last synced: about 17 hours ago - Pushed: about 17 hours ago - Stars: 613 - Forks: 38

tonper19/ballgame

European Baseball and Softball Statistics Data warehouse and BI application

Language: Python - Size: 238 MB - Last synced: about 23 hours ago - Pushed: about 24 hours ago - Stars: 0 - Forks: 0

blockchain-etl/ethereum-etl-airflow

Airflow DAGs for exporting, loading, and parsing the Ethereum blockchain data. How to get any Ethereum smart contract into BigQuery https://towardsdatascience.com/how-to-get-any-ethereum-smart-contract-into-bigquery-in-8-mins-bab5db1fdeee

Language: Python - Size: 2.76 MB - Last synced: about 16 hours ago - Pushed: about 23 hours ago - Stars: 387 - Forks: 180

blockchain-etl/ethereum-etl

Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions. Data is available in Google BigQuery https://goo.gl/oY5BCQ

Language: Python - Size: 1.92 MB - Last synced: about 23 hours ago - Pushed: 16 days ago - Stars: 2,823 - Forks: 794

kharigardner/pyfivetran

Simple python interface for the Fivetran API. Powered by HTTPx.

Language: Python - Size: 124 KB - Last synced: about 22 hours ago - Pushed: about 23 hours ago - Stars: 2 - Forks: 0

MobileTeleSystems/horizon-hwm-store

Horizon HWM Store for onETL

Language: Python - Size: 146 KB - Last synced: about 22 hours ago - Pushed: about 23 hours ago - Stars: 3 - Forks: 0

PeerDB-io/peerdb

Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage

Language: Go - Size: 10.1 MB - Last synced: about 23 hours ago - Pushed: about 23 hours ago - Stars: 1,621 - Forks: 47

dagster-io/dagster

An orchestration platform for the development, production, and observation of data assets.

Language: Python - Size: 933 MB - Last synced: about 23 hours ago - Pushed: about 23 hours ago - Stars: 10,213 - Forks: 1,264

MobileTeleSystems/etl-entities

Basic ETL Entity classes for onETL

Language: Python - Size: 479 KB - Last synced: about 23 hours ago - Pushed: 1 day ago - Stars: 4 - Forks: 0

zhaoyachao/zdh_web

大数据采集,抽取平台,zdh_web是zdh系列服务的可视化管理平台，包含数据采集,调度,权限,审批流,私域营销等模块

Language: Java - Size: 138 MB - Last synced: 3 days ago - Pushed: 7 days ago - Stars: 408 - Forks: 156

kestra-io/kestra

Infinitely scalable, event-driven, language-agnostic orchestration and scheduling platform to manage millions of workflows declaratively in code.

Language: Java - Size: 34.4 MB - Last synced: 2 days ago - Pushed: 2 days ago - Stars: 6,340 - Forks: 346

apache/doris

Apache Doris is an easy-to-use, high performance and unified analytics database.

Language: Java - Size: 696 MB - Last synced: 2 days ago - Pushed: 2 days ago - Stars: 11,354 - Forks: 3,045

slingdata-io/sling-cli

Sling is a CLI tool that extracts data from a source storage/database and loads it in a target storage/database.

Language: Go - Size: 72.8 MB - Last synced: about 10 hours ago - Pushed: about 18 hours ago - Stars: 256 - Forks: 10

mozilla/python_mozetl

ETL jobs for Firefox Telemetry

Language: Python - Size: 2.66 MB - Last synced: 22 minutes ago - Pushed: 1 day ago - Stars: 25 - Forks: 35

answerdigital/oxford-omop-data-mapper

A documentation-centric ETL tool, implementing transformations of a number of common clincial datasets into the OMOP CDM.

Language: C# - Size: 21.3 MB - Last synced: 1 day ago - Pushed: 1 day ago - Stars: 3 - Forks: 0

turbot/steampipe-sqlite

Steampipe SQLite is a zero-ETL engine for SQLite. Virtual tables translate queries into live API calls for cloud services and APIs. Hundreds of plugins with thousands of documented examples.

Language: Go - Size: 375 KB - Last synced: 1 day ago - Pushed: 1 day ago - Stars: 43 - Forks: 1

turbot/steampipe-export

Steampipe Export is a zero-ETL CLI to fetch data from cloud services and APIs. Hundreds of plugins with thousands of documented examples.

Language: Go - Size: 347 KB - Last synced: about 2 hours ago - Pushed: 1 day ago - Stars: 4 - Forks: 1

kids-first/kf-lib-data-ingest

🏭 Kids First Data Ingest Library

Language: Python - Size: 11 MB - Last synced: about 12 hours ago - Pushed: about 22 hours ago - Stars: 5 - Forks: 0

devopscorner/nifi

Production Grade Nifi & Nifi Registry. Deploy for VM (Virtual Machine) with Terraform + Ansible, Helm & Helmfile for Kubernetes (EKS)

Language: Python - Size: 12.2 MB - Last synced: about 16 hours ago - Pushed: 1 day ago - Stars: 10 - Forks: 0

geopython/stetl

Stetl, Streaming ETL, is a lightweight geospatial processing and ETL framework written in Python.

Language: Python - Size: 6.89 MB - Last synced: 12 days ago - Pushed: about 1 month ago - Stars: 82 - Forks: 34

atolcd/sdis-remocra

🔥 Remocra - Plateforme métier opensource conçue par et pour les SDIS.

Language: JavaScript - Size: 84 MB - Last synced: 1 day ago - Pushed: 1 day ago - Stars: 15 - Forks: 4

goto/optimus Fork of raystack/optimus

Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.

Language: Go - Size: 25.9 MB - Last synced: 1 day ago - Pushed: 1 day ago - Stars: 3 - Forks: 1

Multiwoven/multiwoven

🔥🔥🔥 Open Source Hightouch, Census, and RudderStack Alternative

Language: Ruby - Size: 4.17 MB - Last synced: about 19 hours ago - Pushed: 1 day ago - Stars: 622 - Forks: 28

aurelienmorgan/abnormal_vibrations_watchdog

Time Series Anomaly detection. The monitored signal is made-up of machinery vibration sensor measurements.

Language: Python - Size: 8.71 MB - Last synced: 1 day ago - Pushed: over 3 years ago - Stars: 12 - Forks: 2

ConduitIO/conduit

Conduit streams data between data stores. Kafka Connect replacement. No JVM required.

Language: Go - Size: 9.87 MB - Last synced: about 15 hours ago - Pushed: about 16 hours ago - Stars: 345 - Forks: 41

turbot/steampipe-plugin-linkedin

Use SQL to instantly query LinkedIn for profiles, companies, connections & more. Open source CLI. No DB required.

Language: Go - Size: 319 KB - Last synced: 13 days ago - Pushed: 15 days ago - Stars: 11 - Forks: 0

turbot/steampipe-plugin-consul

Use SQL to instantly query Consul ACLs, namespaces, services & more. Open source CLI. No DB required.

Language: Go - Size: 198 KB - Last synced: 13 days ago - Pushed: 15 days ago - Stars: 1 - Forks: 0

turbot/steampipe-plugin-pipes

Use SQL to instantly query Turbot Pipes resources across workspaces. Open source CLI. No DB required.

Language: Go - Size: 326 KB - Last synced: 13 days ago - Pushed: 15 days ago - Stars: 2 - Forks: 0

chronicle-app/chronicle-etl

📜 A CLI toolkit for extracting and working with your digital history

Language: Ruby - Size: 396 KB - Last synced: 1 day ago - Pushed: 1 day ago - Stars: 119 - Forks: 2

halestudio/hale

(Spatial) data harmonisation with hale studio (formerly HUMBOLDT Alignment Editor)

Language: Java - Size: 348 MB - Last synced: 1 day ago - Pushed: 2 days ago - Stars: 138 - Forks: 44

bpolaszek/bentools-etl

PHP ETL (Extract / Transform / Load) library with SOLID principles + almost no dependency.

Language: PHP - Size: 333 KB - Last synced: about 5 hours ago - Pushed: 4 months ago - Stars: 120 - Forks: 4

google/fhir-data-pipes

A collection of tools for extracting FHIR resources and analytics services on top of that data.

Language: Java - Size: 383 MB - Last synced: about 20 hours ago - Pushed: 1 day ago - Stars: 125 - Forks: 77

flow-php/etl

PHP - ETL (Extract Transform Load) data processing library

Language: PHP - Size: 2.68 MB - Last synced: about 23 hours ago - Pushed: 2 days ago - Stars: 339 - Forks: 21

bcgsc/pori_graphkb_loader

The Loaders for GraphKB. Imports content from external sources via the GraphKB REST API

Language: JavaScript - Size: 9.61 MB - Last synced: 2 days ago - Pushed: 3 days ago - Stars: 6 - Forks: 4

Phelipe-Sempreboni/tutorials-informations-notes

Repository for tutorials, information and notes on technology in general.

Language: Python - Size: 34.4 MB - Last synced: 2 days ago - Pushed: 2 days ago - Stars: 1 - Forks: 0

dataform-co/dataform

Dataform is a framework for managing SQL based data operations in BigQuery

Language: TypeScript - Size: 15.5 MB - Last synced: 2 days ago - Pushed: 2 days ago - Stars: 790 - Forks: 145

wherobots/airflow-providers-wherobots

Airflow extensions for communicating with Wherobots Cloud

Language: Python - Size: 127 KB - Last synced: 1 day ago - Pushed: 2 days ago - Stars: 0 - Forks: 0

MarceloAtencio/Videogames_Project

Primer proyecto individual de HENRY

Language: Jupyter Notebook - Size: 208 MB - Last synced: 2 days ago - Pushed: 2 days ago - Stars: 0 - Forks: 0

catalyst-cooperative/pudl

The Public Utility Data Liberation Project provides analysis-ready energy system data to climate advocates, researchers, policymakers, and journalists.

Language: Python - Size: 409 MB - Last synced: 3 days ago - Pushed: 4 days ago - Stars: 443 - Forks: 99

instill-ai/helm-charts

⎈ The Helm charts of Instill AI

Size: 146 KB - Last synced: 2 days ago - Pushed: 2 days ago - Stars: 2 - Forks: 1

atlas-bi/LDAP-ETL

⚙️ Atlas ETL | LDAP User Profiles

Language: Python - Size: 709 KB - Last synced: 2 days ago - Pushed: 2 days ago - Stars: 2 - Forks: 1

night-fury-me/Methods-of-Advanced-Data-Engineering Fork of jvalue/made-template

A repository for the Methods of Advanced Data Engineering course at FAU

Size: 1.46 MB - Last synced: 2 days ago - Pushed: 2 days ago - Stars: 0 - Forks: 0

AbsaOSS/cobrix

A COBOL parser and Mainframe/EBCDIC data source for Apache Spark

Language: Scala - Size: 4.61 MB - Last synced: 2 days ago - Pushed: 2 days ago - Stars: 133 - Forks: 79

Cinchoo/ChoETL

ETL framework for .NET (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)

Language: C# - Size: 39.6 MB - Last synced: about 12 hours ago - Pushed: 2 days ago - Stars: 735 - Forks: 132

SwapnilGit31/Patient-HealthCare-Dashborad-using-PowerBI

Powerful Patient HealthCare Dashboard built with Power BI. Visualize demographics, medical history & treatment progress for informed decision-making. Improve patient care outcomes efficiently

Size: 0 Bytes - Last synced: 2 days ago - Pushed: 2 days ago - Stars: 0 - Forks: 0

schemamap/schemamap

The Postgres-native data movement platform for multi-tenant SaaS apps

Language: Clojure - Size: 13.7 MB - Last synced: 1 day ago - Pushed: 2 days ago - Stars: 5 - Forks: 0

RADar-AZDelta/Rabbit-in-a-Blender

An ETL pipeline to transform your EMP data to OMOP

Language: Python - Size: 3.28 MB - Last synced: 2 days ago - Pushed: 2 days ago - Stars: 7 - Forks: 2

superlinked/superlinked

A compute framework for turning complex data into vectors.

Language: Jupyter Notebook - Size: 10.5 MB - Last synced: 2 days ago - Pushed: 2 days ago - Stars: 148 - Forks: 6

sh16ma/gitpress

TIL(=Today I learned.)

Size: 1.5 MB - Last synced: 2 days ago - Pushed: 2 days ago - Stars: 1 - Forks: 1

173TECH/sayn

Data processing and modelling framework for automating tasks (incl. Python & SQL transformations).

Language: Python - Size: 4.52 MB - Last synced: 2 days ago - Pushed: 2 days ago - Stars: 117 - Forks: 14

data-solution-automation-engine/DIRECT

DIRECT, the Data Integration Run-time Execution Control Tool, is a data logistics control framework that can be used to monitor, log, audit and control data integration / ETL processes.

Language: TSQL - Size: 12.7 MB - Last synced: 2 days ago - Pushed: 2 days ago - Stars: 25 - Forks: 8

tomexiskandar/bintang

A tiny and temporary db for quick data cleansing and transformation. It is a high-level python coding and would help any Pythonista up to speed with ETL work.

Language: Python - Size: 171 KB - Last synced: 1 day ago - Pushed: 2 days ago - Stars: 2 - Forks: 1

turbot/steampipe-plugin-virustotal

Use SQL to instantly query file, domain, URL and IP scanning results from VirusTotal.

Language: Go - Size: 31.3 MB - Last synced: 13 days ago - Pushed: 15 days ago - Stars: 22 - Forks: 1

JimeFioni/Deaths-of-Drugs-Overdose-

Este repositorio contiene un análisis exhaustivo de las sobredosis de opioides en Estados Unidos, un problema de salud pública que ha alcanzado proporciones alarmantes en las últimas décadas. A través de una combinación de datos estadísticos, visualizaciones informativas y análisis en profundidad. 💉🩺

Language: Jupyter Notebook - Size: 50.4 MB - Last synced: 3 days ago - Pushed: about 2 months ago - Stars: 1 - Forks: 2

gsiems/bac-tract

Extract data from bacpac files

Language: Go - Size: 85.9 KB - Last synced: 3 days ago - Pushed: 3 days ago - Stars: 5 - Forks: 3

AMPATH/etl-rest-server

This project hosts scripts to generate flat tables used for reporting purposes.

Language: JavaScript - Size: 9.2 MB - Last synced: 3 days ago - Pushed: 5 days ago - Stars: 10 - Forks: 51

polarbeargo/Data-Engineering-Capstone-Project

Language: Jupyter Notebook - Size: 834 KB - Last synced: 3 days ago - Pushed: over 2 years ago - Stars: 1 - Forks: 0

SpareCores/sc-crawler

Pull and standardize data on cloud compute resources.

Language: Python - Size: 1.64 MB - Last synced: 3 days ago - Pushed: 3 days ago - Stars: 1 - Forks: 0

mara/mara-pipelines

A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow

Language: Python - Size: 3.29 MB - Last synced: 1 day ago - Pushed: 4 months ago - Stars: 2,053 - Forks: 102

hawa1222/data-stream-etl

Python-MySQL ETL pipeline to centralise personal data from sources like YouTube and Apple into a structured database, enabling advanced data analysis and application development.

Language: Python - Size: 246 KB - Last synced: 3 days ago - Pushed: 3 days ago - Stars: 0 - Forks: 0

JumpMind/metl

Metl is a simple, web-based integration platform that allows for several different styles of data integration including messaging, file based Extract/Transform/Load (ETL), and remote procedure invocation via Web Services. Read more at www.jumpmind.com/products/metl/overview

Language: Java - Size: 24.9 MB - Last synced: 3 days ago - Pushed: 3 days ago - Stars: 202 - Forks: 82