GitHub topics: etl-automation | Ecosyste.ms: Repos

superglue-ai/superglue

Self-healing open source data connector. Use it as a layer between you and any complex / legacy APIs and always get the data that you want in the format you expect.

Language: TypeScript - Size: 1.41 MB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 1,691 - Forks: 74

ETL system utilizing the DuxSoup API for programmatic LinkedIn extraction. The project is a data extraction pipeline that automatically retrieves extensive LinkedIn profile data from first-degree connections, processes the data, and stores it in MongoDB Atlas. The system utilizes Render's web service to run code persistent in the background.

Language: JavaScript - Size: 385 KB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 1 - Forks: 0

redis-field-engineering/redis-connect-dist

Real-Time Event Streaming & Change Data Capture

Language: Shell - Size: 37.8 MB - Last synced at: 14 days ago - Pushed at: about 2 months ago - Stars: 48 - Forks: 11

TheCocoTeam/source-watcher-core

This is a PHP project which combines ETL with different strategies to extract data from multiple databases, files, and services, transform it and load it into multiple destinations.

Language: PHP - Size: 1.29 MB - Last synced at: 18 days ago - Pushed at: 18 days ago - Stars: 9 - Forks: 0

restarone/violet_rails

an app engine for your business. Seamlessly implement business logic with a powerful API. Out of the box CMS, blog, forum and email functionality. Developer friendly & easily extendable for your next SaaS/XaaS project. Built with Rails 6, Devise, Sidekiq & PostgreSQL

Language: Ruby - Size: 38 MB - Last synced at: 17 days ago - Pushed at: 5 months ago - Stars: 100 - Forks: 43

velocitybolt/open-extract

Structured Data Extractor for AI Agents. Search your documents or the web for specific data and get it back in JSON or Markdown in a single tool call.

Language: Python - Size: 8.91 MB - Last synced at: 23 days ago - Pushed at: 23 days ago - Stars: 162 - Forks: 13

crate/cratedb-fivetran-destination

CrateDB Fivetran Destination connector, for loading data into CrateDB.

Language: Python - Size: 44.9 KB - Last synced at: 24 days ago - Pushed at: 24 days ago - Stars: 0 - Forks: 0

FadhiilDzaki/etl_superstore

This project automates ETL for Superstore data, extracting from PostgreSQL, transforming in Python, and reloading into PostgreSQL weekly. I conducted data analysis in Jupyter Notebook and built a Metabase dashboard for insights.

Language: Jupyter Notebook - Size: 0 Bytes - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 0 - Forks: 0

data-solution-automation-engine/DIRECT

DIRECT, the Data Integration Run-time Execution Control Tool, is a data logistics control framework that can be used to monitor, log, audit and control data integration / ETL processes.

Language: TSQL - Size: 14.8 MB - Last synced at: 4 days ago - Pushed at: about 1 month ago - Stars: 26 - Forks: 9

imsanjoykb/ETL-Project

The goal of this project is to illustrate Extract Transform Load (ETL) using Python and SQL. ETL is a process commonly done in computing, which takes raw data, cleans it and stores it for later use. The extraction phase targets and retrieves the data. Transform manipulates and cleans the data. Then load stores the data, typically in a data warehouse.

Language: Jupyter Notebook - Size: 285 KB - Last synced at: 16 days ago - Pushed at: over 3 years ago - Stars: 22 - Forks: 9

FadhiilDzaki/ETL_Inventory_Management

Optimized inventory management by automating ETL pipelines with Airflow and Elasticsearch, and developed Kibana dashboards to monitor inventory levels and trends to ensure efficient stock management by reducing out-of-stock and overstock issues, ultimately improving operational efficiency and cost management.

Language: Jupyter Notebook - Size: 4.19 MB - Last synced at: 17 days ago - Pushed at: about 1 month ago - Stars: 0 - Forks: 0

data-solution-automation-engine/virtual-data-warehouse

The Virtual Data Warehouse is a code generation and template management tool. It is part of the data solution automation ecosystem - the 'engine' for data solution automation.

Language: Handlebars - Size: 4.51 MB - Last synced at: 4 days ago - Pushed at: about 1 month ago - Stars: 45 - Forks: 13

data-solution-automation-engine/data-warehouse-automation-metadata-schema

Generic interface exchange format for Data Warehouse Automation and ETL generation.

Language: C# - Size: 23 MB - Last synced at: 13 days ago - Pushed at: 10 months ago - Stars: 41 - Forks: 12

aaduhalde/Databases

Size: 36.5 MB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 0 - Forks: 0

potreic/ETL-Fashion-Trend-Analysis

✨ Automate fashion trend analysis with Apache Airflow! Extract data from X & Pinterest, transform into insights, and load into PostgreSQL. Predict seasonal styles & visualize trends. 💃📊

Language: Python - Size: 168 KB - Last synced at: 16 days ago - Pushed at: 5 months ago - Stars: 2 - Forks: 0

LIoccoUMD/ETL-Analysis

This project automates ETL for gym exercise data, predicting safety scores using KNN and optimizing with GridSearchCV. It generates recommendations, statistical summaries, and visualizations to improve gym safety and client retention. Logging ensures transparency.

Language: Python - Size: 1.72 MB - Last synced at: 15 days ago - Pushed at: about 2 months ago - Stars: 2 - Forks: 0

ManoharVit/Stock-CloseCast

Used MLOps for automation and monitoring stock market analysis with forecasting combing data from various sources like FRED, Fama-French data developing ETL pipeline with GCP, GitHub Actions, DVC and Airflow

Language: Jupyter Notebook - Size: 30.9 MB - Last synced at: 16 days ago - Pushed at: 4 months ago - Stars: 1 - Forks: 0

An4PDM/ETL-process-notifier

Projeto de automação de ETL com notificadores para sucesso ou falha no processamento.

Language: Python - Size: 191 KB - Last synced at: 18 days ago - Pushed at: 2 months ago - Stars: 0 - Forks: 0

zulfaan/intro-to-data-engineer-project_pacmann

Proyek ini adalah pembuatan pipeline ETL untuk mengelola data dari berbagai sumber, termasuk PostgreSQL, file CSV, dan web scraping dari Lazada serta Tokopedia. Data diekstraksi, dibersihkan, dan ditransformasikan sebelum dimuat ke dalam database PostgreSQL untuk analisis lebih lanjut, mendukung kebutuhan tim di Perusahaan XYZ.

Language: Jupyter Notebook - Size: 23.5 MB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

Nel-zi/Nuga_bank

Developed an automated data exploration and cleaning pipeline for Nuga Bank to streamline data preparation, ensure consistent data quality, and normalize datasets into structured databases for efficient analysis and reporting.

Language: Jupyter Notebook - Size: 616 KB - Last synced at: 2 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

hyslan/Val

Sistema Val.

Language: Python - Size: 94.1 MB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 2 - Forks: 0

mikeroyal/Apache-Arrow-Guide

Apache Arrow Guide

Size: 160 KB - Last synced at: 22 days ago - Pushed at: over 3 years ago - Stars: 10 - Forks: 4

Birmingham-and-Solihull-ICS/WorkforceETL

This repository is used to store scripts for processing monthly provider workforce returns and temporary (agency) staffing returns.

Language: R - Size: 26.4 KB - Last synced at: about 1 month ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0

OndraZizka/csv-cruncher

Treats CSV and JSON files as SQL tables, and exports SQL SELECTs back to CSV or JSON.

Language: Kotlin - Size: 13.1 MB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 44 - Forks: 12

vijeni/cyberleak-etl-process

Script de ETL para análise de vazamento de dados

Language: Python - Size: 140 KB - Last synced at: 18 days ago - Pushed at: 18 days ago - Stars: 1 - Forks: 0

skyffel/airbyte-connector-generator-poc

proof of concept to generate Airbyte low-code YAML connectors from API documentation

Language: Python - Size: 184 KB - Last synced at: 5 months ago - Pushed at: about 1 year ago - Stars: 22 - Forks: 2

SitiHassan/UEC-Performance

An ETL tool to extract daily raw local submissions of Urgent and Emergency Care (UEC) performance metrics.

Language: R - Size: 29.3 KB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 0 - Forks: 0

Ibrahim-Maiga/Big-Data-Analysis-with-PySpark

A comprehensive big data analysis examining correlations between temperature changes and societal metrics (crime rates, birth rates, and energy consumption) across the US and Canada. The project leverages multiple database systems and cloud computing to process and analyze large-scale climate and social data.

Language: Jupyter Notebook - Size: 454 KB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 0 - Forks: 0

herianc/dados-ar-rj

Sistema de Informação dos dados de poluição do Rio

Language: Python - Size: 11.7 MB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 1 - Forks: 0

TheNJineer/NJRealtor-Scrapper

Full scale portfolio project which scrapes the NJ Realtor website of its monthly median sales data pdfs. The pdf's contents will be extracted to be cleaned and transformed to then be stores in a SQL data base for future use in a machine learning project.

Language: Jupyter Notebook - Size: 3.37 MB - Last synced at: 6 months ago - Pushed at: 6 months ago - Stars: 0 - Forks: 0

visiologyofficial/vixtract

Language: HTML - Size: 916 KB - Last synced at: 6 months ago - Pushed at: 6 months ago - Stars: 44 - Forks: 13

jibbs1703/Weather-Gas-ETL-Pipeline

This repository contains a in ETL pipeline for collecting, transforming and storing hourly weather and atmospheric gas data. The pipeline leverages Docker containerization, AWS cloud infrastructure resources and is orchestrated using Apache-Airflow.

Language: Python - Size: 17.6 KB - Last synced at: about 1 month ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

Seifo321/Microsoft-Data-Engineer-Project

Leveraging Microsoft AZURE Services , DEVELOPING a high performance ETL pipeline that extracts and transform the BikeStores data and loads it to Azure data warehouse

Language: HTML - Size: 8.05 MB - Last synced at: 24 days ago - Pushed at: 6 months ago - Stars: 0 - Forks: 1

aws-samples/amazon-redshift-serverless-rsql-etl-framework

Amazon Redshift Serverless RSQL ETL Framework

Language: TypeScript - Size: 1.08 MB - Last synced at: 6 months ago - Pushed at: 6 months ago - Stars: 7 - Forks: 2

EvilLord666/ReportGenerator

A small cross-database tool for building excel documents (reports) based on data from database that extacts via View or Stored Procedures with parametres, ordering e.t.c.

Language: C# - Size: 500 KB - Last synced at: 12 days ago - Pushed at: almost 3 years ago - Stars: 7 - Forks: 6

SanjinKurelic/AntennaDistribution

Antenna Distribution is a project that shows how to run business analysis tools on a set of a data.

Language: TSQL - Size: 23.5 MB - Last synced at: 2 months ago - Pushed at: about 3 years ago - Stars: 2 - Forks: 0

SitiHassan/Provider-Workforce

An ETL script to process the monthly raw submissions of NHS Provider workforce data.

Language: R - Size: 10.7 KB - Last synced at: 8 months ago - Pushed at: 8 months ago - Stars: 0 - Forks: 0

SitiHassan/CVD-Prevention

An ETL tool to scrape data from the CVD Prevent website using an API.

Language: R - Size: 5.86 KB - Last synced at: 8 months ago - Pushed at: 8 months ago - Stars: 0 - Forks: 0

wambugu71/tommorow.io_Data

Fetch data weekly using ETL approach from tommorow.io weather forecast data.

Language: Python - Size: 42 KB - Last synced at: 26 days ago - Pushed at: 8 months ago - Stars: 0 - Forks: 0

Sharifah-Malhan/ETL-with-pyspark

python script for ETL job with pyspark, both the source and destination databses are MySQL (the spark job is embedded into flask for the sake of deployment)

Language: Python - Size: 3.91 KB - Last synced at: about 2 months ago - Pushed at: 8 months ago - Stars: 0 - Forks: 0

mnassrib/sales-data-warehouse-project

This project is a data warehouse solution for sales analysis, featuring an automated ETL pipeline to process, clean, and load sales data into a PostgreSQL database. It integrates Metabase and Superset for advanced data visualization, enabling detailed insights into sales performance and profitability.

Language: Python - Size: 558 KB - Last synced at: 8 months ago - Pushed at: 8 months ago - Stars: 0 - Forks: 0

YoungSong99/city-finder-data-ETL

CityFinder project

Size: 2.8 MB - Last synced at: about 2 months ago - Pushed at: 8 months ago - Stars: 0 - Forks: 0

joshua-dada-mayowa/Data-Engineer-ETL-Project

ETL Project using Airflow, DBT, Bigquery,Postgres,Docker and Docker Compose

Language: Python - Size: 41.3 MB - Last synced at: 8 months ago - Pushed at: 8 months ago - Stars: 0 - Forks: 0

Reddi-Srija-R/SQLite3

Automated Data Extraction And Database Integration

Language: Python - Size: 9.77 KB - Last synced at: 8 months ago - Pushed at: 8 months ago - Stars: 0 - Forks: 0

Oficialsac/ETL-Automation-Tool

This tool is designed to streamline and automate ETL (Extract, Transform, Load) processes sequentially. It provides a comprehensive analytical package that centralizes all ETL tasks in one place, simplifying the management and optimization of data workflows.

Language: Python - Size: 24.4 KB - Last synced at: 8 months ago - Pushed at: 9 months ago - Stars: 1 - Forks: 0

elrf3lipes/Python_Automation_Projects

Scripts to automate general time-consuming tasks

Language: Jupyter Notebook - Size: 42 KB - Last synced at: about 2 months ago - Pushed at: 9 months ago - Stars: 0 - Forks: 0

mikeAdamss/tidychef

Python framework for transforming tabulated data with visual relationships into tidy data

Language: Jupyter Notebook - Size: 16.3 MB - Last synced at: 9 months ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

charliemarshall1996/subscriber-cancellation-pipeline

Creation of ETL pipeline to manage cancellations of subscribers for any online education platform.

Language: Jupyter Notebook - Size: 1.18 MB - Last synced at: 9 months ago - Pushed at: 9 months ago - Stars: 0 - Forks: 0

techysanjo/Extract-Transform-and-Load-ETL-Project

Developed an Extract, Transform and Load (ETL) program to extract dataset from various types of sources, applied various transformation techniques and loaded to various destination types.

Language: Jupyter Notebook - Size: 771 KB - Last synced at: 10 months ago - Pushed at: 10 months ago - Stars: 1 - Forks: 0

nl2go/hetzner-invoice

Automatically download and transform Hetzner invoices.

Language: Python - Size: 59.6 KB - Last synced at: 17 days ago - Pushed at: almost 5 years ago - Stars: 12 - Forks: 1

Wissance/ReportGeneratorWebGui

An ASP NET MVC 6 Web GUI (Net core) for easy reports generation using ReportGenerator

Language: C# - Size: 2.14 MB - Last synced at: 17 days ago - Pushed at: almost 2 years ago - Stars: 8 - Forks: 9

heliomarpm/SQLDataTransfer

Ferramenta para Cópia de Dados SQL Server, que foi desenvolvida para auxiliar na geração de arquivos e cópia eficiente de dados entre bases de dados SQL Server.

Language: C# - Size: 4.84 MB - Last synced at: 23 days ago - Pushed at: 11 months ago - Stars: 1 - Forks: 2

OTRABAZOS/RealTimeNews_GoogleWorkspace

Optimize your marketing agency's data workflow with this repository, focusing solely on news integration. Leverage Google Workspace and TrawlingWeb for efficient news acquisition and analysis, transforming complex information into actionable insights with fast, autonomous processes.

Language: JavaScript - Size: 95.7 KB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 1 - Forks: 0

al-ghaly/RuleEngineUsingScala

A Rule-Based discount calculating engine for a retail store. The engine reads transaction data, applies various discount rules based on the product, date, and other criteria, and writes the results back to a database.

Language: Scala - Size: 259 KB - Last synced at: 29 days ago - Pushed at: 11 months ago - Stars: 0 - Forks: 0

vaugood/swimport

An ETL templating system that takes the busywork out of your data migrations.

Language: Python - Size: 19.5 KB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 0 - Forks: 0

cybergeekgyan/Data-Engineering-Portfolio

Data Engineering portfolio projects, resources used to study data tools...

Language: Jupyter Notebook - Size: 2.92 MB - Last synced at: 12 months ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 0

kyaiooiayk/ETL-and-ML-Pipelines-Notes

Notes, tutorials, code snippets and templates focused on ETL pipelines for Machine Learning

Language: Jupyter Notebook - Size: 3.71 MB - Last synced at: 5 months ago - Pushed at: 12 months ago - Stars: 0 - Forks: 0

omega1x/stmik

Real-time connector to BTSK-telemetry service

Language: Java - Size: 49.8 KB - Last synced at: 12 months ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0

rohitkulkarni08/Azure-ETL-AmazonSalesAnalysis

A comprehensive ETL pipeline and sales analysis project leveraging Microsoft Azure and PySpark, designed to optimize e-commerce sales by providing actionable insights through detailed data analysis.

Language: Jupyter Notebook - Size: 8.04 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

rohitkulkarni08/Azure-ETL-Pipeline-MovieAnalytics

This project demonstrates an ETL pipeline using Microsoft Azure for IMDb Movie Rating Dataset analysis. It covers data extraction from Azure Blob Storage, transformation with Azure Databricks, and loading into Azure SQL using Azure Data Factory. The pipeline automates insights generation and is a practical example of cloud-based data engineering.

Language: Jupyter Notebook - Size: 15.9 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

stevehoober254/ETL_Data_Pipeline_For_Retail_Store

ETL (Extract, Transform, Load) pipeline to integrate sales data from various sources into a central data warehouse

Language: Python - Size: 5.86 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 0

MasterMindRomii/Awesome-Chocolate-Sales-Insights

Welcome to the Awesome Chocolate Sales Insights repository! ❤️

Size: 2.6 MB - Last synced at: 8 days ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

Dev-analysis/Four-year-sales-report

Power BI sales project. Creating an automated Power BI dashboard.

Size: 12.2 MB - Last synced at: about 1 year ago - Pushed at: almost 3 years ago - Stars: 0 - Forks: 0

rizkyirw/Pipeline-Project

Resource for ETL & Data Ingestion program using Apache Airflow

Language: Python - Size: 207 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

wonderakwei/Automated-Traffic-Data-ETL-Project

Automated Traffic Data ETL: Python scripts convert, reformat, and upload traffic data to BigQuery via GCS. Terraform ensures efficient resource provisioning, and APScheduler automates daily cron jobs.

Language: Python - Size: 616 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

rickluizms/price-miner

Ferramenta dedicada a descobrir promoções diárias oferecidas por fornecedores online.

Size: 58.6 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

jmoc3/ETL_with_Airflow

Automatizacion de la extracción, transformación y carga de datos desde una API a bases de datos como MySQL. PostgreSQL y Redis.

Language: Python - Size: 734 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 3 - Forks: 0

TheNJineer/GSMLS-Analysis

Full Scale portfolio project used to aggregate sales data from the GSMLS (quarterly), clean and transform the data and store in a SQL database for future machine learning analysis

Language: Python - Size: 71.3 KB - Last synced at: 12 months ago - Pushed at: 12 months ago - Stars: 0 - Forks: 0

wlopezm-unal/Machine-learning

Modelos de machine learning. you can see different notebook where i worked with machine learning model, data exploring data cleaning.

Language: Jupyter Notebook - Size: 1.36 MB - Last synced at: about 2 months ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0

pavelmaksimov/FlowMaster

ETL flow framework based on Yaml configs in Python

Language: Python - Size: 606 KB - Last synced at: 12 months ago - Pushed at: over 1 year ago - Stars: 21 - Forks: 3

JonFillip/transloc_api_gcp_pipeline

Stream data directly from an API using Apache Beam to BigQuery.

Language: Python - Size: 40.9 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

iambhat/python-script-to-fetch-html-data

It's an python script used in one of the project to access the data from html page using beautiful soup.

Language: Python - Size: 8.79 KB - Last synced at: over 1 year ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 0

Olamylo/CI-CD-with-Azure-Pipelines

This Repo shows how to build a CI/CD process with Azure Pipelines. A python script places in an Azure Repo which extracts data from an API and exports the data to a blob on Azure cloud is utilized for this process.

Language: Python - Size: 7.81 KB - Last synced at: over 1 year ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 0

DanieleDepiro/ETL_pipeline

This project is an example of how to create a basic ETL pipeline, including web scraping, data transformation using pandas, and data loading into a pgAdmin4 database.

Language: Python - Size: 8.79 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

DATASCIENTISTSHENRY/PF_DataScience_Migraciones

Repositorio para Proyecto Final de Data Science en bootcamp Henry, se analizan los datos de migraciones a nivel mudial y nacional. Aplicando un stack tecnológico como Google Cloud Platform, con Machine learning, presentación de KPIs y visualizaciones en PowerBi

Language: Jupyter Notebook - Size: 124 MB - Last synced at: about 1 year ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 2

seyedmahdiamin1998/ETL_catawiki

ETL : Extract --> transform --> load

Language: Python - Size: 260 KB - Last synced at: over 1 year ago - Pushed at: about 2 years ago - Stars: 1 - Forks: 0

MarcoZazzini1989/ETL_import_financial_data_to_postgresql

Simple ETL script with Apache Airflow , downloading financial data from alphavantage trought API , trasform into pandas dataframe and uplod to PostgreSQL

Language: Jupyter Notebook - Size: 2.93 KB - Last synced at: about 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 1

matewz/simple_etl_example

Little ETL example. Extracting Data, Store and Visualization

Language: Python - Size: 26.4 KB - Last synced at: over 1 year ago - Pushed at: almost 4 years ago - Stars: 0 - Forks: 0

AnveshJarabani/END-END-ETL-PIPELINES

ETL Pipelines and Dashboard visualizatons

Language: Python - Size: 51.3 MB - Last synced at: about 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

dataopstix/modelt

Modelt(mow·delt) is a modern data integration solution that connects data to data for advanced analytics.

Language: Shell - Size: 463 KB - Last synced at: 24 days ago - Pushed at: almost 3 years ago - Stars: 2 - Forks: 1

smmiri/etl-visuals

Codes for data flow between models, data post-process, and visualization

Language: Jupyter Notebook - Size: 3.81 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

lwdovico/LDS-Project

Repository of a Data Science Project

Language: Jupyter Notebook - Size: 14.7 MB - Last synced at: over 1 year ago - Pushed at: about 2 years ago - Stars: 0 - Forks: 0

Usaid-Bin-Rehan/FAST_Resources_Reverse_Indexing

Search-Engine for FAST-Resources

Language: Jupyter Notebook - Size: 1.66 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 2

AMPATH/etl-flat-table-sync

Sync service for seamless synchronization and transformation of data from AMRS to ETL flat tables

Language: JavaScript - Size: 148 KB - Last synced at: about 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 4

MikeBidinger/Python_ETL

ETL processes using a Tkinter GUI with Python

Language: Python - Size: 101 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

pyprogrammerblog/tiny-blocks

Tiny Blocks to build large and complex data pipelines!

Language: Python - Size: 70.8 MB - Last synced at: 12 days ago - Pushed at: about 1 year ago - Stars: 3 - Forks: 0

timothypesi/ETL-Extract-Transform-Load---using-Pygrametl-

Language: Jupyter Notebook - Size: 2.93 KB - Last synced at: about 2 years ago - Pushed at: about 2 years ago - Stars: 0 - Forks: 0

julientoucoula17/apache_airflow-with-Docker

Apache Airflow installation with Docker 🌬️

Language: Python - Size: 10.7 KB - Last synced at: 2 months ago - Pushed at: almost 3 years ago - Stars: 1 - Forks: 0

moelhaj/elt-pipeline

Extracting data from csv, transforming it, and loading into a Data Warehouse.

Language: Jupyter Notebook - Size: 112 KB - Last synced at: about 2 years ago - Pushed at: over 5 years ago - Stars: 2 - Forks: 0

Pawsanie/Luigi_ETL

Universal Luigi ETL pipeline. Validates data received from external sources. Extracts, transforms them and lands.

Language: Python - Size: 95.7 KB - Last synced at: about 2 months ago - Pushed at: about 2 years ago - Stars: 0 - Forks: 0

mariajosemv/ETL-for-news-websites

♻️ Pipeline for Extract, Transform and Load articles from news websites into an SQLite database.

Language: Python - Size: 7.81 KB - Last synced at: almost 2 years ago - Pushed at: over 4 years ago - Stars: 3 - Forks: 1

SohaT7/Movies-ETL

Creates an automated ETL (Extract, Transform, Load) pipeline that extracts (from three data files), transforms, and loads data into a movies database. Uses Python (Pandas), PostgreSQL, and SQL.

Language: Jupyter Notebook - Size: 15.9 MB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 1 - Forks: 0

OscarTHZhang/docker-airflow Fork of wmorin/docker-airflow-1 📦

Demo for AgDH data pipeline

Language: Python - Size: 31.8 MB - Last synced at: about 2 years ago - Pushed at: over 5 years ago - Stars: 0 - Forks: 0

manoj9788/spark-etl-tests

A sample repository showcasing, implementation of testing for ETL pipeline developed with Apache Spark

Size: 1000 Bytes - Last synced at: about 1 month ago - Pushed at: about 5 years ago - Stars: 1 - Forks: 0

cloudquery/deploy-cq-aws 📦

Cloudformation Template that deploys CloudQuery in an AWS Account

Language: Makefile - Size: 24.4 KB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 0 - Forks: 2