GitHub topics: distributed-storage

NVIDIA/aistore

AIStore: scalable storage for AI applications

Language: Go - Size: 94.9 MB - Last synced at: about 17 hours ago - Pushed at: about 19 hours ago - Stars: 1,617 - Forks: 221

ZettaScaleLabs/zenoh Fork of eclipse-zenoh/zenoh

zenoh unifies data in motion, data in-use, data at rest and computations. It carefully blends traditional pub/sub with geo-distributed storages, queries and computations, while retaining a level of time and space efficiency that is well beyond any of the mainstream stacks.

Language: Rust - Size: 23 MB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 13 - Forks: 4

moosefs/moosefs

MooseFS Distributed Storage – Open Source, Petabyte, Fault-Tolerant, Highly Performing, Scalable Network Distributed File System / Software-Defined Storage

Language: C - Size: 7.07 MB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 1,891 - Forks: 229

cubefs/cubefs

cloud-native distributed storage

Language: Go - Size: 163 MB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 5,293 - Forks: 678

dirvine/communitas

Communitas: A decentralized collaboration platform built on the Saorsa P2P ecosystem. Features four-word addresses, end-to-end encryption, and distributed storage

Language: Rust - Size: 349 MB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 4 - Forks: 1

Agnellalocal723/MyEtcd

🔑 Explore a complete distributed key-value storage system with MyEtcd, built in Go, to learn core principles of distributed systems step by step.

Language: Go - Size: 148 KB - Last synced at: 1 day ago - Pushed at: 2 days ago - Stars: 0 - Forks: 0

storj/storj

Ongoing Storj v3 development. Decentralized cloud object storage that is affordable, easy to use, private, and secure.

Language: Go - Size: 162 MB - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 3,192 - Forks: 418

storacha/storacha.ai

Source code for the storacha.ai microsite

Language: TypeScript - Size: 706 KB - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 0 - Forks: 1

Will436851/Raspberry-pi-OS-Documentation

Raspberry pi OS installation and system call demonstration

Language: Shell - Size: 1.32 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 0 - Forks: 0

kiwek123/lustre

lustre game engine

Size: 4.46 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 0 - Forks: 0

ceph/ceph

Ceph is a distributed object, block, and file storage platform

Language: C++ - Size: 846 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 15,658 - Forks: 6,185

beemesh/beemesh

Global Mesh Computing

Language: Rust - Size: 13.4 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 21 - Forks: 2

leil-io/saunafs

SaunaFS is a free-and open source, distributed POSIX file system inspired by Google File System.

Language: C++ - Size: 16.7 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 94 - Forks: 13

seaweedfs/seaweedfs

SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, xDC replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding. Enterprise version is at seaweedfs.com.

Language: Go - Size: 146 MB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 26,632 - Forks: 2,506

daos-stack/daos

DAOS Storage Stack (client libraries, storage engine, control plane)

Language: C - Size: 170 MB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 887 - Forks: 331

lmco/ChaordicLedger

The ChaordicLedger is the implementation of a design for a combination of Distributed Ledger Technology (DLT) and a Distributed File System (DFS) to create a secure, enterprise-grade platform for storing interlinked project artifacts.

Language: Shell - Size: 123 MB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 9 - Forks: 6

marcelmittelstaedt/BigData

Lecture: Big Data

Language: HTML - Size: 624 MB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 14 - Forks: 14

eclipse-zenoh/zenoh

zenoh unifies data in motion, data in-use, data at rest and computations. It carefully blends traditional pub/sub with geo-distributed storages, queries and computations, while retaining a level of time and space efficiency that is well beyond any of the mainstream stacks.

Language: Rust - Size: 22.7 MB - Last synced at: 5 days ago - Pushed at: 6 days ago - Stars: 2,162 - Forks: 219

gojek/awesome-distributed-systems

Awesome list of distributed systems resources

Size: 824 KB - Last synced at: 5 days ago - Pushed at: over 1 year ago - Stars: 836 - Forks: 110

vadniks/Klenalo

A secured realtime P2P LAN chat and distributed storage manager (desktop linux version)

Language: C - Size: 1.12 MB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 1 - Forks: 0

AutoCSer/AutoCSer2

AutoCSer is an out-of-the-box RPC framework implemented by C#, which has high concurrency, high throughput and other high performance characteristics, and is an ideal distributed application infrastructure.

Language: C# - Size: 525 MB - Last synced at: 3 days ago - Pushed at: 2 months ago - Stars: 31 - Forks: 1

suhasramanand/minigcs

Distributed Object Storage with Erasure Coding and Multi-Region Replication

Language: Python - Size: 24.4 KB - Last synced at: 10 days ago - Pushed at: 10 days ago - Stars: 0 - Forks: 0

sptuan/dist-storage-memo

分布式存储漫游指南

Size: 5.06 MB - Last synced at: 12 days ago - Pushed at: 12 days ago - Stars: 13 - Forks: 0

lustre/lustre-release

Mirror of official Lustre development repository http://git.whamcloud.com/

Language: C - Size: 171 MB - Last synced at: 16 days ago - Pushed at: 17 days ago - Stars: 222 - Forks: 70

m91michel/awesome-blockchain-projects

Curated list of blockchain projects and use cases

Size: 7.81 KB - Last synced at: 18 days ago - Pushed at: over 3 years ago - Stars: 28 - Forks: 7

mikeroyal/Distributed-Systems-Guide

Distributed Systems Guide

Size: 538 KB - Last synced at: 19 days ago - Pushed at: about 4 years ago - Stars: 30 - Forks: 2

zettastor/dbs

ZettaStor DBS provides enterprise-level business storage solutions with high availability, high performance, easy expansion and easy maintenance for large-scale virtualization, private cloud and container environments.

Size: 929 KB - Last synced at: 2 days ago - Pushed at: 10 months ago - Stars: 74 - Forks: 11

freakmaxi/kertish-dos

Kertish-dos is a simple distributed object storage platform, implements object storage on a single distributed computer cluster, and provides interfaces for file/folder handling. Kertish-dos aims primarily for completely distributed operation without a single point of failure, scalable to the exabyte level.

Language: Go - Size: 4.33 MB - Last synced at: 12 days ago - Pushed at: 11 months ago - Stars: 109 - Forks: 9

cqllum/CloudStore

Flask-based multi-cloud file storage system with automatic replication across Local, Mega, Dropbox, AWS S3, and Google Cloud. Features web UI, user management, and RESTful API.

Language: HTML - Size: 90.8 KB - Last synced at: 28 days ago - Pushed at: 28 days ago - Stars: 0 - Forks: 0

baggepinnen/LazyWAVFiles.jl

Lazily treat wav (audio) files as arrays. Arrays can be distributed over many wav files.

Language: Julia - Size: 37.1 KB - Last synced at: 22 days ago - Pushed at: almost 4 years ago - Stars: 6 - Forks: 1

faroukchebaiki/aegis-fs

Encrypted, redundant cloud filesystem: mounts like a disk, encrypts locally, splits into Reed–Solomon shards, and spreads chunks across multiple MEGA accounts—managed by a single master-password vault (Ubuntu & Arch).

Language: Rust - Size: 111 KB - Last synced at: 24 days ago - Pushed at: 30 days ago - Stars: 0 - Forks: 0

jd-opensource/dtc

High performance Distributed Table Cache system designed by JD.com, Offering hotspot data cache from databases in order to reduce pressure of database and improve QPS.

Language: C++ - Size: 26.7 MB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 34 - Forks: 12

flowerinthenight/hedge

A distributed systems library for Kubernetes deployments built on spindle and Cloud Spanner.

Language: Go - Size: 1.17 MB - Last synced at: 24 days ago - Pushed at: about 2 months ago - Stars: 23 - Forks: 4

rotationalio/honu

Adaptive consistency replication with reinforcement learning for large scale globally distributed storage.

Language: Go - Size: 6.74 MB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 13 - Forks: 2

matrixorigin/matrixcube

Fundamental Building Block for Elastic Storage With Strong Consistency and Reliability

Language: Go - Size: 3.96 MB - Last synced at: about 1 month ago - Pushed at: over 3 years ago - Stars: 209 - Forks: 40

mentalfaculty/LLVS

Low-Level Versioned Store

Language: Swift - Size: 1.44 MB - Last synced at: 26 days ago - Pushed at: about 2 years ago - Stars: 207 - Forks: 9

zuoyebang/bitalosdb

Bitalosdb is a high-performance KV storage engine.

Language: Go - Size: 2.49 MB - Last synced at: 2 months ago - Pushed at: 2 months ago - Stars: 164 - Forks: 16

nicomwong/multi-paxos-distributed-database

Distributed key-value database via Multi-Paxos Consensus Algorithm with fault-tolerance and safety

Language: Python - Size: 66.4 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 1 - Forks: 0

kmgowda/SBK

Storage Benchmark Kit

Language: Java - Size: 26.1 MB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 33 - Forks: 64

bsc-dom/dataclay

Python distributed data store that enables remotely access and method execution.

Language: Python - Size: 20.1 MB - Last synced at: about 1 month ago - Pushed at: about 2 months ago - Stars: 9 - Forks: 4

zuoyebang/bitalostored

Bitalostored is a high-performance distributed storage system, core engine based on bitalosdb(self-developed), compatible with Redis protocol.

Language: Go - Size: 9.62 MB - Last synced at: 2 months ago - Pushed at: 2 months ago - Stars: 2,165 - Forks: 150

pineman/psis

Multithreaded distributed clipboard

Language: C - Size: 508 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0

eventql/eventql

Distributed "massively parallel" SQL query engine

Language: C++ - Size: 76.8 MB - Last synced at: 25 days ago - Pushed at: over 8 years ago - Stars: 1,178 - Forks: 104

lni/dragonboat

A feature complete and high performance multi-group Raft library in Go.

Language: Go - Size: 6.15 MB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 5,203 - Forks: 558

cycneuramus/seaweedfs-docker-swarm

Performant, highly available distributed storage using SeaweedFS in Docker Swarm

Language: Shell - Size: 18.6 KB - Last synced at: about 2 months ago - Pushed at: almost 3 years ago - Stars: 13 - Forks: 1

pitchpoint-solutions/sfs

The legacy distributed object storage server developed by PitchPoint Solutions can store billions of large and small files using minimal resources. Object data is stored in replicated volumes implemented like Facebooks Haystack Object Store. Object metadata which essentially maps an object name to a volume position is stored in an elasticsearch index. (Development by PitchPoint Solutions has been discontinued)

Language: Java - Size: 2.03 MB - Last synced at: 9 days ago - Pushed at: about 3 years ago - Stars: 87 - Forks: 11

beemesh/beemesh.io

Home of BeeMesh

Language: SCSS - Size: 370 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 4 - Forks: 0

71youssef71/go-etcd-cron

Library to run a distributed cron-like task scheduler

Language: Go - Size: 23.4 KB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 0 - Forks: 0

baidu/braft

An industrial-grade C++ implementation of RAFT consensus algorithm based on brpc, widely used inside Baidu to build highly-available distributed systems.

Language: C++ - Size: 4.53 MB - Last synced at: 5 months ago - Pushed at: about 1 year ago - Stars: 4,110 - Forks: 903

ArchiveBox/good-karma-kit

😇 A Docker Compose bundle to run on servers with spare CPU, RAM, disk, and bandwidth to help the world. Includes Tor, ArchiveWarrior, BOINC, and more...

Size: 98.6 KB - Last synced at: 5 months ago - Pushed at: 6 months ago - Stars: 368 - Forks: 12

leo-project/leofs

The LeoFS Storage System

Language: Erlang - Size: 30 MB - Last synced at: 5 months ago - Pushed at: over 5 years ago - Stars: 1,564 - Forks: 156

340Lab/waverless

waverless: A serverless framework written by rust with WASM, CRIU, FunctionGraph, Integrated Storage

Language: Rust - Size: 23.4 MB - Last synced at: 6 months ago - Pushed at: 6 months ago - Stars: 12 - Forks: 7

Oaklight/Vesper-raft

A fault-tolerant distributed key-value store using the Raft consensus algorithm.

Language: Python - Size: 581 KB - Last synced at: about 1 month ago - Pushed at: almost 6 years ago - Stars: 54 - Forks: 11

SUSE/DeepSea

A collection of Salt files for deploying, managing and automating Ceph.

Language: Python - Size: 6.42 MB - Last synced at: 4 months ago - Pushed at: over 3 years ago - Stars: 163 - Forks: 75

uxff/flexdrive

A flexible extending drive, distributed cloud storage system using gRPC and distributed election algorithm.

Language: Go - Size: 6.19 MB - Last synced at: 6 months ago - Pushed at: 6 months ago - Stars: 28 - Forks: 4

pravega/pravega

Pravega - Streaming as a new software defined storage primitive

Language: Java - Size: 47 MB - Last synced at: 6 months ago - Pushed at: 8 months ago - Stars: 1,998 - Forks: 409

diguage/mysql-notes

MySQL 学习笔记

Language: Shell - Size: 16.5 MB - Last synced at: 5 months ago - Pushed at: 8 months ago - Stars: 282 - Forks: 81

Qihoo360/huststore

High-performance Distributed Storage

Language: C - Size: 30.4 MB - Last synced at: 5 months ago - Pushed at: over 7 years ago - Stars: 828 - Forks: 174

ThinkParQ/beegfs

Public repository for the BeeGFS Parallel File System

Language: C++ - Size: 29.3 MB - Last synced at: 6 months ago - Pushed at: 6 months ago - Stars: 120 - Forks: 18

ks-amit/Distributed-Database

A travel agency app with a distributed database implemented from scratch!

Language: Python - Size: 776 KB - Last synced at: 7 months ago - Pushed at: almost 3 years ago - Stars: 17 - Forks: 6

mulbc/gosbench

Distributed S3 benchmarking tool - Replacement of Cosbench

Language: Go - Size: 731 KB - Last synced at: 7 months ago - Pushed at: 11 months ago - Stars: 48 - Forks: 16

Seagate/cortx 📦

CORTX Community Object Storage is 100% open source object storage uniquely optimized for mass capacity storage devices.

Language: Jupyter Notebook - Size: 1.98 GB - Last synced at: 7 months ago - Pushed at: over 1 year ago - Stars: 622 - Forks: 342

wenweihu86/raft-java

Raft Java implementation which is simple and easy to understand.

Language: Java - Size: 370 KB - Last synced at: 7 months ago - Pushed at: almost 2 years ago - Stars: 1,212 - Forks: 415

sheepdog/sheepdog

Distributed Storage System for QEMU

Language: C - Size: 23.2 MB - Last synced at: 8 months ago - Pushed at: almost 5 years ago - Stars: 992 - Forks: 263

skalenetwork/filestorage.js

Filestorage.js is client library for SKALE sidechain decentralized file storage. Used to transfer files between a browser and a SKALE sidechain.

Language: JavaScript - Size: 13.4 MB - Last synced at: 11 days ago - Pushed at: almost 2 years ago - Stars: 29 - Forks: 9

chrislusf/vasto

A distributed key-value store. On Disk. Able to grow or shrink without service interruption.

Language: Go - Size: 1000 KB - Last synced at: 6 months ago - Pushed at: over 6 years ago - Stars: 263 - Forks: 30

SuperJMN/SeaweedFS-GUI

A graphical user interface for Seaweed File System so you can manage your files easily.

Language: C# - Size: 130 KB - Last synced at: 7 days ago - Pushed at: over 2 years ago - Stars: 10 - Forks: 0

douban/gobeansdb

Distributed object storage server from Douban Inc.

Language: Go - Size: 3.2 MB - Last synced at: 5 months ago - Pushed at: over 1 year ago - Stars: 450 - Forks: 63

sile/plumtuna

An implementation of Optuna storage layer based on Plumtree

Language: Rust - Size: 112 KB - Last synced at: 4 months ago - Pushed at: 11 months ago - Stars: 3 - Forks: 1

amirajoodani/GlusterFS

GlusterFS Repository for Learning and Deploy Cluster

Size: 6.84 KB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 1 - Forks: 0

roll-w/cloudhub 📦

A scalable distributed file system.

Language: Java - Size: 10.3 MB - Last synced at: about 1 month ago - Pushed at: over 1 year ago - Stars: 2 - Forks: 0

a1brz/eth-private-network

This repository provides a step-by-step implementation of a private Ethereum network using the Proof of Authority (PoA) consensus algorithm. It is designed for permissioned blockchain environments where participants are known and trusted, making it ideal for testing, development, or enterprise use cases. The repository includes configuration files,

Language: Dockerfile - Size: 14.6 KB - Last synced at: 6 months ago - Pushed at: over 6 years ago - Stars: 28 - Forks: 19

aimanamri/raspberry-pi4-hadoop-spark-cluster

This is a self-documentation of learning distributed data storage, parallel processing, and Linux OS using Apache Hadoop, Apache Spark and Raspbian OS. In this project, 3-node cluster will be setup using Raspberry Pi 4, install HDFS and run Spark processing jobs via YARN.

Language: Shell - Size: 5.21 MB - Last synced at: 7 months ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 0

fruitlee/public

my public projects repo

Language: Java - Size: 38.1 KB - Last synced at: about 1 year ago - Pushed at: over 6 years ago - Stars: 0 - Forks: 0

ji-podhead/Network-Guides

in this series i will cover different network topics like DNS, DHCP, SSL (certmanager + letsencrypt). The guides derive from my personal research, work and will also be available on medium as blogs.

Size: 558 KB - Last synced at: 8 months ago - Pushed at: over 1 year ago - Stars: 3 - Forks: 0

atroo/node-weedfs Fork of cruzrr/seaweedfs.js

Node.js Client Library for SeaweedFS.

Language: JavaScript - Size: 1.24 MB - Last synced at: 3 days ago - Pushed at: about 7 years ago - Stars: 17 - Forks: 9

Calvineng72/NPMdex

NPMdex is a fully distributed search engine for NPM packages developed from scratch with 3.5k+ lines of code

Language: JavaScript - Size: 16.3 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

joebobmiles/yjson

Yjs backed Plain Data Objects.

Language: JavaScript - Size: 481 KB - Last synced at: 7 months ago - Pushed at: over 4 years ago - Stars: 13 - Forks: 1

beyond-decentralized/AIRport

Beyond Decentralized Apps

Language: TypeScript - Size: 43.9 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 3 - Forks: 1

lyupy/distributed-systems

Distributed systems resources

Size: 6.84 KB - Last synced at: about 1 year ago - Pushed at: about 4 years ago - Stars: 5 - Forks: 0

TonyZhangND/GoOvid

Framework for composing distributed agents

Language: Go - Size: 2.4 MB - Last synced at: about 2 years ago - Pushed at: about 6 years ago - Stars: 1 - Forks: 0

priyangshupal/cas-distributed-file-system

This project offers a peer-to-peer content-addressable distributed file storage in Go with a peer-to-peer library built on top of TCP from scratch. It also supports data encryption during storage and transmission

Language: Go - Size: 274 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 0

vesoft-inc/nebula-storage 📦

A distributed consistent graph storage. This is an archived repo for v2.5 only, from 2.6.0 +, NebulaGraph switched back to https://github.com/vesoft-inc/nebula

Language: C++ - Size: 4.77 MB - Last synced at: over 1 year ago - Pushed at: about 4 years ago - Stars: 59 - Forks: 49

dsukhoroslov/bagri

XML/Document DB on top of distributed cache

Language: Java - Size: 20.1 MB - Last synced at: 4 months ago - Pushed at: over 6 years ago - Stars: 41 - Forks: 5

rohitf/kvs

Valetta, a scalable, fault-tolerant key-value store written in Python, tested with Docker

Language: Python - Size: 67.4 KB - Last synced at: over 1 year ago - Pushed at: almost 8 years ago - Stars: 0 - Forks: 0

abhishekgupta-1/CloudHaoma

Cloud Project (Distributed Storage)

Language: Python - Size: 4.09 MB - Last synced at: over 1 year ago - Pushed at: almost 8 years ago - Stars: 1 - Forks: 0

apkp7/quick-store

Efficient and Stable Peer-To-Peer Distributed Hash Table System

Language: Python - Size: 19 MB - Last synced at: 11 months ago - Pushed at: almost 3 years ago - Stars: 3 - Forks: 0

MarcoFontana48/GoogleChubby

Implementation of Google's Chubby distributed file system in Java using Raft consensus algorithm

Language: Java - Size: 921 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 0

wenke1020/CollapsarDB

CollapsarDB is a new scale-out database targetted for HTAP and new type of DataBase.

Language: C++ - Size: 5.86 KB - Last synced at: over 1 year ago - Pushed at: over 7 years ago - Stars: 0 - Forks: 0

chillsoul/piefs 📦

Piefs is a simple distributed small file system based on Facebook Haystack paper.

Language: Go - Size: 284 KB - Last synced at: over 1 year ago - Pushed at: over 3 years ago - Stars: 2 - Forks: 0

qiyancos/dnfs

An open-source distributed on-file-system fs server which supports NFSV3 protocol.

Language: C - Size: 11.3 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 2 - Forks: 0

Seagate/cortx-mio 📦

Maestro IO Interface: a higher-level API to the CORTX internal motr interface.

Language: C - Size: 313 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 11

Seagate/cortx-experiments 📦

A collection of experiments, feasibility studies, and proofs of concept using CORTX. Feel free to join an existing experiment or start your own!

Language: Shell - Size: 71.6 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 7 - Forks: 41