GitHub topics: distributed-storage

71youssef71/go-etcd-cron

Library to run a distributed cron-like task scheduler

Language: Go - Size: 23.4 KB - Last synced at: about 2 hours ago - Pushed at: about 3 hours ago - Stars: 0 - Forks: 0

340Lab/waverless

waverless: A serverless framework written by rust with WASM, CRIU, FunctionGraph, Integrated Storage

Language: Rust - Size: 23.3 MB - Last synced at: about 10 hours ago - Pushed at: about 12 hours ago - Stars: 12 - Forks: 7

kiwek123/lustre

lustre game engine

Size: 4.46 MB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 0 - Forks: 0

leil-io/saunafs

SaunaFS is a free-and open source, distributed POSIX file system inspired by Google File System.

Language: C++ - Size: 14.8 MB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 78 - Forks: 5

bsc-dom/dataclay

Python distributed data store that enables remotely access and method execution.

Language: Python - Size: 19.8 MB - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 8 - Forks: 3

zenoh unifies data in motion, data in-use, data at rest and computations. It carefully blends traditional pub/sub with geo-distributed storages, queries and computations, while retaining a level of time and space efficiency that is well beyond any of the mainstream stacks.

Language: Rust - Size: 20.7 MB - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 1,801 - Forks: 189

storj/storj

Ongoing Storj v3 development. Decentralized cloud object storage that is affordable, easy to use, private, and secure.

Language: Go - Size: 158 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 3,099 - Forks: 415

cubefs/cubefs

cloud-native distributed storage

Language: Go - Size: 243 MB - Last synced at: 3 days ago - Pushed at: 4 days ago - Stars: 5,051 - Forks: 659

vadniks/Klenalo

A secured realtime P2P LAN chat and distributed storage manager (desktop linux version)

Language: C - Size: 1.09 MB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 1 - Forks: 0

uxff/flexdrive

A flexible extending drive, distributed cloud storage system using gRPC and distributed election algorithm.

Language: Go - Size: 6.19 MB - Last synced at: 4 days ago - Pushed at: 5 days ago - Stars: 28 - Forks: 4

AutoCSer/AutoCSer2

AutoCSer is an out-of-the-box RPC framework implemented by C#, which has high concurrency, high throughput and other high performance characteristics, and is an ideal distributed application infrastructure.

Language: C# - Size: 354 MB - Last synced at: 1 day ago - Pushed at: 16 days ago - Stars: 11 - Forks: 1

daos-stack/daos

DAOS Storage Stack (client libraries, storage engine, control plane)

Language: C - Size: 165 MB - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 826 - Forks: 313

ceph/ceph

Ceph is a distributed object, block, and file storage platform

Language: C++ - Size: 794 MB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 14,965 - Forks: 6,053

seaweedfs/seaweedfs

SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.

Language: Go - Size: 68.5 MB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 24,268 - Forks: 2,389

ArchiveBox/good-karma-kit

😇 A Docker Compose bundle to run on servers with spare CPU, RAM, disk, and bandwidth to help the world. Includes Tor, ArchiveWarrior, BOINC, and more...

Size: 86.9 KB - Last synced at: 11 days ago - Pushed at: 11 days ago - Stars: 362 - Forks: 12

ZettaScaleLabs/zenoh Fork of eclipse-zenoh/zenoh

zenoh unifies data in motion, data in-use, data at rest and computations. It carefully blends traditional pub/sub with geo-distributed storages, queries and computations, while retaining a level of time and space efficiency that is well beyond any of the mainstream stacks.

Language: Rust - Size: 21.2 MB - Last synced at: 11 days ago - Pushed at: 11 days ago - Stars: 11 - Forks: 4

gojek/awesome-distributed-systems

Awesome list of distributed systems resources

Size: 824 KB - Last synced at: 4 days ago - Pushed at: 10 months ago - Stars: 827 - Forks: 108

kmgowda/SBK

Storage Benchmark Kit

Language: Java - Size: 25.6 MB - Last synced at: 15 days ago - Pushed at: 15 days ago - Stars: 31 - Forks: 65

lustre/lustre-release

Mirror of official Lustre development repository http://git.whamcloud.com/

Language: C - Size: 161 MB - Last synced at: 15 days ago - Pushed at: 15 days ago - Stars: 176 - Forks: 56

ThinkParQ/beegfs

Public repository for the BeeGFS Parallel File System

Language: C++ - Size: 29.3 MB - Last synced at: 16 days ago - Pushed at: 17 days ago - Stars: 120 - Forks: 18

pravega/pravega

Pravega - Streaming as a new software defined storage primitive

Language: Java - Size: 47 MB - Last synced at: 17 days ago - Pushed at: 2 months ago - Stars: 1,996 - Forks: 409

storacha/storacha.ai

Source code for the storacha.ai microsite

Language: TypeScript - Size: 700 KB - Last synced at: 18 days ago - Pushed at: 18 days ago - Stars: 0 - Forks: 0

baidu/braft

An industrial-grade C++ implementation of RAFT consensus algorithm based on brpc, widely used inside Baidu to build highly-available distributed systems.

Language: C++ - Size: 4.53 MB - Last synced at: 20 days ago - Pushed at: 7 months ago - Stars: 4,095 - Forks: 897

lni/dragonboat

A feature complete and high performance multi-group Raft library in Go.

Language: Go - Size: 6.18 MB - Last synced at: 17 days ago - Pushed at: 10 months ago - Stars: 5,154 - Forks: 548

rotationalio/honu

Adaptive consistency replication with reinforcement learning for large scale globally distributed storage.

Language: Go - Size: 6.7 MB - Last synced at: 24 days ago - Pushed at: 24 days ago - Stars: 13 - Forks: 2

ks-amit/Distributed-Database

A travel agency app with a distributed database implemented from scratch!

Language: Python - Size: 776 KB - Last synced at: 22 days ago - Pushed at: over 2 years ago - Stars: 17 - Forks: 6

zuoyebang/bitalostored

Bitalostored is a high-performance distributed storage system, core engine based on bitalosdb(self-developed), compatible with Redis protocol.

Language: Go - Size: 9.56 MB - Last synced at: 29 days ago - Pushed at: 10 months ago - Stars: 2,045 - Forks: 148

moosefs/moosefs

MooseFS Distributed Storage – Open Source, Petabyte, Fault-Tolerant, Highly Performing, Scalable Network Distributed File System / Software-Defined Storage

Language: C - Size: 6.59 MB - Last synced at: 30 days ago - Pushed at: about 1 month ago - Stars: 1,775 - Forks: 217

lmco/ChaordicLedger

The ChaordicLedger is the implementation of a design for a combination of Distributed Ledger Technology (DLT) and a Distributed File System (DFS) to create a secure, enterprise-grade platform for storing interlinked project artifacts.

Language: Shell - Size: 123 MB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 9 - Forks: 5

mulbc/gosbench

Distributed S3 benchmarking tool - Replacement of Cosbench

Language: Go - Size: 731 KB - Last synced at: 29 days ago - Pushed at: 5 months ago - Stars: 48 - Forks: 16

Seagate/cortx 📦

CORTX Community Object Storage is 100% open source object storage uniquely optimized for mass capacity storage devices.

Language: Jupyter Notebook - Size: 1.98 GB - Last synced at: about 1 month ago - Pushed at: about 1 year ago - Stars: 622 - Forks: 342

wenweihu86/raft-java

Raft Java implementation which is simple and easy to understand.

Language: Java - Size: 370 KB - Last synced at: about 1 month ago - Pushed at: over 1 year ago - Stars: 1,212 - Forks: 415

flowerinthenight/hedge

A distributed systems library for Kubernetes deployments built on spindle and Cloud Spanner.

Language: Go - Size: 1.42 MB - Last synced at: about 1 month ago - Pushed at: about 2 months ago - Stars: 23 - Forks: 4

matrixorigin/matrixcube

Fundamental Building Block for Elastic Storage With Strong Consistency and Reliability

Language: Go - Size: 3.96 MB - Last synced at: 17 days ago - Pushed at: almost 3 years ago - Stars: 209 - Forks: 40

cycneuramus/seaweedfs-docker-swarm

Performant, highly available distributed storage using SeaweedFS in Docker Swarm

Language: Shell - Size: 18.6 KB - Last synced at: about 1 month ago - Pushed at: over 2 years ago - Stars: 11 - Forks: 1

SUSE/DeepSea

A collection of Salt files for deploying, managing and automating Ceph.

Language: Python - Size: 6.42 MB - Last synced at: about 2 months ago - Pushed at: almost 3 years ago - Stars: 162 - Forks: 75

leo-project/leofs

The LeoFS Storage System

Language: Erlang - Size: 30 MB - Last synced at: about 1 month ago - Pushed at: almost 5 years ago - Stars: 1,563 - Forks: 155

sheepdog/sheepdog

Distributed Storage System for QEMU

Language: C - Size: 23.2 MB - Last synced at: about 2 months ago - Pushed at: over 4 years ago - Stars: 992 - Forks: 263

beemesh/beemesh.io

Home of BeeMesh

Language: SCSS - Size: 332 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 4 - Forks: 0

skalenetwork/filestorage.js

Filestorage.js is client library for SKALE sidechain decentralized file storage. Used to transfer files between a browser and a SKALE sidechain.

Language: JavaScript - Size: 13.4 MB - Last synced at: 10 days ago - Pushed at: over 1 year ago - Stars: 29 - Forks: 9

m91michel/awesome-blockchain-projects

Curated list of blockchain projects and use cases

Size: 7.81 KB - Last synced at: 4 days ago - Pushed at: about 3 years ago - Stars: 22 - Forks: 5

diguage/mysql-notes

MySQL 学习笔记

Language: Shell - Size: 16.5 MB - Last synced at: about 1 month ago - Pushed at: about 2 months ago - Stars: 282 - Forks: 81

pitchpoint-solutions/sfs

The legacy distributed object storage server developed by PitchPoint Solutions can store billions of large and small files using minimal resources. Object data is stored in replicated volumes implemented like Facebooks Haystack Object Store. Object metadata which essentially maps an object name to a volume position is stored in an elasticsearch index. (Development by PitchPoint Solutions has been discontinued)

Language: Java - Size: 2.03 MB - Last synced at: about 1 month ago - Pushed at: over 2 years ago - Stars: 85 - Forks: 11

chrislusf/vasto

A distributed key-value store. On Disk. Able to grow or shrink without service interruption.

Language: Go - Size: 1000 KB - Last synced at: about 1 month ago - Pushed at: about 6 years ago - Stars: 263 - Forks: 30

eventql/eventql

Distributed "massively parallel" SQL query engine

Language: C++ - Size: 76.8 MB - Last synced at: 27 days ago - Pushed at: about 8 years ago - Stars: 1,174 - Forks: 102

zuoyebang/bitalosdb

Bitalosdb is a high-performance KV storage engine.

Language: Go - Size: 2.39 MB - Last synced at: 28 days ago - Pushed at: 10 months ago - Stars: 164 - Forks: 17

SuperJMN/SeaweedFS-GUI

A graphical user interface for Seaweed File System so you can manage your files easily.

Language: C# - Size: 130 KB - Last synced at: about 1 month ago - Pushed at: almost 2 years ago - Stars: 10 - Forks: 0

douban/gobeansdb

Distributed object storage server from Douban Inc.

Language: Go - Size: 3.2 MB - Last synced at: about 1 month ago - Pushed at: about 1 year ago - Stars: 450 - Forks: 63

mikeroyal/Distributed-Systems-Guide

Distributed Systems Guide

Size: 538 KB - Last synced at: 16 days ago - Pushed at: over 3 years ago - Stars: 27 - Forks: 2

zettastor/dbs

ZettaStor DBS provides enterprise-level business storage solutions with high availability, high performance, easy expansion and easy maintenance for large-scale virtualization, private cloud and container environments.

Size: 929 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 75 - Forks: 11

sile/plumtuna

An implementation of Optuna storage layer based on Plumtree

Language: Rust - Size: 112 KB - Last synced at: 30 days ago - Pushed at: 5 months ago - Stars: 3 - Forks: 1

amirajoodani/GlusterFS

GlusterFS Repository for Learning and Deploy Cluster

Size: 6.84 KB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 1 - Forks: 0

freakmaxi/kertish-dos

Kertish-dos is a simple distributed object storage platform, implements object storage on a single distributed computer cluster, and provides interfaces for file/folder handling. Kertish-dos aims primarily for completely distributed operation without a single point of failure, scalable to the exabyte level.

Language: Go - Size: 4.33 MB - Last synced at: 6 days ago - Pushed at: 5 months ago - Stars: 109 - Forks: 9

roll-w/cloudhub 📦

A scalable distributed file system.

Language: Java - Size: 10.3 MB - Last synced at: about 2 months ago - Pushed at: about 1 year ago - Stars: 2 - Forks: 0

a1brz/eth-private-network

This repository provides a step-by-step implementation of a private Ethereum network using the Proof of Authority (PoA) consensus algorithm. It is designed for permissioned blockchain environments where participants are known and trusted, making it ideal for testing, development, or enterprise use cases. The repository includes configuration files,

Language: Dockerfile - Size: 14.6 KB - Last synced at: 7 days ago - Pushed at: about 6 years ago - Stars: 28 - Forks: 19

aimanamri/raspberry-pi4-hadoop-spark-cluster

This is a self-documentation of learning distributed data storage, parallel processing, and Linux OS using Apache Hadoop, Apache Spark and Raspbian OS. In this project, 3-node cluster will be setup using Raspberry Pi 4, install HDFS and run Spark processing jobs via YARN.

Language: Shell - Size: 5.21 MB - Last synced at: about 1 month ago - Pushed at: 10 months ago - Stars: 1 - Forks: 0

Oaklight/Vesper-raft

A fault-tolerant distributed key-value store using the Raft consensus algorithm.

Language: Python - Size: 581 KB - Last synced at: about 1 month ago - Pushed at: over 5 years ago - Stars: 53 - Forks: 11

Qihoo360/huststore

High-performance Distributed Storage

Language: C - Size: 30.4 MB - Last synced at: 23 days ago - Pushed at: almost 7 years ago - Stars: 828 - Forks: 174

datahiv3/Storage-Layer

Protocol-agnostic, highly redundant storage infrastructure supporting multiple storage protocols and networks, with primary integration for 0G Protocol and IPFS. Provides unified storage interfaces while maintaining redundancy across various storage solutions.

Size: 15.6 KB - Last synced at: about 1 month ago - Pushed at: 6 months ago - Stars: 2 - Forks: 0

marcelmittelstaedt/BigData

Lecture: Big Data

Language: HTML - Size: 588 MB - Last synced at: 6 months ago - Pushed at: 6 months ago - Stars: 15 - Forks: 12

mentalfaculty/LLVS

Low-Level Versioned Store

Language: Swift - Size: 1.44 MB - Last synced at: 28 days ago - Pushed at: over 1 year ago - Stars: 207 - Forks: 9

jd-opensource/dtc

High performance Distributed Table Cache system designed by JD.com, Offering hotspot data cache from databases in order to reduce pressure of database and improve QPS.

Language: C++ - Size: 26.6 MB - Last synced at: about 1 month ago - Pushed at: 8 months ago - Stars: 31 - Forks: 11

fruitlee/public

my public projects repo

Language: Java - Size: 38.1 KB - Last synced at: 9 months ago - Pushed at: about 6 years ago - Stars: 0 - Forks: 0

ji-podhead/Network-Guides

in this series i will cover different network topics like DNS, DHCP, SSL (certmanager + letsencrypt). The guides derive from my personal research, work and will also be available on medium as blogs.

Size: 558 KB - Last synced at: about 2 months ago - Pushed at: 10 months ago - Stars: 3 - Forks: 0

atroo/node-weedfs Fork of cruzrr/seaweedfs.js

Node.js Client Library for SeaweedFS.

Language: JavaScript - Size: 1.24 MB - Last synced at: 7 days ago - Pushed at: over 6 years ago - Stars: 17 - Forks: 9

Calvineng72/NPMdex

NPMdex is a fully distributed search engine for NPM packages developed from scratch with 3.5k+ lines of code

Language: JavaScript - Size: 16.3 MB - Last synced at: 10 months ago - Pushed at: 10 months ago - Stars: 0 - Forks: 0

joebobmiles/yjson

Yjs backed Plain Data Objects.

Language: JavaScript - Size: 481 KB - Last synced at: 26 days ago - Pushed at: almost 4 years ago - Stars: 13 - Forks: 1

beyond-decentralized/AIRport

Beyond Decentralized Apps

Language: TypeScript - Size: 43.9 MB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 3 - Forks: 1

lyupy/distributed-systems

Distributed systems resources

Size: 6.84 KB - Last synced at: 6 months ago - Pushed at: over 3 years ago - Stars: 5 - Forks: 0

nilebit/bitstore

Bitstore is a distributed kv storage.

Language: Go - Size: 175 KB - Last synced at: 11 months ago - Pushed at: over 5 years ago - Stars: 7 - Forks: 3

TonyZhangND/GoOvid

Framework for composing distributed agents

Language: Go - Size: 2.4 MB - Last synced at: over 1 year ago - Pushed at: over 5 years ago - Stars: 1 - Forks: 0

priyangshupal/cas-distributed-file-system

This project offers a peer-to-peer content-addressable distributed file storage in Go with a peer-to-peer library built on top of TCP from scratch. It also supports data encryption during storage and transmission

Language: Go - Size: 274 KB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 1 - Forks: 0

vesoft-inc/nebula-storage 📦

A distributed consistent graph storage. This is an archived repo for v2.5 only, from 2.6.0 +, NebulaGraph switched back to https://github.com/vesoft-inc/nebula

Language: C++ - Size: 4.77 MB - Last synced at: 11 months ago - Pushed at: over 3 years ago - Stars: 59 - Forks: 49

rohitf/kvs

Valetta, a scalable, fault-tolerant key-value store written in Python, tested with Docker

Language: Python - Size: 67.4 KB - Last synced at: about 1 year ago - Pushed at: over 7 years ago - Stars: 0 - Forks: 0

abhishekgupta-1/CloudHaoma

Cloud Project (Distributed Storage)

Language: Python - Size: 4.09 MB - Last synced at: about 1 year ago - Pushed at: over 7 years ago - Stars: 1 - Forks: 0

apkp7/quick-store

Efficient and Stable Peer-To-Peer Distributed Hash Table System

Language: Python - Size: 19 MB - Last synced at: 5 months ago - Pushed at: over 2 years ago - Stars: 3 - Forks: 0

MarcoFontana48/GoogleChubby

Implementation of Google's Chubby distributed file system in Java using Raft consensus algorithm

Language: Java - Size: 921 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 0

wenke1020/CollapsarDB

CollapsarDB is a new scale-out database targetted for HTAP and new type of DataBase.

Language: C++ - Size: 5.86 KB - Last synced at: about 1 year ago - Pushed at: almost 7 years ago - Stars: 0 - Forks: 0

chillsoul/piefs 📦

Piefs is a simple distributed small file system based on Facebook Haystack paper.

Language: Go - Size: 284 KB - Last synced at: about 1 year ago - Pushed at: almost 3 years ago - Stars: 2 - Forks: 0

qiyancos/dnfs

An open-source distributed on-file-system fs server which supports NFSV3 protocol.

Language: C - Size: 11.3 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 2 - Forks: 0

Seagate/cortx-mio 📦

Maestro IO Interface: a higher-level API to the CORTX internal motr interface.

Language: C - Size: 313 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 11

Seagate/cortx-experiments 📦

A collection of experiments, feasibility studies, and proofs of concept using CORTX. Feel free to join an existing experiment or start your own!

Language: Shell - Size: 71.6 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 7 - Forks: 41

Dev-29/dgcache

A distributed cache system in Golang. It spans across multiple servers with a leader server and follower servers.

Language: Go - Size: 5.04 MB - Last synced at: 20 days ago - Pushed at: over 1 year ago - Stars: 3 - Forks: 1

markoelez/basicfs

Simple distributed key-value store for small files

Language: Python - Size: 55.7 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

srinathln7/gossip-gloomers

A series of distributed system challenges in go from Fly.io

Language: Go - Size: 12.5 MB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

OrleansContrib/Orleans.Extensibility.IdentityServer

Makes Orleans the backend storage for IdentityServer4

Language: C# - Size: 13.7 KB - Last synced at: 30 days ago - Pushed at: over 7 years ago - Stars: 11 - Forks: 7

guilherme-marcello/node-db

An efficient C-based key-value distributed storage service with data serialization and replication capabilities, inspired by Amazon's web service infrastructure.

Language: C - Size: 278 KB - Last synced at: 2 months ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

maiwen/DeepTuning

Automated parameter tuning with deep reinforcement learning

Language: Python - Size: 3.49 MB - Last synced at: 14 days ago - Pushed at: about 6 years ago - Stars: 3 - Forks: 1

vascospereira/Distributed-Backup-Service

Distributed system to backup files between 'peers'

Language: Java - Size: 1.76 MB - Last synced at: over 1 year ago - Pushed at: almost 8 years ago - Stars: 0 - Forks: 0

scherepiuk/fleet

Container Orchestration System

Language: Go - Size: 102 KB - Last synced at: 7 days ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

hari-sikchi/capella

Language: Python - Size: 84 KB - Last synced at: over 1 year ago - Pushed at: about 6 years ago - Stars: 1 - Forks: 0

clitor-is-protocol/kevacoin-cli

CLI tools for KevaCoin blockchain

Language: PHP - Size: 16.6 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0

akmaganti11/cache-rest

Distributed Cache for SpringBoot Micro Services

Size: 59.6 KB - Last synced at: over 1 year ago - Pushed at: about 6 years ago - Stars: 0 - Forks: 0

ihepcc-storage/DeepTuning Fork of maiwen/DeepTuning

Automated parameter tuning with deep reinforcement learning

Language: Python - Size: 3.49 MB - Last synced at: 14 days ago - Pushed at: about 6 years ago - Stars: 1 - Forks: 0