s3git - s3git: git for Cloud Storage

  •        66

s3git is a simple CLI tool that allows you to create a distributed, decentralized and versioned repository. It scales limitlessly to 100s of millions of files and PBs of storage and stores your data safely in S3. Yet huge repos can be cloned on the SSD of your laptop for making local changes, committing and pushing back. Exactly like git, s3git does not require any server-side components, just download and run the executable. It imports the golang package s3git-go that can be used from other applications as well. Or see the Python module or Ruby gem.




Related Projects

Sia - Your decentralized private cloud

  •    Go

Sia is a new decentralized cloud storage platform that radically alters the landscape of cloud storage. By leveraging smart contracts, client-side encryption, and sophisticated redundancy (via Reed-Solomon codes), Sia allows users to safely store their data with hosts that they do not know or trust. The result is a cloud storage marketplace where hosts compete to offer the best service at the lowest price. And since there is no barrier to entry for hosts, anyone with spare storage capacity can join the network and start making money.

Seafile - A file syncing and collaboration platform for teams.

  •    C

Seafile is an open source cloud storage system with privacy protection and teamwork features. Collections of files are called libraries. Each library can be synced separately. A library can also be encrypted with a user chosen password. Seafile also allows users to create groups and easily sharing files into groups.

git-bug - Distributed bug tracker embedded in Git

  •    Go

Distributed bug tracker embedded in Git. git-bug is a distributed bug tracker embedded in git. It uses git's internal storage so no files are added in your project.

dvc - ⚡️ML models version control, make them shareable and reproducible

  •    Python

It aims to replace tools like Excel and Docs that are being commonly used as a knowledge repo and a ledger for the team, ad-hoc scripts to track and move deploy different model versions, ad-hoc data file suffixes and prefixes. DVC is compatible with Git for storing code and the dependency graph (DAG), but not data files cache. To store and share data files cache DVC supports remotes - any cloud (S3, Azure, Google Cloud, etc) or any on-premise network storage (via SSH, for example).

OpenStack - Software for building Private and Public Clouds

  •    Python

OpenStack is a cloud operating system that controls large pools of compute, storage, and networking resources throughout a datacenter, all managed through a dashboard that gives administrators control while empowering their users to provision resources through a web interface.

gitchain - Decentralized, peer-to-peer Git repositories aka "Git meets Bitcoin"

  •    Go

Git is a free and open source distributed version control system designed to handle everything from small to very large projects with speed and efficiency. Gitchain is an application of ideas behind Bitcoin, Namecoin and DHT applied to Git hosting. Once you install it, it acts as a local proxy to the entire Gitchain P2P network.

cryptosphere - Encrypted peer-to-peer web application platform for decentralized, privacy-preserving applications

  •    Ruby

The Cryptosphere is a global peer-to-peer cryptosystem for publishing and securely distributing content pseudonymously with no central point of failure. The system is openly federated and anyone can join. To ensure quality service and prevent abuse, the Cryptosphere uses an integrated cryptographically secure reputation system which provides a distributed web of trust. There are several systems with similar goals to the Cryptosphere, such as MNet, Freenet, and Tahoe-LAFS. These systems serve as inspiration for the Cryptosphere's design. The Cryptosphere is also heavily influenced by Git, the distributed version control system.


  •    C

Git is a free & open source, distributed version control system designed to handle everything from small to very large projects with speed and efficiency.


  •    C

Fossil is a distributed version control like Git and Mercurial. Fossil also supports distributed bug tracking and distributed wiki all in a single integrated package. It is simple, high-reliability, distributed software configuration management.

swarmDB - A Decentralized Database

  •    C++

Bluzelle brings together the sharing economy and token economy. Bluzelle enables people to rent out their computer storage space to earn a token while dApp developers pay with a token to have their data stored and managed in the most efficient way. It is a a decentralized data ecosystem that allows individuals and businesses to have full data control and the ability to monetize that data.

orbit-db - Peer-to-Peer Databases for the Decentralized Web

  •    Javascript

OrbitDB is a serverless, distributed, peer-to-peer database. OrbitDB uses IPFS as its data storage and IPFS Pubsub to automatically sync databases with peers. It's an eventually consistent database that uses CRDTs for conflict-free database merges making OrbitDB an excellent choice for decentralized apps (dApps), blockchain applications and offline-first web applications. All databases are implemented on top of ipfs-log, an immutable, operation-based conflict-free replicated data structure (CRDT) for distributed systems. If none of the OrbitDB database types match your needs and/or you need case-specific functionality, you can easily implement and use a custom database store of your own.

Darcs - Distributed Revision Control in Haskell

  •    Haskell

Darcs is a distributed advanced revision control system written in Haskell. It is similar to Git, Mercurial and Bazaar. User will have own personnel repository and commits his changes to it. Later the changes are pushed to the centralized repository. Every repository is a branch and it provides support to integrate the changes between them. It provides support to send the changes by email.

Alluxio - Data orchestration for analytics and machine learning in the cloud

  •    Java

Alluxio (formerly known as Tachyon) is a virtual distributed storage system. It bridges the gap between computation frameworks and storage systems, enabling computation applications to connect to numerous storage systems through a common interface.

Noms - The versioned, forkable, syncable database

  •    Go

Noms is a decentralized database based on ideas from Git. It has different features from database. It is Decentralized, You can work with your own copy of data and merge them to main database. Duplicate data is automatically ignored. When you commit data to Noms, you aren't overwriting anything. Instead you're adding to a historical record. By default, data is never removed from Noms.

dvcs-ripper - Rip web accessible (distributed) version control systems: SVN/GIT/HG...

  •    Perl

Rip web accessible (distributed) version control systems: SVN, GIT, Mercurial/hg, bzr, ... It can rip repositories even when directory browsing is turned off.

BigchainDB - The Scalable Blockchain Database

  •    Python

BigchainDB allows developers and enterprise to deploy blockchain proof-of-concepts, platforms and applications with a scalable blockchain database, supporting a wide range of industries and use cases. It is a decentralization ecosystem: a decentralized database, at scale. It can perform 1 million writes per second throughput, store petabytes of data, and sub-second latency.

brig - A file synchronization tool on top of ipfs (and more!)

  •    Go

brig is a distributed & secure file synchronization tool with version control. It is based on ipfs, written in Go. It supports Simplified git version control, Sync algorithm that can handle moved files and empty directories and files, Encryption of data in rest and transport + compression on the fly and lot more.

Hypertable - A high performance, scalable, distributed storage and processing system for structured

  •    C++

Hypertable is based on Google's Bigtable Design, which is a proven scalable design that powers hundreds of Google services. Many of the current scalable NoSQL database offerings are based on a hash table design which means that the data they manage is not kept physically ordered. Hypertable keeps data physically sorted by a primary key and it is well suited for Analytics.

radon - RadonDB is an open source, cloud-native MySQL database for building global, scalable cloud services

  •    Go

RadonDB is an open source, Cloud-native MySQL database for unlimited scalability and performance. RadonDB is a cloud-native database based on MySQL,and architected in fully distributed cluster that enable unlimited scalability (scale-out), capacity and performance. It supported distributed transaction that ensure high data consistency, and leveraged MySQL as storage engine for trusted data reliability. RadonDB is compatible with MySQL protocol, and sup-porting automatic table sharding as well as batch of automation feature for simplifying the maintenance and operation workflow.

hubzilla - build community websites that can interact with one another

  •    PHP

Hubzilla is a general purpose communication server integrated with a web publishing system and a decentralised permission system. If this sounds like a bunch of technical mumbo-jumbo to you, just think of it as an independent platform for sharing stuff online (publicly or privately). Hubzilla contains some social network bits, some cloud storage bits, some blog and forum bits, and some content management bits. These are all integrated within a common privacy framework - and it is all decentralised.