awesome-distributed-systems - Awesome list of distributed systems resources

•    

---

Awesome list of distributed systems resources

distributed-systems awesome awesome-list

awesome-distributed-systems - A curated list to learn about distributed systems

•    

---

A (hopefully) curated list on awesome material on distributed systems, inspired by other awesome frameworks like awesome-python. Most links will tend to be readings on architecture itself rather than code itself. Read things here before you start.

distributed-systems paper architecture paxos lamport consensus

awesome-time-series-database - :clock7: A curated list of awesome time series databases, benchmarks and papers

•    Javascript

---

OpenTSDB is a Classical time series database on top of HBase. Now support Cassandra and Bigtable. BTrDB (Berkeley Tree Database) is a High performance time series database designed to support high density data storage applications.

awesome time-series database tsdb monitoring metrics awesome-list distributed-systems

distsys-class - Class materials for a distributed systems lecture series

•    

---

This outline accompanies a 12-16 hour overview class on distributed systems fundamentals. The course aims to introduce software engineers to the practical basics of distributed systems, through lecture and discussion. Participants will gain an intuitive understanding of key distributed systems terms, an overview of the algorithmic landscape, and explore production concerns.A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable.

jvm

awesome-blockchains - A collection about awesome blockchains - open distributed public databases w/ crypto hashes incl

•    Ruby

---

A collection about awesome blockchains - open distributed databases w/ crypto hashes incl. git ;-). Blockchains are the new tulips 🌷🌷🌷.

blockchain git distributed-database content-addressable-storage tulipmania merkletree cryptohash sha256 transactions tx

LizardFS - Software Defined Storage, Distributed, Parallel, Scalable, Fault-tolerant, Geo-Redundant and Highly available file system

•    C++

---

LizardFS Software Defined Storage is a distributed, parallel, scalable, fault-tolerant, Geo-Redundant and highly available file system. It spreads data over a number of physical servers, making it visible to an end user as a single file system.

distributed-filesystem filesystem nas distributed-systems distributed-computing distributed-storage fault-tolerance snapshot qos erasure-coding replication replicas geo-replication hsm hierarchical-storage hadoop

distributed-consensus-reading-list - List of academic papers on distributed consensus

•    

---

This markdown file contains a list of academic papers (and other works) in the field of distributed consensus. Many of the papers listed below fit into more than one section. However, for simplicity, each paper is listed only in the most relevant section. Where possible, open access links for each paper have been provided. Contributions are welcome. This section lists theoretical results relating to distributed consensus.

distributed-systems networked-systems consensus-algorithm paxos atomic-broadcast-protocol zookeeper fault-tolerance distributed-computing

awesome-scalability - Scalable, Available, Stable, Performant, and Intelligent System Design Patterns

•    

---

An updated and curated list of readings to illustrate best practices and patterns in building scalable, available, stable, performant, and intelligent large-scale systems. Concepts are explained in the articles of prominent engineers and credible references. Case studies are taken from battle-tested systems that serve millions to billions of users. Understand your problems: scalability problem (fast for a single user but slow under heavy load) or performance problem (slow for a single user) by reviewing some design principles and checking how scalability and performance problems are solved at tech companies. The section of intelligence are created for those who work with data and machine learning at big (data) and deep (learning) scale.

system-design backend scalability site-reliability-engineering sre interview architecture devops site-reliability design-patterns back-end back-end-development interview-questions design-systems awesome-list microservices distributed-systems design-system tech big-data

awesome-chaos-engineering - A curated list of awesome Chaos Engineering resources.

•    

---

A curated list of awesome Chaos Engineering resources. Chaos Engineering is the discipline of experimenting on a distributed system in order to build confidence in the system’s capability to withstand turbulent conditions in production. - Principles Of Chaos Engineering website.

netflix-chaos-monkey chaos-engineering chaos-monkey simian-army site-reliability-engineering resilience chaos chaos-community chaos-testing awesome awesome-list

dist-prog-book

•    CSS

---

Source repo for the book that I and my students in my course at Northeastern University, CS7680 Special Topics in Computing Systems: Programming Models for Distributed Computing, are writing on the topic of programming models for distributed systems. This is a book about the programming constructs we use to build distributed systems. These range from the small, RPC, futures, actors, to the large; systems built up of these components like MapReduce and Spark. We explore issues and concerns central to distributed systems like consistency, availability, and fault tolerance, from the lens of the programming models and frameworks that the programmer uses to build these systems.

testing-distributed-systems - Curated list of resources on testing distributed systems

•    HTML

---

List of resources on testing distributed systems curated by Andrey Satarin (@asatarin). Colin Scott shares his viewpoint from academia on testing distributed systems, specifically regression testing for correctness and performance bugs.

testing distributed-systems jepsen-tests jepsen fault-injection fuzzing

rqlite - The lightweight, distributed relational database built on SQLite.

•    Go

---

rqlite is a distributed relational database, which uses SQLite as its storage engine. rqlite uses Raft to achieve consensus across all the instances of the SQLite databases, ensuring that every change made to the system is made to a quorum of SQLite databases, or none at all. It also gracefully handles leader elections, and tolerates failures of machines, including the leader. rqlite is available for Linux, OSX, and Microsoft Windows.rqlite gives you the functionality of a rock solid, fault-tolerant, replicated relational database, but with very easy installation, deployment, and operation. With it you've got a lightweight and reliable distributed relational data store. Think etcd or Consul, but with relational data modelling also available.

sqlite distributed-systems database relational-database raft consensus distributed-database sql

OrangeFS - Scale-out Network File System

•    C

---

OrangeFS is a scale-out network file system designed for use on high-end computing (HEC) systems that provides very high-performance access to multi-server-based disk storage, in parallel. The OrangeFS server and client are user-level code, making them very easy to install and manage. OrangeFS has optimized MPI-IO support for parallel and distributed applications, and it is leveraged in production installations and used as a research platform for distributed and parallel storage.

filesystem distributed-filesystem storage distributed-storage parallel-virtual-filesystem nas

service-fabric - Service Fabric is a distributed systems platform for packaging, deploying, and managing stateless and stateful distributed applications and containers at large scale

•    C++

---

Service Fabric is a distributed systems platform for packaging, deploying, and managing stateless and stateful distributed applications and containers at large scale. Service Fabric runs on Windows and Linux, on any cloud, any datacenter, across geographic regions, or on your laptop. Learn about Service Fabric's Core Subsystems, mapped to this repo's folder structure.

containers orchestration microservices azure distributed-systems cloud-computing

Atomix - Scalable, fault-tolerant distributed systems protocols and primitives for the JVM

•    Java

---

Atomix is an event-driven framework for coordinating fault-tolerant distributed systems built on the Raft consensus algorithm. It provides the building blocks that solve many common distributed systems problems including group membership, leader election, distributed concurrency control, partitioning, and replication.

raft consensus consensus-algorithm raft-protocol raft-consensus-algorithm distributed-systems leader-election distributed-lock data-structures

tikv - Distributed transactional key-value database, originally created to complement TiDB

•    Rust

---

TiKV is an open-source, distributed, and transactional key-value database. Unlike other traditional NoSQL systems, TiKV not only provides classical key-value APIs, but also transactional APIs with ACID compliance. Built in Rust and powered by Raft, TiKV was originally created to complement TiDB, a distributed HTAP database compatible with the MySQL protocol. The design of TiKV ('Ti' stands for titanium) is inspired by some great distributed systems from Google, such as BigTable, Spanner, and Percolator, and some of the latest achievements in academia in recent years, such as the Raft consensus algorithm.

distributed-transactions raft key-value tikv consensus rocksdb tidb cncf

GeoMesa - Suite of tools for working with big geo-spatial data in a distributed fashion

•    Scala

---

GeoMesa is an open-source, distributed, spatio-temporal database built on a number of distributed cloud data storage systems, including Accumulo, HBase, Cassandra, and Kafka. Leveraging a highly parallelized indexing strategy, GeoMesa aims to provide as much of the spatial querying and data manipulation to Accumulo as PostGIS does to Postgres.

database geospatial-database spatial-database geospatial geospatial-analytics maps

paracel - Distributed training framework with parameter server

•    C++

---

Paracel is a distributed computational framework, designed for many machine learning problems: Logistic Regression, SVD, Matrix Factorization(BFGS, sgd, als, cg), LDA, Lasso... Firstly, paracel splits both massive dataset and massive parameter space. Unlike Mapreduce-Like Systems, paracel offers a simple communication model, allowing you to work with a global and distributed key-value storage, which is called parameter server.

machine-learning distributed-computing graph c-plus-plus

tidis - Distributed transactional NoSQL database, Redis protocol compatible using tikv as backend

•    Go

---

Tidis is a Distributed NoSQL database, providing a Redis protocol API (string, list, hash, set, sorted set), written in Go. Tidis is like TiDB layer, providing protocol transform and data structure compute, powered by TiKV backend distributed storage which use Raft for data replication and 2PC for distributed transaction.

redis nosql database tikv 2pc raft distributed-nosql-database transaction-storage transaction rocksdb

OpenTSDB - A scalable, distributed Time Series Database.

•    Java

---

OpenTSDB is a distributed, scalable Time Series Database (TSDB) written on top of HBase. OpenTSDB was written to address a common need: store, index and serve metrics collected from computer systems (network gear, operating systems, applications) at a large scale, and make this data easily accessible and graphable.

monitoring graph scalable time-series time-series-database database hbase