gleam - Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly

•    Go

Gleam is a high performance and efficient distributed execution system, and also simple, generic, flexible and easy to customize.Gleam is built in Go, and the user defined computation can be written in Go, Unix pipe tools, or any streaming programs.

protoactor-go - Proto Actor - Ultra fast distributed actors for Go, C# and Java/Kotlin

•    Go

Introducing cross platform actor support between Go and C#.Can I use this? The Go implementation is still in beta, there are users using Proto Actor for Go in production already. But be aware that the API might change over time until 1.0.

NakedTensor - Bare bone examples of machine learning in TensorFlow

•    Python

This is a bare bones example of TensorFlow, a machine learning package published by Google. You will not find a simpler introduction to it. In each example, a straight line is fit to some data. Values for the slope and y-intercept of the line that best fit the data are determined using gradient descent. If you do not know about gradient descent, check out the Wikipedia page.

elephas - Distributed Deep learning with Keras & Spark

•    Python

Schematically, elephas works as follows. Elephas brings deep learning with Keras to Spark. Elephas intends to keep the simplicity and high usability of Keras, thereby allowing for fast prototyping of distributed models, which can be run on massive data sets. For an introductory example, see the following iPython notebook.

boinc - Open-source software for volunteer computing and grid computing.

•    PHP

The University of California holds the copyright on all BOINC source code. By submitting contributions to the BOINC code, you irrevocably assign all right, title, and interest, including copyright and all copyright rights, in such contributions to The Regents of the University of California, who may then use the code for any purpose that it desires. BOINC is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

fiber - Distributed Computing for AI Made Simple

•    Python

This project is experimental and the APIs are not considered stable. Fiber is a Python distributed computing library for modern computer clusters.

catalyst - Accelerated deep learning R&D

•    Python

Accelerated deep learning R&D

protoactor-dotnet - Proto Actor - Ultra fast distributed actors for Go, C# and Java/Kotlin

•    CSharp

Ultra-fast, distributed, cross-platform actors.This is the .NET repository for Proto Actor.

distributed - Distributed computation in Python

•    Python

A library for distributed computation. See documentation for more details.

moosefs - MooseFS – Open Source, Petabyte, Fault-Tolerant, Highly Performing, Scalable Network Distributed File System

•    C

MooseFS is a Petabyte Open Source Network Distributed File System. It is easy to deploy and maintain, fault tolerant, highly performing, easily scalable, POSIX compliant. MooseFS Linux Client uses FUSE. MooseFS macOS Client uses FUSE for macOS.

rain - Framework for large distributed pipelines

•    Rust

Rain is an open-source distributed computational framework for processing of large-scale task-based pipelines. Rain aims to lower the entry barrier to the world of distributed computing. Our intention is to provide a light yet robust distributed framework that features an intuitive Python API, straightforward installation and deployment with insightful monitoring on top.

transient - A full stack, reactive architecture for general purpose programming

•    Haskell

It's a bit mind bending in that it's like using a higher-level list monad, but it's very, very cool. For beginning Haskellers, what would be really useful is a visualisation of what happens when you do various distributed/parallel stuff. It's almost shocking how effortlessly you can run computations across threads/nodes. The cool part is the composability in the distributed setting. You can make higher-order monadic functions that allow you to compose & reuse a long chain of distributed transactions via wormhole and teleport. Another benefit is that the transaction becomes first class and you can see exactly what's going on in one place instead of distributing the logic across actors making the code equivalent to event callbacks, as you've stated.

SmartSql - SmartSql = MyBatis

•    CSharp

SmartSql = MyBatis + Cache(Memory | Redis) + R/W Splitting +Dynamic Repository + Diagnostics ...... SmartSql draws on MyBatis's ideas, uses XML to manage SQL, and provides several filter tags to eliminate various if/else judgment branches at the code level. SmartSql will manage your SQL and filter the tags to maintain your various conditional judgments at the code level to make your code more beautiful.

couler - Unified Interface for Constructing and Managing Workflows on different workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow

•    Python

Couler aims to provide a unified interface for constructing and managing workflows on different workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow. Couler is included in CNCF Cloud Native Landscape and LF AI Landscape.

LizardFS - Software Defined Storage, Distributed, Parallel, Scalable, Fault-tolerant, Geo-Redundant and Highly available file system

•    C++

LizardFS Software Defined Storage is a distributed, parallel, scalable, fault-tolerant, Geo-Redundant and highly available file system. It spreads data over a number of physical servers, making it visible to an end user as a single file system.

JPPF - Parallelize computationally intensive tasks and execute them on a Grid

•    Java

JPPF enables applications with large processing power requirements to be run on any number of computers, in order to dramatically reduce their processing time. This is done by splitting an application into smaller parts that can be executed simultaneously on different machines.

future - :rocket: R package: future: Unified Parallel and Distributed Processing in R for Everyone

•    R

The purpose of the future package is to provide a very simple and uniform way of evaluating R expressions asynchronously using various resources available to the user. In programming, a future is an abstraction for a value that may be available at some point in the future. The state of a future can either be unresolved or resolved. As soon as it is resolved, the value is available instantaneously. If the value is queried while the future is still unresolved, the current process is blocked until the future is resolved. It is possible to check whether a future is resolved or not without blocking. Exactly how and when futures are resolved depends on what strategy is used to evaluate them. For instance, a future can be resolved using a sequential strategy, which means it is resolved in the current R session. Other strategies may be to resolve futures asynchronously, for instance, by evaluating expressions in parallel on the current machine or concurrently on a compute cluster.

holochain - The new, performant, and simplified version of Holochain on Rust (sometimes called Holochain RSM for Refactored State Model)

•    Rust

This repository contains the core Holochain libraries and binaries. This is the most recent and well maintained version of Holochain with a refactored state model (you may see references to it as Holochain RSM).

paracel - Distributed training framework with parameter server

•    C++

Paracel is a distributed computational framework, designed for many machine learning problems: Logistic Regression, SVD, Matrix Factorization(BFGS, sgd, als, cg), LDA, Lasso... Firstly, paracel splits both massive dataset and massive parameter space. Unlike Mapreduce-Like Systems, paracel offers a simple communication model, allowing you to work with a global and distributed key-value storage, which is called parameter server.

distributed-consensus-reading-list - List of academic papers on distributed consensus

This markdown file contains a list of academic papers (and other works) in the field of distributed consensus. Many of the papers listed below fit into more than one section. However, for simplicity, each paper is listed only in the most relevant section. Where possible, open access links for each paper have been provided. Contributions are welcome. This section lists theoretical results relating to distributed consensus.