ex_hash_ring - A fast consistent hash ring implementation in Elixir.

  •        25

A pure Elixir consistent hash ring implemention based on the excellent C hash-ring lib by Chris Moos. The hashring provides fast lookup, but ring creation isn't optimized (though it's not slow). It deliberately does not provide encapsulation within a GenServer and leaves that up to the user. At Discord we found using a GenServer for such frequently accessed data proved to be overwhelming so we rewrote the hash ring in pure Elixir and paired it with FastGlobal to allow the calling process to use it's CPU time to interact with the hash ring and therefore avoiding overloading a central GenServer.

https://github.com/discordapp/ex_hash_ring

Tags
Implementation
License
Platform

   




Related Projects

node-hashring - hashring is a consistent hashing algorithm for Node

  •    Javascript

The HashRing module provides consistent hashing that is compatible with the original libketama library that was developed at last.fm. In addition to beeing compatible with libketama it's also compatible with the hash_ring module for Python. See the compatiblity section of the API for more details on this. The advised installation of module is done through the Node package manager (npm).

torus - Torus - distributed storage system

  •    Go

Torus provides a resource pool and basic file primitives from a set of daemons running atop multiple nodes. These primitives are made consistent by being append-only and coordinated by [etcd]. From these primitives, a Torus server can support multiple types of volumes, the semantics of which can be broken into subprojects. It ships with a simple block-device volume plugin, but is extensible to more. Sharding is done via a consistent hash function, controlled in the simple case by a hash ring algorithm, but fully extensible to arbitrary maps, rack-awareness, and other nice features. The project name comes from this: a hash 'ring' plus a 'volume' is a torus.

hyperbahn - Service discovery and routing for large scale microservice operations

  •    Javascript

Hyperbahn enables service discovery and routing for large-scale systems comprised of many microservices. Distributed, fault tolerant, and highly available, it lets one service find and communicate with others simply and reliably without having to know where those services run. Hyperbahn is an overlay network of routers designed to support the TChannel RPC protocol. Hyperbahn itself is based on Ringpop. Its router nodes dynamically converge and gossip known services onto a consistent hash ring topology, forming a mesh of services that can send requests to each other without human intervention or knowledge of specific ports and addresses.

hashring - Consistent hashing "hashring" implementation in golang (using the same algorithm as libketama)

  •    Go

Implements consistent hashing that can be used when the number of server nodes can increase or decrease (like in memcached). The hashing ring is built using the same algorithm as libketama. This is a port of Python hash_ring library https://pypi.python.org/pypi/hash_ring/ in Go with the extra methods to add and remove nodes.

elixir_express

  •    Elixir

Elixir provides the joy and productivity of Ruby with the concurrency and fault-tolerance of Erlang. Together, we'll take a guided tour through the language, going from the very basics to macros and distributed programming. Along the way, we'll see how Elixir embraces concurrency and how we can construct self-healing programs that restart automatically on failure. Attendees should leave with a great head-start into Elixir, some minor language envy, and a strong desire to continue exploration. Erlang is a functional language with focuses on concurrency and fault tolerance. It first appeared in 1986 as part of Ericsson's quest to build fault tolerant, scalable telecommunication systems. It was open-sourced in 1998 and has gone on to power large portions of the worlds telecom systems, in addition to some of the most popular online services.


ElixirBooks - List of Elixir books

  •    

Official Elixir starting guide that will take you through the language foundations. You will also explore how to build projects with Mix and OTP, and it will introduce you to more advanvced techniques suchs as meta-programming. Adoption is more than programming. Elixir is an exciting new language, but to successfully get your application from start to finish, you’re going to need to know more than just the language. You need the case studies and strategies in this book. Learn the best practices for the whole life of your application, from design and team-building, to managing stakeholders, to deployment and monitoring. Go beyond the syntax and the tools to learn the techniques you need to develop your Elixir application from concept to production.

Elixir - Dynamic, Functional Language designed for building scalable and maintainable application

  •    Elixir

Elixir is a dynamic, functional language designed for building scalable and maintainable applications. Elixir leverages the Erlang VM, known for running low-latency, distributed and fault-tolerant systems, while also being successfully used in web development and the embedded software domain.

ringpop-go - Scalable, fault-tolerant application-layer sharding for Go applications

  •    Go

Ringpop is a library that brings cooperation and coordination to distributed applications. It maintains a consistent hash ring on top of a membership protocol and provides request forwarding as a routing convenience. It can be used to shard your application in a way that's scalable and fault tolerant.to install remaining golang dependencies and install the pre-commit hook.

raft-rs - Raft distributed consensus algorithm implemented in Rust.

  •    Rust

When building a distributed system one principal goal is often to build in fault-tolerance. That is, if one particular node in a network goes down, or if there is a network partition, the entire cluster does not fall over. The cluster of nodes taking part in a distributed consensus protocol must come to agreement regarding values, and once that decision is reached, that choice is final. Distributed Consensus Algorithms often take the form of a replicated state machine and log. Each state machine accepts inputs from its log, and represents the value(s) to be replicated, for example, a hash table. They allow a collection of machines to work as a coherent group that can survive the failures of some of its members.

raft-rs - Raft distributed consensus algorithm implemented in Rust.

  •    Rust

When building a distributed system one principal goal is often to build in fault-tolerance. That is, if one particular node in a network goes down, or if there is a network partition, the entire cluster does not fall over. The cluster of nodes taking part in a distributed consensus protocol must come to agreement regarding values, and once that decision is reached, that choice is final. Distributed Consensus Algorithms often take the form of a replicated state machine and log. Each state machine accepts inputs from its log, and represents the value(s) to be replicated, for example, a hash table. They allow a collection of machines to work as a coherent group that can survive the failures of some of its members.

OpenDHT - A C++11 Distributed Hash Table implementation

  •    C++

A lightweight C++11 Distributed Hash Table implementation. OpenDHT provides an easy to use distributed in-memory data store. Every node in the network can read and write values to the store. Values are distributed over the network, with redundancy.

roma - ROMA: A Distributed Key-Value Store in Ruby

  •    Ruby

ROMA is one of the data storing systems for distributed key-value stores. It is a completely decentralized distributed system that consists of multiple processes, called nodes, on several machines. It is based on pure P2P architecture like a distributed hash table, thus it provides high availability and scalability. ROMA is written in Ruby. However, following choices are available to access to ROMA.

cachex - A powerful caching library for Elixir with support for transactions, fallbacks and expirations

  •    Elixir

All of these features are optional and are off by default so you can pick and choose those you wish to enable. For anything else, please see the documentation.

Ex-Crawler

  •    Java

Ex-Crawler is divided into 3 subprojects (Crawler Daemon, distributed gui Client, (web) search engine) which together provide a flexible and powerful search engine supporting distributed computing. More informations: http://ex-crawler.sourceforge.net

Atomix - Scalable, fault-tolerant distributed systems protocols and primitives for the JVM

  •    Java

Atomix is an event-driven framework for coordinating fault-tolerant distributed systems built on the Raft consensus algorithm. It provides the building blocks that solve many common distributed systems problems including group membership, leader election, distributed concurrency control, partitioning, and replication.

ClickHouse - Column-oriented database management system that allows generating analytical data reports in real time

  •    C++

ClickHouse is a fast open-source OLAP database management system. It is column-oriented and allows to generate analytical reports using SQL queries in real-time. Large queries are parallelized using multiple cores, taking all the necessary resources available on the current server. It supports distributed processing, data can reside on different shards. Each shard can be a group of replicas used for fault tolerance. All shards are used to run a query in parallel, transparently for the user.

elchemy - Write Elixir code using statically-typed Elm-like syntax (compatible with Elm tooling)

  •    Elm

elchemy will find all *.elm files specified in elchemy_path and compile it into corresponding *.ex files in lib directory. You can override output directory specifying elixirc_paths.

testing-distributed-systems - Curated list of resources on testing distributed systems

  •    HTML

List of resources on testing distributed systems curated by Andrey Satarin (@asatarin). Colin Scott shares his viewpoint from academia on testing distributed systems, specifically regression testing for correctness and performance bugs.

dist-prog-book

  •    CSS

Source repo for the book that I and my students in my course at Northeastern University, CS7680 Special Topics in Computing Systems: Programming Models for Distributed Computing, are writing on the topic of programming models for distributed systems. This is a book about the programming constructs we use to build distributed systems. These range from the small, RPC, futures, actors, to the large; systems built up of these components like MapReduce and Spark. We explore issues and concerns central to distributed systems like consistency, availability, and fault tolerance, from the lens of the programming models and frameworks that the programmer uses to build these systems.






We have large collection of open source products. Follow the tags from Tag Cloud >>


Open source products are scattered around the web. Please provide information about the open source projects you own / you use. Add Projects.