serf - Service orchestration and management tool

  •        97

Serf is a decentralized solution for service discovery and orchestration that is lightweight, highly available, and fault tolerant.Serf runs on Linux, Mac OS X, and Windows. An efficient and lightweight gossip protocol is used to communicate with other nodes. Serf can detect node failures and notify the rest of the cluster. An event system is built on top of Serf, letting you use Serf's gossip protocol to propagate events such as deploys, configuration changes, etc. Serf is completely masterless with no single point of failure.

Its other features include:

  • Discovering web servers and automatically adding them to a load balancer
  • Organizing many memcached or redis nodes into a cluster, perhaps with something like twemproxy or maybe just configuring an application with the address of all the nodes
  • Triggering web deploys using the event system built on top of Serf
  • Propagating changes to configuration to relevant nodes.
  • Updating DNS records to reflect cluster changes as they occur.
  • Much, much more.

https://www.serf.io/
https://github.com/hashicorp/serf

Tags
Implementation
License
Platform

   




Related Projects

Tars - Tars is a highly performance rpc framework based on naming service using tars protocol and provides a semi-automatic operation platform

  •    C++

Tars comes from the robot in Interstellar movie. Tars is a high-performance RPC framework based on name service and Tars protocol, also integrated administration platform, and implemented hosting-service via flexible schedule. Tars, aka TAF(Total Application Framework), has been used in Tencent since 2008. It supports C++,Java,Nodejs and php for now. This framework offers a set of solution for development, maintenance and testing, which making develop, deploy and testing service efficiently. It integrated extensible protocol for encoding/decoding, high-performance RPC communication framework, name service, monitor, statistics and configuration. You can use it to develop your reliable distributed application based on microservice fast, and reach fully efficient service management.

NSQ - A realtime distributed messaging platform in Go

  •    Go

NSQ is a realtime distributed messaging platform designed to operate at scale, handling billions of messages per day. It promotes distributed and decentralized topologies without single points of failure, enabling fault tolerance and high availability coupled with a reliable message delivery guarantee. It scales horizontally, without any centralized brokers. Built-in discovery simplifies the addition of nodes to the cluster.

containerpilot - A service for autodiscovery and configuration of applications running in containers

  •    Go

An init system for cloud-native distributed applications that automates the process of service discovery, configuration, and lifecycle management inside the container, so you can focus on your apps.We can write our new applications to do that, but existing apps will need some help. We can wrap each application in a shell script that registers itself with the discovery service easily enough, but watching for changes to that service and ensuring that health checks are being made is more complicated. We can put a second process in the container, but as soon as we do that we need an init system running inside the container as well.


circuit - Circuit: Dynamic cloud orchestration http://gocircuit.org

  •    Go

The CIRCUIT is a new way of thinking. It is deceptively similar to existing software, while being quite different.Circuit is a programmable platform-as-a-service (PaaS) and/or Infrastructure-as-a-Service (IaaS), for management, discovery, synchronization and orchestration of services and hosts comprising cloud applications.

Dubbo - High-performance, java based, open source RPC framework

  •    Java

Dubbo is a high-performance, java based RPC framework open-sourced by Alibaba. As in many RPC systems, dubbo is based around the idea of defining a service, specifying the methods that can be called remotely with their parameters and return types. On the server side, the server implements this interface and runs a dubbo server to handle client calls. On the client side, the client has a stub that provides the same methods as the server.

OpenDJ - LDAPv3 compliant directory service

  •    Java

OpenDJ is a new LDAPv3 compliant directory service, providing a high performance, highly available and secure store for the identities managed by enterprises. Its easy installation process, combined with the power of the Java platform makes of OpenDJ the simplest and fastest directory server to deploy and manage.

ecs-refarch-service-discovery - An EC2 Container Service Reference Architecture for providing Service Discovery to containers using CloudWatch Events, Lambda and Route 53 private hosted zones

  •    Go

Service discovery is a key component of most distributed systems and service-oriented architectures. With service discovery, services are automatically discovered as they get created and terminated on a given infrastructure. This reference architecture illustrates how service discovery can be built on AWS.Many AWS customers build service-oriented, distributed applications using services such as Amazon EC2 Container Service (Amazon ECS) or Amazon EC2. The distributed nature of this type of architecture requires a fair amount of integration and synchronization, and the answer to that problem is not trivial. Quite often, our customers build such a functionality themselves and this can be time-consuming. Or they use a third-party solution and this often comes with a financial cost.

skydock - Service discovery via DNS for docker

  •    Go

Docker supports DNS based service discovery now. You should use the Docker implementation instead of this project. Skydock was built at a time when Docker did not support DNS discovery or auto registration. I'll keep the repo up for past years and as reference for others but don't use it if you have a recent version of Docker.Skydock monitors docker events when containers start, stop, die, kill, etc and inserts records into a dynamic DNS server skydns. This allows standard DNS queries for services running inside docker containers. Because lets face it, if you have to modify your application code to work with other service discovery solutions you might as well just give up. DNS just works and it works well. Also you cannot be expected to modify application code that you don't own. Passing service urls via the cli or in static config files (nginx) will not be possible if your service discovery solution requires a client library just to fetch an IP.

Redisson - Redis based In-Memory Data Grid for Java

  •    Java

Redisson - distributed Java objects and services (Set, Multimap, SortedSet, Map, List, Queue, BlockingQueue, Deque, BlockingDeque, Semaphore, Lock, AtomicLong, Map Reduce, Publish / Subscribe, Bloom filter, Spring Cache, Executor service, Tomcat Session Manager, Scheduler service, JCache API) on top of Redis server. Rich Redis client.

database - Blazegraph High Performance Graph Database

  •    Java

Blazegraph™ DB is our ultra high-performance graph database supporting Blueprints and RDF/SPARQL APIs. It supports up to 50 Billion edges on a single machine and has a High Availability and Scale-out architecture. It is in production use for customers such as EMC, Syapse, Wikidata Query Service, the British Museum, and many others. GPU acceleration and High Availability (HA) are available in the Enterprise edition. It contains war, jar, deb, rpm, and tar.gz deployment artifacts.

linkerd - Resilient service mesh for cloud native apps

  •    Scala

Linkerd is a transparent service mesh, designed to make modern applications safe and sane by transparently adding service discovery, load balancing, failure handling, instrumentation, and routing to all inter-service communication.linkerd (pronouned "linker-DEE") acts as a transparent HTTP/gRPC/thrift/etc proxy, and can usually be dropped into existing applications with a minimum of configuration, regardless of what language they're written in. It works with many common protocols and service discovery backends, including scheduled environments like Mesos and Kubernetes.

kite - Micro-service framework in Go

  •    Go

Kite is a framework for developing micro-services in Go. Kite is both the name of the framework and the micro-service that is written by using this framework. Basically, Kite is a RPC server as well as a client. It connects to other kites and peers to communicate with each other. They can discover other kites using a service called Kontrol, and communicate with them bidirectionaly. The communication protocol uses a WebSocket (or XHR) as transport in order to allow web applications to connect directly to kites.

Kong - The Microservice API Gateway

  •    Lua

Kong is a cloud-native, fast, scalable, and distributed Microservice Abstraction Layer (also known as an API Gateway, API Middleware or in some cases Service Mesh). Backed by the battle-tested NGINX with a focus on high performance, Kong was made available as an open-source platform in 2015. Under active development, Kong is used in production at thousands of organizations from startups, Global 5000 and Government organizations.

assistant-with-discovery - This application demonstrates the combination of the IBM Watson Assistant and Discovery services

  •    Java

This application demonstrates how you can combine the Watson Assistant and Discovery services to allow customers, employees or the public to get answers to a wide range of questions about a product, service or other topic using plain English. First, users pose a questions to the Watson Assistant service. If Watson Assistant is not able to confidently answer, the app executes a call to Discovery, which to provides a list of helpful answers. The app has a conversational interface that can answer basic questions about a fictitious cognitive car, as well as more obscure questions whose answers can be found in the car’s manual. The app uses two Watson services: Watson Assistant and Discovery. The Watson Assistant service powers the basic Q&A using intents, relationships and natural language, and calls the Discovery app when it encounters questions it can’t answer. Discovery searches and ranks responses from the manual to answer those questions.

swan - A Distributed, Highly Available Mesos Scheduler, Inspired by the design of Google Borg

  •    Go

Swan is a Mesos framework forcus on long running tasks, which inspired by Marathon, written with Golang and base on latest Mesos HTTP API.With Swan you can deploy long running application on mesos cluster, control lifecycle of the application, scale up or scale down any instances size you want, can also do rolling-update with new versions base on flexible health checks defined by you. Swan also designed for high avaliability which means any server crash wouldn't affect running applications. with service discovery and API gateway build-in, service discovery made easy.

marathon-lb - Marathon-lb is a service discovery & load balancing tool for DC/OS

  •    Python

Marathon-lb is a tool for managing HAProxy, by consuming Marathon's app state. HAProxy is a fast, efficient, battle-tested, highly available load balancer with many advanced features which power a number of high-profile websites. Take a look at the marathon-lb wiki for example usage, templates, and more.

doorman - Doorman: Global Distributed Client Side Rate Limiting.

  •    Go

Doorman is a solution for Global Distributed Client Side Rate Limiting. Clients that talk to a shared resource (such as a database, a gRPC service, a RESTful API, or whatever) can use Doorman to voluntarily limit their use (usually in requests per second) of the resource. Doorman is written in Go and uses gRPC as its communication protocol. For some high-availability features it needs a distributed lock manager. We currently support etcd, but it should be relatively simple to make it use Zookeeper instead.The Doorman master server remembers all clients that currently have capacity and whenever a client asks for capacity it inserts the clients request into its memory and runs the algorithm to figure out what this client should get.

kubeadm-ha - Kubernetes high availiability deploy based on kubeadm (for v1

  •    Smarty

kube-apiserver: exposes the Kubernetes API. It is the front-end for the Kubernetes control plane. It is designed to scale horizontally – that is, it scales by deploying more instances. etcd: is used as Kubernetes’ backing store. All cluster data is stored here. Always have a backup plan for etcd’s data for your Kubernetes cluster. kube-scheduler: watches newly created pods that have no node assigned, and selects a node for them to run on. kube-controller-manager: runs controllers, which are the background threads that handle routine tasks in the cluster. Logically, each controller is a separate process, but to reduce complexity, they are all compiled into a single binary and run in a single process. kubelet: is the primary node agent. It watches for pods that have been assigned to its node (either by apiserver or via local configuration file) kube-proxy: enables the Kubernetes service abstraction by maintaining network rules on the host and performing connection forwarding. keepalived cluster config a virtual IP address (192.168.20.10), this virtual IP address point to k8s-master01, k8s-master02, k8s-master03. nginx service as the load balancer of k8s-master01, k8s-master02, k8s-master03's apiserver. The other nodes kubernetes services connect the keepalived virtual ip address (192.168.20.10) and nginx exposed port (16443) to communicate with the master cluster's apiservers.