s4 - Distributed Stream Computing Platform

  •        0

S4 is a general-purpose, distributed, scalable, partially fault-tolerant, pluggable platform that allows programmers to easily develop applications for processing continuous unbounded streams of data. S4 has been deployed in production systems at Yahoo to process thousands of search queries per second.

http://s4.io/

Tags
Implementation
License
Platform

   




Related Projects

Akka - Build Concurrent and Scalable Applications


Akka is the platform for the next generation event-driven, scalable and fault-tolerant architectures on the JVM. It helps to write simpler correct concurrent applications using Actors, STM & Transactors. It could scale out on multi-core or multiple nodes using asynchronous message passing. For fault-tolerance it adopts the Let it crash or Embrace failure model to build applications that self-heals, systems that never stop.

ejabberd - Robust, Scalable and Extensible XMPP Server


ejabberd is a distributed, fault-tolerant technology that allows the creation of large-scale instant messaging applications. The server can reliably support thousands of simultaneous users on a single node and has been designed to provide exceptional standards of fault tolerance. As an open source technology, based on industry-standards, ejabberd can be used to build bespoke solutions very cost effectively.

Atomix - Scalable, fault-tolerant distributed systems protocols and primitives for the JVM


Atomix is an event-driven framework for coordinating fault-tolerant distributed systems built on the Raft consensus algorithm. It provides the building blocks that solve many common distributed systems problems including group membership, leader election, distributed concurrency control, partitioning, and replication.

lifeguard - Lifeguard is a highly scalable, fault tolerant distributed infrastructure and application monitoring service


Lifeguard is still alpha quality and not ready for production use.Lifeguard is a highly scalable, fault tolerant distributed infrastructure and application monitoring service. The health of your systems are determined by checks written in JavaScript that run across the monitoring cluster. The data that the checks inspect can be retrieved from various sources, such as a database, Graphite, or remote SSH commands. Failing checks notify your IT team as necessary.

NSQ - A realtime distributed messaging platform in Go


NSQ is a realtime distributed messaging platform designed to operate at scale, handling billions of messages per day. It promotes distributed and decentralized topologies without single points of failure, enabling fault tolerance and high availability coupled with a reliable message delivery guarantee. It scales horizontally, without any centralized brokers. Built-in discovery simplifies the addition of nodes to the cluster.

Pinot - A realtime distributed OLAP datastore


Pinot is a realtime distributed OLAP datastore, which is used at LinkedIn to deliver scalable real time analytics with low latency. It can ingest data from offline data sources (such as Hadoop and flat files) as well as online sources (such as Kafka). Pinot is designed to scale horizontally, so that it can scale to larger data sets and higher query rates as needed.

Elixir - Dynamic, Functional Language designed for building scalable and maintainable application


Elixir is a dynamic, functional language designed for building scalable and maintainable applications. Elixir leverages the Erlang VM, known for running low-latency, distributed and fault-tolerant systems, while also being successfully used in web development and the embedded software domain.

placid - A REST toolkit for building highly-scalable and fault-tolerant HTTP APIs with Elixir


A REST toolkit for building highly-scalable and fault-tolerant HTTP APIs with Elixir

Hyperdex - A Searchable Distributed Key-Value Store


HyperDex is a distributed, searchable key-value store. HyperDex provides a unique search primitive which enables searches over stored values. By design, HyperDex retains the performance of traditional key-value stores while enabling support for the search operation. It is fast, scalable, Consistent, Fault tolerant.

JSBus - Fault tolerant messaging between browser and server


Fault tolerant messaging between browser and server

distributed-ftp - Byzantine Fault tolerant distributed file system


Byzantine Fault tolerant distributed file system

FlockDB - A distributed, fault-tolerant graph database from Twitter


FlockDB is much simpler than other graph databases such as neo4j because it tries to solve fewer problems. It scales horizontally and is designed for on-line, low-latency, high throughput environments such as web-sites. Twitter uses FlockDB to store social graphs (who follows whom, who blocks whom) and secondary indices. As of April 2010, the Twitter FlockDB cluster stores 13+ billion edges and sustains peak traffic of 20k writes/second and 100k reads/second.

Gearman - Application Framework to farm out work to other Machines


Gearman provides a generic application framework to farm out work to other machines or processes that are better suited to do the work. It allows you to do work in parallel by doing load balancing. It also supports to call functions between languages. It is the nervous system for how distributed processing communicates. It is fault tolerant.

ejabberd - Fault-tolerant distributed Jabber/XMPP server


Fault-tolerant distributed Jabber/XMPP server

mfs - MooseFS - distributed, fault tolerant file system


MooseFS - distributed, fault tolerant file system

Shoggoth - Simple distributed fault-tolerant file system


Simple distributed fault-tolerant file system

speed - Fault tolerant distributed in memory filesystem based on Membase


Fault tolerant distributed in memory filesystem based on Membase

elock - A simple, fault-tolerant distributed lock server in erlang.


A simple, fault-tolerant distributed lock server in erlang.

dataminer - Node.js fault tolerant, distributed queue workers for your everyday computation.


Node.js fault tolerant, distributed queue workers for your everyday computation.

twisted-lock - Fault-tolerant distributed lock with REST interface.


Fault-tolerant distributed lock with REST interface.