Pulsar - Distributed pub-sub Messaging System from Yahoo

  •        279

Pulsar is a distributed pub-sub messaging platform with a very flexible messaging model and an intuitive client API. It is horizontally scalable (Millions of independent topics and millions of messages published per second), Strong ordering and consistency guarantees, Low latency , REST API, Geo Replication and lot more.

https://github.com/yahoo/pulsar

Tags
Implementation
License
Platform

   




Related Projects

Apache Pulsar - Distributed pub-sub messaging platform

  •    Java

Pulsar is a distributed pub-sub messaging platform with a very flexible messaging model and an intuitive client API. It has run in production at Yahoo scale for over 3 years, with millions of messages per second across millions of topics. It is Horizontally scalable, Low latency, High throughput, Multi-tenancy, Geo-replication, Transparent batching of messages, Transparent handling of partitioned topics, REST API for provisioning, admin and stats and lot more.

RocketMQ - Distributed messaging and streaming data platform

  •    Java

Apache RocketMQ is a distributed messaging and streaming platform with low latency, high performance and reliability, trillion-level capacity and flexible scalability.

Celery - Distributed Task Queue

  •    Python

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well. The execution units, called tasks, are executed concurrently on a single or more worker servers using multiprocessing, Eventlet, or gevent. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).

Luxun - A high-throughput, persistent, distributed, publish-subscribe messaging system based on memo

  •    Java

A high-throughput, persistent, distributed, publish-subscribe messaging system based on memory mapped file and Thrift RPC.

Bee Queue - A simple, fast, robust job/task queue for Node.js, backed by Redis

  •    Javascript

A simple, fast, robust job/task queue for Node.js, backed by Redis.Bee-Queue is meant to power a distributed worker pool and was built with short, real-time jobs in mind. A web server can enqueue a job, wait for a worker process to complete it, and return its results within an HTTP request. Scaling is as simple as running more workers.


gnatsd - High-Performance server for NATS, the cloud native messaging system.

  •    Go

NATS Server is a simple, high performance open source messaging system for cloud native applications, IoT messaging, and microservices architectures. It implements a highly scalable and elegant publish-subscribe (pub/sub) distribution model. The performant nature of NATS make it an ideal base for building modern, reliable, scalable cloud native distributed systems.

MassTransit - Distributed Application Framework for .NET

  •    CSharp

MassTransit is a lightweight message bus for creating distributed applications using the .NET framework. MassTransit provides an extensive set of features on top existing message transports, resulting in a developer friendly way to asynchronously connect services using message-based conversation patterns. Message-based communication is a reliable and scalable way to implement a service oriented architecture.

Dramatiq - Simple distributed task processing for Python 3.

  •    Python

dramatiq is a distributed task processing library for Python with a focus on simplicity, reliability and performance.

Apache Apollo - ActiveMQ's next generation of messaging

  •    Java

ActiveMQ Apollo is a faster, more reliable, easier to maintain messaging broker built from the foundations of the original ActiveMQ. It accomplishes this using a radically different threading and message dispatching architecture. Like ActiveMQ, Apollo is a multi-protocol broker and supports STOMP, AMQP, MQTT, Openwire, SSL, and WebSockets.

Faye - Simple pub/sub messaging for the web

  •    Javascript

Faye is a set of tools for simple publish-subscribe messaging between web clients. It ships with easy-to-use message routing servers for Node.js and Rack applications, and clients that can be used on the server and in the browser. It is based on the Bayeux protocol.

NSQ - A realtime distributed messaging platform in Go

  •    Go

NSQ is a realtime distributed messaging platform designed to operate at scale, handling billions of messages per day. It promotes distributed and decentralized topologies without single points of failure, enabling fault tolerance and high availability coupled with a reliable message delivery guarantee. It scales horizontally, without any centralized brokers. Built-in discovery simplifies the addition of nodes to the cluster.

RabbitMQ - Robust messaging for applications

  •    Erlang

RabbitMQ is a messaging broker - an intermediary for messaging. It gives your applications a common platform to send and receive messages, and your messages a safe place to live until received. It features include reliability, high availability, Clustering and Federation. RabbitMQ ships with an easy-to use management UI that allows you to monitor and control every aspect of your message broker. There are RabbitMQ clients for almost any language you can think of.

liftbridge - Lightweight, fault-tolerant message streams.

  •    Go

Liftbridge provides lightweight, fault-tolerant message streams by implementing a durable stream augmentation for the NATS messaging system. It extends NATS with a Kafka-like publish-subscribe log API that is highly available and horizontally scalable. Use Liftbridge as a simpler and lighter alternative to systems like Kafka and Pulsar or use it to add streaming semantics to an existing NATS deployment. See the introduction post on Liftbridge and this post for more context and some of the inspiration behind it.

SwiftQ - Distributed Task Queue

  •    Swift

SwiftQ is a distributed task queue for server side swift applications. Task queues are used as a mechanism to distribute work across machines. SwiftQ uses messages to communicate between clients and workers. In this case the message broker is Redis. SwiftQ uses the reliable queue pattern. This ensures that all tasks get processed even in the event of networking problems or consumer crashes. SwiftQ can be used for real time operations as well as delayed execution of tasks. SwiftQ can consist of multiple producers and consumers allowing for high availability and horizontal scaling. It is recommended that a separate Redis database is used to avoid conflicting name space.

rq - Simple job queues for Python

  •    Python

RQ (Redis Queue) is a simple Python library for queueing jobs and processing them in the background with workers. It is backed by Redis and it is designed to have a low barrier to entry. It should be integrated in your web stack easily.RQ requires Redis >= 2.7.0.

mangos - package mangos is an implementation in pure Go of the SP ("Scalable Protocols") protocols.

  •    Go

package mangos is an implementation in pure Go of the SP ("Scalable Protocols") protocols. This makes heavy use of go channels, internally, but it can operate on systems that lack support for cgo. It has no external dependencies.At present, all of the Req/Rep, Pub/Sub, Pair, Bus, Push/Pull, and Surveyor/Respondent patterns are supported.

golongpoll - golang long polling library

  •    Go

Golang long polling library. Makes web pub-sub easy via an HTTP long-poll server.To use, create a LongpollManager and then use it to publish events and expose an HTTP handler to subscribe to events.