Bee Queue - A simple, fast, robust job/task queue for Node.js, backed by Redis

•    Javascript

---

A simple, fast, robust job/task queue for Node.js, backed by Redis.Bee-Queue is meant to power a distributed worker pool and was built with short, real-time jobs in mind. A web server can enqueue a job, wait for a worker process to complete it, and return its results within an HTTP request. Scaling is as simple as running more workers.

redis-queue job-queue worker-queue job-scheduler bee-queue redis task job queue worker distributed system message-queue task-queue messaging

Machinery - Asynchronous task queue/job queue based on distributed message passing

•    Go

---

Machinery is an asynchronous task queue/job queue based on distributed message passing.

task job-queue worker-queue task-scheduler queue amqp rabbitmq redis memcached mongodb message-queue

Celery - Distributed Task Queue

•    Python

---

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well. The execution units, called tasks, are executed concurrently on a single or more worker servers using multiprocessing, Eventlet, or gevent. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).

task-queue task queue worker distributed messaging message-queue job-queue task-manager task-scheduler task-runner sqs-queue

Redisson - Redis based In-Memory Data Grid for Java

•    Java

---

Redisson - distributed Java objects and services (Set, Multimap, SortedSet, Map, List, Queue, BlockingQueue, Deque, BlockingDeque, Semaphore, Lock, AtomicLong, Map Reduce, Publish / Subscribe, Bloom filter, Spring Cache, Executor service, Tomcat Session Manager, Scheduler service, JCache API) on top of Redis server. Rich Redis client.

cache distributed-caching distributed-locks redis-client redis-cluster collections java-collections hashmap set queue

RabbitMQ - Robust messaging for applications

•    Erlang

---

RabbitMQ is a messaging broker - an intermediary for messaging. It gives your applications a common platform to send and receive messages, and your messages a safe place to live until received. It features include reliability, high availability, Clustering and Federation. RabbitMQ ships with an easy-to use management UI that allows you to monitor and control every aspect of your message broker. There are RabbitMQ clients for almost any language you can think of.

messaging message-queue task-queue queue asynchronous stomp

MassTransit - Distributed Application Framework for .NET

•    CSharp

---

MassTransit is a lightweight message bus for creating distributed applications using the .NET framework. MassTransit provides an extensive set of features on top existing message transports, resulting in a developer friendly way to asynchronously connect services using message-based conversation patterns. Message-based communication is a reliable and scalable way to implement a service oriented architecture.

rabbitmq gitter-chat appveyor azure-service-bus topshelf messaging nuget dotnet-core2 queue message-queue task-queue

RocketMQ - Distributed messaging and streaming data platform

•    Java

---

Apache RocketMQ is a distributed messaging and streaming platform with low latency, high performance and reliability, trillion-level capacity and flexible scalability.

message-queue messaging queue pub-sub distributed

enqueue-dev - PHP7.1+. Message queue packages for PHP, Symfony, Laravel, Yii, and Magento

•    PHP

---

Enqueue is production ready, battle-tested messaging solution for PHP. Provides a common way for programs to create, send, read messages. This is a main development repository. It provides a friendly environment for productive development and testing of all Enqueue related features&packages.

message-queue message-bus rabbitmq amqp stomp job-queue amazon-sqs redis doctrine-dbal rabbitmq-client message-passing beanstalkd gearman kafka symfony google-pubsub

Foundatio - Pluggable foundation blocks for building distributed apps.

•    CSharp

---

Pluggable foundation blocks for building loosely coupled distributed apps.Includes implementations in Redis, Azure, AWS and in memory (for development).

azure c-sharp lock message-bus storage redis metrics aws distributed-systems foundatio s3 jobs job-scheduler queue messaging statsd distributed-locks distributed logging abstractions

rq - Simple job queues for Python

•    Python

---

RQ (Redis Queue) is a simple Python library for queueing jobs and processing them in the background with workers. It is backed by Redis and it is designed to have a low barrier to entry. It should be integrated in your web stack easily.RQ requires Redis >= 2.7.0.

queue message-queue task-queue messaging job-queue

Pulsar - Distributed pub-sub Messaging System from Yahoo

•    Java

---

Pulsar is a distributed pub-sub messaging platform with a very flexible messaging model and an intuitive client API. It is horizontally scalable (Millions of independent topics and millions of messages published per second), Strong ordering and consistency guarantees, Low latency , REST API, Geo Replication and lot more.

messaging message-queue task-queue queue distributed pub-sub

SwiftQ - Distributed Task Queue

•    Swift

---

SwiftQ is a distributed task queue for server side swift applications. Task queues are used as a mechanism to distribute work across machines. SwiftQ uses messages to communicate between clients and workers. In this case the message broker is Redis. SwiftQ uses the reliable queue pattern. This ensures that all tasks get processed even in the event of networking problems or consumer crashes. SwiftQ can be used for real time operations as well as delayed execution of tasks. SwiftQ can consist of multiple producers and consumers allowing for high availability and horizontal scaling. It is recommended that a separate Redis database is used to avoid conflicting name space.

server-side-swift

huststore - High-performance Distributed Storage

•    C

---

huststore is a open source high performance distributed database system. It not only provides key-value storage service with extremely high performance, up to 100 thousand QPS, but also supports data structures like hash, set, sorted set, etc. Also, it can store binary data as value from a key-value pair, and thus can be used as an alternative of Redis.In addtion, huststore implements a distributed message queue by integrating a special HA module, features including message Push Stream, and message Publish-SubScribe, these features can be used as replacements of the corresponding features in rabbitmq and gearman.

architecture benchmark redis database-engine nginx message-queue distributed-storage

siberite - Siberite is a simple, lightweight, leveldb backed message queue written in Go.

•    Go

---

Siberite is a simple leveldb backed message queue server (twitter/kestrel, wavii/darner rewritten in Go). Siberite is a very simple message queue server. Unlike in-memory servers such as redis, Siberite is designed to handle queues much larger than what can be held in RAM. And unlike enterprise queue servers such as RabbitMQ, Siberite keeps all messages out of process, using goleveldb as a persistent storage.

queue message-queue kestrel darner

TaskQueue - A Task Queue Class developed in Swift (by Marin Todorov)

•    Swift

---

TaskQueue is a Swift library which allows you to schedule tasks once and then let the queue execute them in a synchronous manner. The great thing about TaskQueue is that you get to decide on which GCD queue each of your tasks should execute beforehand and leave TaskQueue to do switching of the queues as it goes. Even if your tasks are asynchronious like fetching location, downloading files, etc. TaskQueue will wait until they are finished before going on with the next task.

disque - Disque is a distributed message broker

•    C

---

Disque is an ongoing experiment to build a distributed, in-memory, message broker. Its goal is to capture the essence of the "Redis as a jobs queue" use case, which is usually implemented using blocking list operations, and move it into an ad-hoc, self-contained, scalable, and fault tolerant design, with simple to understand properties and guarantees, but still resembling Redis in terms of simplicity, performance, and implementation as a C non-blocking networked server.Currently (2 Jan 2016) the project is in release candidate state. People are encouraged to start evaluating it and report bugs and experiences.

message-queue in-memory distributed message-broker queue

NSQ - A realtime distributed messaging platform in Go

•    Go

---

NSQ is a realtime distributed messaging platform designed to operate at scale, handling billions of messages per day. It promotes distributed and decentralized topologies without single points of failure, enabling fault tolerance and high availability coupled with a reliable message delivery guarantee. It scales horizontally, without any centralized brokers. Built-in discovery simplifies the addition of nodes to the cluster.

message-queue messaging distributed publish-subscribe pubsub queue

Bull - Premium package for handling jobs and messages in NodeJS

•    Javascript

---

The fastest, most reliable, Redis-based queue for Node. Carefully written for rock solid stability and atomicity.

priority-queue nodejs message-queue job-queue job queue task parallel worker-queue

darner - simple, lightweight message queue

•    C++

---

Darner is a very simple message queue server. Unlike in-memory servers such as redis, Darner is designed to handle queues much larger than what can be held in RAM. And unlike enterprise queue servers such as RabbitMQ, Darner keeps all messages out of process, relying instead on the kernel's virtual memory manager via log-structured storage. The result is a durable queue server that uses a small amount of in-resident memory regardless of queue size, while still achieving remarkable performance.

rsmq - Redis Simple Message Queue

•    CoffeeScript

---

A lightweight message queue for Node.js that requires no dedicated queue server. Just a Redis server. tl;dr: If you run a Redis server and currently use Amazon SQS or a similar message queue you might as well use this fast little replacement. Using a shared Redis server multiple Node.js processes can send / receive messages.

rsmq queue message-queue fifo redis nodejs npm messagequeue jobs