Bee Queue - A simple, fast, robust job/task queue for Node.js, backed by Redis

•    Javascript

---

A simple, fast, robust job/task queue for Node.js, backed by Redis.Bee-Queue is meant to power a distributed worker pool and was built with short, real-time jobs in mind. A web server can enqueue a job, wait for a worker process to complete it, and return its results within an HTTP request. Scaling is as simple as running more workers.

redis-queue job-queue worker-queue job-scheduler bee-queue redis task job queue worker distributed system message-queue task-queue messaging

Bull - Premium package for handling jobs and messages in NodeJS

•    Javascript

---

The fastest, most reliable, Redis-based queue for Node. Carefully written for rock solid stability and atomicity.

priority-queue nodejs message-queue job-queue job queue task parallel worker-queue

rsmq - Redis Simple Message Queue

•    CoffeeScript

---

A lightweight message queue for Node.js that requires no dedicated queue server. Just a Redis server. tl;dr: If you run a Redis server and currently use Amazon SQS or a similar message queue you might as well use this fast little replacement. Using a shared Redis server multiple Node.js processes can send / receive messages.

rsmq queue message-queue fifo redis nodejs npm messagequeue jobs

rq - Simple job queues for Python

•    Python

---

RQ (Redis Queue) is a simple Python library for queueing jobs and processing them in the background with workers. It is backed by Redis and it is designed to have a low barrier to entry. It should be integrated in your web stack easily.RQ requires Redis >= 2.7.0.

queue message-queue task-queue messaging job-queue

Kue - Priority job queue backed by redis, built for node.js.

•    Javascript

---

Kue is a priority job queue backed by redis, built for node.js.

priority-queue worker-queue job-queue job queue worker redis

rmq - Message queue system written in Go and backed by Redis

•    Go

---

rmq is short for Redis message queue. It's a message queue system written in Go and backed by Redis. It's similar to redismq, but implemented independently with a different interface in mind. Lets take a look at how to use rmq.

redis message-queue

disque - Disque is a distributed message broker

•    C

---

Disque is an ongoing experiment to build a distributed, in-memory, message broker. Its goal is to capture the essence of the "Redis as a jobs queue" use case, which is usually implemented using blocking list operations, and move it into an ad-hoc, self-contained, scalable, and fault tolerant design, with simple to understand properties and guarantees, but still resembling Redis in terms of simplicity, performance, and implementation as a C non-blocking networked server.Currently (2 Jan 2016) the project is in release candidate state. People are encouraged to start evaluating it and report bugs and experiences.

message-queue in-memory distributed message-broker queue

Machinery - Asynchronous task queue/job queue based on distributed message passing

•    Go

---

Machinery is an asynchronous task queue/job queue based on distributed message passing.

task job-queue worker-queue task-scheduler queue amqp rabbitmq redis memcached mongodb

swoole-jobs - Dynamic multi process worker queue base on swoole, like gearman but high performance

•    PHP

---

redis/rabbitmq/zeromq and any other queue message store (currently only redis/rabbitmq). use swoole process to realize multi process management, the number of processes can be configured, and the worker process will automatically pull up after exiting.

swoole-jobs swoole jobs queue

node-resque - It's Resque... For node! Background jobs using redis.

•    Javascript

---

new queue requires only the "queue" variable to be set. You can also pass the jobs hash to it.The configuration hash passed to new NodeResque.Worker, new NodeResque.Scheduler or new NodeResque.Queue can also take a connection option.

delayed queue resque redis work worker background job task

django-rq - A simple app that provides django integration for RQ (Redis Queue)

•    Python

---

Django integration with RQ, a Redis based Python queuing library. Django-RQ is a simple app that allows you to configure your queues in django's settings.py and easily use them in your project. With this setting, job decorator will set result_ttl to 5000 unless it's specified explicitly.

rq django task-queue background-jobs job-queue redis

enqueue-dev - PHP7.1+. Message queue packages for PHP, Symfony, Laravel, Yii, and Magento

•    PHP

---

Enqueue is production ready, battle-tested messaging solution for PHP. Provides a common way for programs to create, send, read messages. This is a main development repository. It provides a friendly environment for productive development and testing of all Enqueue related features&packages.

message-queue message-bus rabbitmq amqp stomp job-queue amazon-sqs redis doctrine-dbal rabbitmq-client message-passing beanstalkd gearman kafka symfony google-pubsub

Resque - Job queue in Ruby

•    Ruby

---

Resque is a Redis-backed Ruby library for creating background jobs, placing them on multiple queues, and processing them later. Queues are picked off in order of their priority. A job from a lower priority queue will only be picked off if there are no jobs for a higher priority queue available.

job-queue task job queue worker-queue background-task workflow

qless - Queue / Pipeline Management

•    Ruby

---

Qless is a powerful Redis-based job queueing system inspired by resque, but built on a collection of Lua scripts, maintained in the qless-core repo. A job is a unit of work identified by a job id or jid. A queue can contain several jobs that are scheduled to be run at a certain time, several jobs that are waiting to run, and jobs that are currently running. A worker is a process on a host, identified uniquely, that asks for jobs from the queue, performs some process associated with that job, and then marks it as complete. When it's completed, it can be put into another queue.

siberite - Siberite is a simple, lightweight, leveldb backed message queue written in Go.

•    Go

---

Siberite is a simple leveldb backed message queue server (twitter/kestrel, wavii/darner rewritten in Go). Siberite is a very simple message queue server. Unlike in-memory servers such as redis, Siberite is designed to handle queues much larger than what can be held in RAM. And unlike enterprise queue servers such as RabbitMQ, Siberite keeps all messages out of process, using goleveldb as a persistent storage.

queue message-queue kestrel darner

Redisson - Redis based In-Memory Data Grid for Java

•    Java

---

Redisson - distributed Java objects and services (Set, Multimap, SortedSet, Map, List, Queue, BlockingQueue, Deque, BlockingDeque, Semaphore, Lock, AtomicLong, Map Reduce, Publish / Subscribe, Bloom filter, Spring Cache, Executor service, Tomcat Session Manager, Scheduler service, JCache API) on top of Redis server. Rich Redis client.

cache distributed-caching distributed-locks redis-client redis-cluster collections java-collections hashmap set queue

Faktory - "Takin' care of business, workin' overtime"

•    Go

---

At a high level, Faktory is a work server. It is the repository for background jobs within your application. Jobs have a type and a set of arguments and are placed into queues for workers to fetch and execute.You can use this server to distribute jobs to one or hundreds of machines. Jobs can be executed with any language by clients using the Faktory API to fetch a job from a queue.

message-queue job-queue task-queue background-task

sidekiq-unique-jobs - The missing unique jobs in sidekiq

•    Ruby

---

See https://github.com/mperham/sidekiq#requirements for what is required. Starting from 5.0.0 only sidekiq >= 4 is supported and support for MRI <= 2.1 is dropped. Sidekiq consists of a client and a server. The client is responsible for pushing jobs to the queue and the worker is responsible for popping jobs from the queue. Most of the uniqueness is handled when the client is pushing jobs to the queue. The client checks if it is allowed to put a job on the queue. This is probably the most common way of locking.

sidekiq redis

pg-boss - Queueing jobs in Node.js using PostgreSQL like a boss

•    Javascript

---

pg-boss is a job queue built in Node.js on top of PostgreSQL in order to provide background processing and reliable asynchronous execution to Node.js applications. pg-boss relies on SKIP LOCKED, a feature introduced in PostgreSQL 9.5 written specifically for message queues, in order to resolve record locking challenges inherent with relational databases. This brings the safety of guaranteed atomic commits of a relational database to your asynchronous job processing.

queue message-queue job-queue postgresql-library postgresql

Foundatio - Pluggable foundation blocks for building distributed apps.

•    CSharp

---

Pluggable foundation blocks for building loosely coupled distributed apps.Includes implementations in Redis, Azure, AWS and in memory (for development).

azure c-sharp lock message-bus storage redis metrics aws distributed-systems foundatio s3 jobs job-scheduler queue messaging statsd distributed-locks distributed logging abstractions