django-rq - A simple app that provides django integration for RQ (Redis Queue)

•    Python

---

Django integration with RQ, a Redis based Python queuing library. Django-RQ is a simple app that allows you to configure your queues in django's settings.py and easily use them in your project. With this setting, job decorator will set result_ttl to 5000 unless it's specified explicitly.

rq django task-queue background-jobs job-queue redis

Bee Queue - A simple, fast, robust job/task queue for Node.js, backed by Redis

•    Javascript

---

A simple, fast, robust job/task queue for Node.js, backed by Redis.Bee-Queue is meant to power a distributed worker pool and was built with short, real-time jobs in mind. A web server can enqueue a job, wait for a worker process to complete it, and return its results within an HTTP request. Scaling is as simple as running more workers.

redis-queue job-queue worker-queue job-scheduler bee-queue redis task job queue worker distributed system message-queue task-queue messaging

Celery - Distributed Task Queue

•    Python

---

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well. The execution units, called tasks, are executed concurrently on a single or more worker servers using multiprocessing, Eventlet, or gevent. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).

task-queue task queue worker distributed messaging message-queue job-queue task-manager task-scheduler task-runner sqs-queue

Machinery - Asynchronous task queue/job queue based on distributed message passing

•    Go

---

Machinery is an asynchronous task queue/job queue based on distributed message passing.

task job-queue worker-queue task-scheduler queue amqp rabbitmq redis memcached mongodb message-queue

yotaq - yotaq - Your Own Task Queue for Python

•    Python

---

So you need a task queue for your Python project. Sure you could check celery, and after three months trying to understand the basic configuration options you'll be good to go. Or you could use a simpler task queue like huey or rq. Pretty good. Our python code will use dill to serialize the functions to be run and redis to store the tasks.

task-queue task-runner

Faktory - "Takin' care of business, workin' overtime"

•    Go

---

At a high level, Faktory is a work server. It is the repository for background jobs within your application. Jobs have a type and a set of arguments and are placed into queues for workers to fetch and execute.You can use this server to distribute jobs to one or hundreds of machines. Jobs can be executed with any language by clients using the Faktory API to fetch a job from a queue.

message-queue job-queue task-queue background-task

rq-scheduler - A light library that adds job scheduling capabilities to RQ (Redis Queue)

•    Python

---

RQ Scheduler is a small package that adds job scheduling capabilities to RQ, a Redis based Python queuing library. Or you can download the latest stable package from PyPI.

react-native-queue - A React Native Job Queue

•    Javascript

---

A React Native at-least-once priority job queue / task queue backed by persistent Realm storage. Jobs will persist until completed, even if user closes and re-opens app. React Native Queue is easily integrated into OS background processes (services) so you can ensure the queue will continue to process until all jobs are completed even if app isn't in focus. It also plays well with Workers so your jobs can be thrown on the queue, then processed in dedicated worker threads for greatly improved processing performance. React Native Queue is designed to be a swiss army knife for task management in React Native. It abstracts away the many annoyances related to processing complex tasks, like durability, retry-on-failure, timeouts, chaining processes, and more. Just throw your jobs onto the queue and relax - they're covered.

react react-native queue job-queue task-queue

Bull - Premium package for handling jobs and messages in NodeJS

•    Javascript

---

The fastest, most reliable, Redis-based queue for Node. Carefully written for rock solid stability and atomicity.

priority-queue nodejs message-queue job-queue job queue task parallel worker-queue

django-rq - A simple app that provides django integration for RQ (Redis Queue)

•    Python

---

A simple app that provides django integration for RQ (Redis Queue)

sidekiq.cr - Simple, efficient job processing for Crystal

•    Crystal

---

Sidekiq is a well-regarded background job framework for Ruby. Now we're bringing the awesomeness to Crystal, a Ruby-like language. Why? To give you options. Ruby is friendly and flexible but not terribly fast. Crystal is statically-typed, compiled and very fast but retains a similar syntax to Ruby.If you have jobs which are CPU-intensive or require very high throughput, Crystal is an excellent alternative to native Ruby extensions. It compiles to a single executable so deployment is much easier than Ruby.

queue message-queue task-queue messaging

RocketMQ - Distributed messaging and streaming data platform

•    Java

---

Apache RocketMQ is a distributed messaging and streaming platform with low latency, high performance and reliability, trillion-level capacity and flexible scalability.

message-queue messaging queue pub-sub distributed

Fireworq - Lightweight, high-performance, language-independent job queue system

•    Go

---

Fireworq is a lightweight, high-performance job queue system with the following abilities. It is available from ANY programming language which can talk HTTP. It works with a single binary without external dependencies. It is built on top of RDBMS (MySQL), so that jobs won't be lost even if the job queue process dies. You can apply an ordinary replication scheme to the underlying DB for the reliability of the DB itself.

job queue job-queue worker-queue background-task

Resque - Job queue in Ruby

•    Ruby

---

Resque is a Redis-backed Ruby library for creating background jobs, placing them on multiple queues, and processing them later. Queues are picked off in order of their priority. A job from a lower priority queue will only be picked off if there are no jobs for a higher priority queue available.

job-queue task job queue worker-queue background-task workflow

RabbitMQ - Robust messaging for applications

•    Erlang

---

RabbitMQ is a messaging broker - an intermediary for messaging. It gives your applications a common platform to send and receive messages, and your messages a safe place to live until received. It features include reliability, high availability, Clustering and Federation. RabbitMQ ships with an easy-to use management UI that allows you to monitor and control every aspect of your message broker. There are RabbitMQ clients for almost any language you can think of.

messaging message-queue task-queue queue asynchronous stomp

Agenda - Lightweight job scheduling for Node.js

•    Javascript

---

Agenda is a light-weight job scheduling library for Node.js. It aims to keep its code base small. It is backed by Mongo backed persistence layer. It supports scheduling with configurable priority, concurrency, repeating and persistence of job results, Scheduling via cron or human readable syntax, Event backed job queue that you can hook into.

cron scheduler job-scheduler crontab cronjob recurring job queue task runner automated frequency interval jobs delayed job-queue task-queue

Pulsar - Distributed pub-sub Messaging System from Yahoo

•    Java

---

Pulsar is a distributed pub-sub messaging platform with a very flexible messaging model and an intuitive client API. It is horizontally scalable (Millions of independent topics and millions of messages published per second), Strong ordering and consistency guarantees, Low latency , REST API, Geo Replication and lot more.

messaging message-queue task-queue queue distributed pub-sub

enqueue-dev - PHP7.1+. Message queue packages for PHP, Symfony, Laravel, Yii, and Magento

•    PHP

---

Enqueue is production ready, battle-tested messaging solution for PHP. Provides a common way for programs to create, send, read messages. This is a main development repository. It provides a friendly environment for productive development and testing of all Enqueue related features&packages.

message-queue message-bus rabbitmq amqp stomp job-queue amazon-sqs redis doctrine-dbal rabbitmq-client message-passing beanstalkd gearman kafka symfony google-pubsub

MassTransit - Distributed Application Framework for .NET

•    CSharp

---

MassTransit is a lightweight message bus for creating distributed applications using the .NET framework. MassTransit provides an extensive set of features on top existing message transports, resulting in a developer friendly way to asynchronously connect services using message-based conversation patterns. Message-based communication is a reliable and scalable way to implement a service oriented architecture.

rabbitmq gitter-chat appveyor azure-service-bus topshelf messaging nuget dotnet-core2 queue message-queue task-queue

Exq - Job processing library for Elixir - compatible with Resque / Sidekiq

•    Elixir

---

Exq is a job processing library compatible with Resque / Sidekiq for the Elixir language.While you may reach for Sidekiq / Resque / Celery by default when writing apps in other languages, in Elixir there are some good options to consider that are already provided by the language and platform. So before adding Exq or any Redis backed queueing library to your application, make sure to get familiar with OTP and see if that is enough for your needs. Redis backed queueing libraries do add additional infrastructure complexity and also overhead due to serialization / marshalling, so make sure to evaluate whether or it is an actual need.

queue task-queue messaging job-queue cron