Displaying 1 to 20 from 29 results

rq - Simple job queues for Python

  •    Python

RQ (Redis Queue) is a simple Python library for queueing jobs and processing them in the background with workers. It is backed by Redis and it is designed to have a low barrier to entry. It should be integrated in your web stack easily.RQ requires Redis >= 2.7.0.

MassTransit - Distributed Application Framework for .NET

  •    CSharp

MassTransit is a lightweight message bus for creating distributed applications using the .NET framework. MassTransit provides an extensive set of features on top existing message transports, resulting in a developer friendly way to asynchronously connect services using message-based conversation patterns. Message-based communication is a reliable and scalable way to implement a service oriented architecture.

Buildbot - Continuous Integration Framework

  •    Python

Buildbot is a continuous integration system designed to automate the build/test cycle. By automatically rebuilding and testing the tree each time something has changed, build problems are pinpointed quickly, before other developers are inconvenienced by the failure. It supports arbitrary build processes, and is not limited to common build processes for particular languages. Buildbot has a variety of status-reporting tools to get information about builds in a timely manner.




Celery - Distributed Task Queue

  •    Python

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well. The execution units, called tasks, are executed concurrently on a single or more worker servers using multiprocessing, Eventlet, or gevent. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).

RabbitMQ - Robust messaging for applications

  •    Erlang

RabbitMQ is a messaging broker - an intermediary for messaging. It gives your applications a common platform to send and receive messages, and your messages a safe place to live until received. It features include reliability, high availability, Clustering and Federation. RabbitMQ ships with an easy-to use management UI that allows you to monitor and control every aspect of your message broker. There are RabbitMQ clients for almost any language you can think of.

django-rq - A simple app that provides django integration for RQ (Redis Queue)

  •    Python

Django integration with RQ, a Redis based Python queuing library. Django-RQ is a simple app that allows you to configure your queues in django's settings.py and easily use them in your project. With this setting, job decorator will set result_ttl to 5000 unless it's specified explicitly.

flower - Real-time monitor and web admin for Celery distributed task queue

  •    Python

Flower is a web based tool for monitoring and administrating Celery clusters. Flower API enables to manage the cluster via REST API, call tasks and receive task events in real-time via WebSockets.


Pulsar - Distributed pub-sub Messaging System from Yahoo

  •    Java

Pulsar is a distributed pub-sub messaging platform with a very flexible messaging model and an intuitive client API. It is horizontally scalable (Millions of independent topics and millions of messages published per second), Strong ordering and consistency guarantees, Low latency , REST API, Geo Replication and lot more.

huey - a little task queue for python

  •    Python

huey is a little task queue in Python. It supports multi-process, multi-thread or greenlet task execution models. It can schedule tasks to execute at a given time, or after a given delay, schedule recurring tasks, like a crontab, retry tasks that fail automatically, task result storage.

Faktory - "Takin' care of business, workin' overtime"

  •    Go

At a high level, Faktory is a work server. It is the repository for background jobs within your application. Jobs have a type and a set of arguments and are placed into queues for workers to fetch and execute.You can use this server to distribute jobs to one or hundreds of machines. Jobs can be executed with any language by clients using the Faktory API to fetch a job from a queue.

Exq - Job processing library for Elixir - compatible with Resque / Sidekiq

  •    Elixir

Exq is a job processing library compatible with Resque / Sidekiq for the Elixir language.While you may reach for Sidekiq / Resque / Celery by default when writing apps in other languages, in Elixir there are some good options to consider that are already provided by the language and platform. So before adding Exq or any Redis backed queueing library to your application, make sure to get familiar with OTP and see if that is enough for your needs. Redis backed queueing libraries do add additional infrastructure complexity and also overhead due to serialization / marshalling, so make sure to evaluate whether or it is an actual need.

Bee Queue - A simple, fast, robust job/task queue for Node.js, backed by Redis

  •    Javascript

A simple, fast, robust job/task queue for Node.js, backed by Redis.Bee-Queue is meant to power a distributed worker pool and was built with short, real-time jobs in mind. A web server can enqueue a job, wait for a worker process to complete it, and return its results within an HTTP request. Scaling is as simple as running more workers.

Dramatiq - Simple distributed task processing for Python 3.

  •    Python

dramatiq is a distributed task processing library for Python with a focus on simplicity, reliability and performance.

tasktiger - Python task queue. Because celery is gross.

  •    Python

TaskTiger is a Python task queue using Redis.TaskTiger forks a subprocess for each task, This comes with several benefits: Memory leaks caused by tasks are avoided since the subprocess is terminated when the task is finished. A hard time limit can be set for each task, after which the task is killed if it hasn't completed. To ensure performance, any necessary Python modules can be preloaded in the parent process.

Bistro - Flexible distributed scheduler, a high-performance framework supporting multiple paradigms while retaining ease of configuration, management, and monitoring

  •    C++

Bistro is a toolkit for making services that schedule and execute tasks. Bistro is a toolkit for making distributed computation systems. It can schedule and run distributed tasks, including data-parallel jobs. It enforces resource constraints for worker hosts and data-access bottlenecks. It supports remote worker pools, low-latency batch scheduling, dynamic shards, and a variety of other possibilities. It has command-line and web UIs.

sidekiq.cr - Simple, efficient job processing for Crystal

  •    Crystal

Sidekiq is a well-regarded background job framework for Ruby. Now we're bringing the awesomeness to Crystal, a Ruby-like language. Why? To give you options. Ruby is friendly and flexible but not terribly fast. Crystal is statically-typed, compiled and very fast but retains a similar syntax to Ruby.If you have jobs which are CPU-intensive or require very high throughput, Crystal is an excellent alternative to native Ruby extensions. It compiles to a single executable so deployment is much easier than Ruby.

evQueue - Job scheduler and queueing engine

  •    C++

evQueue is an open source job scheduler and queueing engine. It features an event-driven C++ engine and a PHP / MySQL web control interface which provides tasks monitoring and creation. It provides both simple task execution and complex task chaining (workflow) using an easy to use drag & drop web interface. Workflow description includes output linking to input, conditions, loops... Queues management provides an easy way for task parallelization and resource control.

react-native-queue - A React Native Job Queue

  •    Javascript

A React Native at-least-once priority job queue / task queue backed by persistent Realm storage. Jobs will persist until completed, even if user closes and re-opens app. React Native Queue is easily integrated into OS background processes (services) so you can ensure the queue will continue to process until all jobs are completed even if app isn't in focus. It also plays well with Workers so your jobs can be thrown on the queue, then processed in dedicated worker threads for greatly improved processing performance. React Native Queue is designed to be a swiss army knife for task management in React Native. It abstracts away the many annoyances related to processing complex tasks, like durability, retry-on-failure, timeouts, chaining processes, and more. Just throw your jobs onto the queue and relax - they're covered.

yotaq - yotaq - Your Own Task Queue for Python

  •    Python

So you need a task queue for your Python project. Sure you could check celery, and after three months trying to understand the basic configuration options you'll be good to go. Or you could use a simpler task queue like huey or rq. Pretty good. Our python code will use dill to serialize the functions to be run and redis to store the tasks.