Displaying 1 to 17 from 17 results

Sidekiq - Simple, efficient background processing for Ruby

  •    Ruby

Simple, efficient background processing for Ruby. Sidekiq uses threads to handle many jobs at the same time in the same process. It does not require Rails but will integrate tightly with Rails 3/4 to make background processing dead simple. Sidekiq uses multithreading so it is much more memory efficient than Resque (which forks a new process for every job).

Celery - Distributed Task Queue

  •    Python

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well. The execution units, called tasks, are executed concurrently on a single or more worker servers using multiprocessing, Eventlet, or gevent. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).

workq - Job server in Go

  •    Go

Workq is a job scheduling server strictly focused on simplifying job processing and streamlining coordination. It can run jobs in blocking foreground or non-blocking background mode. Workq runs as a standalone TCP server and implements a simple, text based protocol. Clients interact with Workq over a TCP socket in a request/response model with text commands. Please refer to the full protocol doc for details.

huey - a little task queue for python

  •    Python

huey is a little task queue in Python. It supports multi-process, multi-thread or greenlet task execution models. It can schedule tasks to execute at a given time, or after a given delay, schedule recurring tasks, like a crontab, retry tasks that fail automatically, task result storage.




goworker - Go-based background worker

  •    Go

goworker is a Go-based background worker that runs 10 to 100,000* times faster than Ruby-based workers. goworker is compatible with Resque, so you can push your jobs with Rails and Resque, and consume them with Go in the background

tasktiger - Python task queue. Because celery is gross.

  •    Python

TaskTiger is a Python task queue using Redis.TaskTiger forks a subprocess for each task, This comes with several benefits: Memory leaks caused by tasks are avoided since the subprocess is terminated when the task is finished. A hard time limit can be set for each task, after which the task is killed if it hasn't completed. To ensure performance, any necessary Python modules can be preloaded in the parent process.

parallelizer - Simplifies the parallelization of function calls

  •    Go

Running multiple function calls in parallel without a timeout.Running multiple function calls in parallel with a large enough worker pool.

ost - Redis based queues and workers.

  •    Ruby

Redis based queues and workers.Ost makes it easy to enqueue object ids and process them with workers.


Worker - The Hoa\Worker library.

  •    PHP

Hoa is a modular, extensible and structured set of PHP libraries. Moreover, Hoa aims at being a bridge between industrial and research worlds. This library allows to create shared workers in order to lift out some heavy and blocking tasks.

mi-prometheus - Enabling reproducible Machine Learning research

  •    Python

MI-Prometheus (Machine Intelligence - Prometheus), an open-source framework aiming at accelerating Machine Learning Research, by fostering the rapid development of diverse neural network-based models and facilitating their comparison. In its core, to accelerate the computations on their own, MI-Prometheus relies on PyTorch and extensively uses its mechanisms for the distribution of computations on CPUs/GPUs. In MI-Prometheus, the training & testing mechanisms are no longer pinned to a specific model or problem, and built-in mechanisms for easy configuration management & statistics collection facilitate running experiments combining different models with problems.

laravel-aws-worker - Run Laravel (or Lumen) tasks and queue listeners inside of AWS Elastic Beanstalk workers

  •    PHP

Laravel documentation recommends to use supervisor for queue workers and *IX cron for scheduled tasks. However, when deploying your application to AWS Elastic Beanstalk, neither option is available. This package helps you run your Laravel (or Lumen) jobs in AWS worker environments.

toolkit - Collection of useful patterns

  •    Go

These patterns can you use to solve common problems when designing an application or system. If you'd like to contribute to toolkit, please fork, fix, commit and send a pull request for the maintainers to review and merge into the main code base to ensure those changes are in line with the general philosophy of the project and/or get some early feedback which can make both your efforts much lighter as well as our review and merge procedures quick and simple.

gocraft-work-adapter - Implements the github

  •    Go

This package implements the github.com/gobuffalo/buffalo/worker.Worker interface using the github.com/gocraft/work package.

jobserver - A skeleton application for creating and processing background jobs.

  •    PHP

JobServer is a skeleton repository used for creating and processing background jobs backed by Tarantool. It contains configuration files and folders you will need for quick start from scratch. First, create your own docker-compose.override.yml file by copying docker-compose.override.yml.dist (or docker-compose.override.yml.full.dist if you want to test the full setup including a Tarantool cluster with automatic failover and monitoring tools) and customize to your needs. Do the same for .env.dist and all *.dist files located in app/config and res.

async-worker - Microframework para escrever consumer assíncronos em python

  •    Python

Nesse exemplo, o handler drain_handler() recebe mensagens de ambas as filas: asgard/counts e asgard/counts/errors. Se o handler lançar alguma exception, a mensagem é automaticamente devolvida para a fila (reject com requeue=True); Se o handler rodar sem erros, a mensagem é automaticamente confirmada (ack).

gopool - Easy to use worker pool with dynamic pool sizing.

  •    Go

Go Pool makes it easy to set up and manage pools of background workers. Not only can they be started and stopped, but they can also be dynamically sized ensuring you have the optimal amount of workers running for your situation. To start a pool of long running workers who's size will be managed by gopool, you should use the Pool.Start func.

amqp-work-adapter - AMQP worker adapter for Buffalo

  •    Go

This package implements the github.com/gobuffalo/buffalo/worker.Worker interface using the github.com/streadway/amqp package. It allows AMQP-compatible message brokers, such as RabbitMQ, to process Buffalo's background tasks.