taskinator - A simple orchestration library for running complex processes or workflows in Ruby

  •        95

A simple orchestration library for running complex processes or workflows in Ruby. Processes are defined using a simple DSL, where the sequences and tasks are defined. Processes can then be queued for execution. Sequences can be synchronous or asynchronous, and the overall process can be monitored for completion or failure. The configuration and state of each process and their respective tasks is stored using Redis key/values.

https://github.com/virtualstaticvoid/taskinator

Tags
Implementation
License
Platform

   




Related Projects

hirefire

  •    Ruby

HireFire automatically "hires" and "fires" (aka "scales") Delayed Job and Resque workers on Heroku. When there are no queue jobs, HireFire will fire (shut down) all workers. If there are queued jobs, then it'll hire (spin up) workers. The amount of workers that get hired depends on the amount of queued jobs (the ratio can be configured by you). HireFire is great for both high, mid and low traffic applications. It can save you a lot of money by only hiring workers when there are pending jobs, and

meskyanichi-hirefire

  •    Ruby

HireFire automatically "hires" and "fires" (aka "scales") Delayed Job and Resque workers on Heroku. When there are no queue jobs, HireFire will fire (shut down) all workers. If there are queued jobs, then it'll hire (spin up) workers. The amount of workers that get hired depends on the amount of queued jobs (the ratio can be configured by you). HireFire is great for both high, mid and low traffic applications. It can save you a lot of money by only hiring workers when there are pending jobs, and

gush - Fast and distributed workflow runner using only Sidekiq and Redis

  •    Ruby

Gush is a parallel workflow runner using only Redis as storage and ActiveJob for scheduling and executing jobs. Gush relies on directed acyclic graphs to store dependencies, see Parallelizing Operations With Dependencies by Stephen Toub to learn more about this method.

sidekiq-cron - Scheduler / Cron for Sidekiq jobs

  •    Ruby

A scheduling add-on for Sidekiq. Runs a thread alongside Sidekiq workers to schedule jobs at specified times (using cron notation * * * * * parsed by Rufus-Scheduler, more about cron notation.


verk - A job processing system that just verks!

  •    Elixir

Verk is a job processing system backed by Redis. It uses the same job definition of Sidekiq/Resque. The goal is to be able to isolate the execution of a queue of jobs as much as possible.

Hangfire - An easy way to perform background job processing in your

  •    CSharp

Incredibly easy way to perform fire-and-forget, delayed and recurring jobs inside ASP.NET applications. CPU and I/O intensive, long-running and short-running jobs are supported. No Windows Service / Task Scheduler required. Backed by Redis, SQL Server, SQL Azure and MSMQ.Hangfire is a .NET Framework alternative to Resque, Sidekiq, delayed_job, Celery.

carrierwave_backgrounder - Offload CarrierWave's image processing and storage to a background process using Delayed Job, Resque, Sidekiq, Qu, Queue Classic or Girl Friday

  •    Ruby

I like CarrierWave. That being said, I don't like tying up app instances waiting for images to process. This gem addresses that by offloading processing or storage/processing to a background task. We currently support Delayed Job, Resque, Sidekiq, SuckerPunch, Girl Friday, Qu, and Queue Classic.

Faktory - "Takin' care of business, workin' overtime"

  •    Go

At a high level, Faktory is a work server. It is the repository for background jobs within your application. Jobs have a type and a set of arguments and are placed into queues for workers to fetch and execute.You can use this server to distribute jobs to one or hundreds of machines. Jobs can be executed with any language by clients using the Faktory API to fetch a job from a queue.

Sidekiq - Simple, efficient background processing for Ruby

  •    Ruby

Simple, efficient background processing for Ruby. Sidekiq uses threads to handle many jobs at the same time in the same process. It does not require Rails but will integrate tightly with Rails 3/4 to make background processing dead simple. Sidekiq uses multithreading so it is much more memory efficient than Resque (which forks a new process for every job).

recommendable - :+1::-1: A recommendation engine using Likes and Dislikes for your Ruby app

  •    Ruby

Recommendable is a gem that allows you to quickly add a recommendation engine for Likes and Dislikes to your Ruby application using my version of Jaccardian similarity and memory-based collaborative filtering. Bundling one of the queueing systems above is highly recommended to avoid having to manually refresh users' recommendations. If you bundle Sidekiq, you should also include 'sidekiq-middleware' in your Gemfile to ensure that a user will not get enqueued more than once at a time. If bundling Resque, you should include 'resque-loner' for this. As far as I know, there is no current way to avoid duplicate jobs in DelayedJob. Queueing for Torquebox is also supported.

sidekiq-statistic - See statistic about your workers

  •    Ruby

Improved display of statistics for your sidekiq workers and jobs. Open Statistic tab on your sidekiq page.

sidekiq-failures - Keep track of Sidekiq failed jobs

  •    Ruby

Keeps track of Sidekiq failed jobs and adds a tab to the Web UI to let you browse them. Makes use of Sidekiq's custom tabs and middleware chain. It mimics the way Resque keeps track of failures.

sidekiq-status - an extension to the sidekiq message processing to track your jobs

  •    Ruby

An extension to Sidekiq message processing to track your jobs. Inspired by resque-status and mostly copying its features, using Sidekiq's middleware. Fully compatible with ActiveJob.

goworker - Go-based background worker

  •    Go

goworker is a Go-based background worker that runs 10 to 100,000* times faster than Ruby-based workers. goworker is compatible with Resque, so you can push your jobs with Rails and Resque, and consume them with Go in the background

php-resque - PHP port of resque (Workers and Queueing)

  •    PHP

Resque is a Redis-backed library for creating background jobs, placing those jobs on one or more queues, and processing them later. Resque was pioneered and is developed by the fine folks at GitHub (yes, I am a kiss-ass), and written in Ruby. What you're seeing here is an almost direct port of the Resque worker and enqueue system to PHP.

node-resque - It's Resque... For node! Background jobs using redis.

  •    Javascript

new queue requires only the "queue" variable to be set. You can also pass the jobs hash to it.The configuration hash passed to new NodeResque.Worker, new NodeResque.Scheduler or new NodeResque.Queue can also take a connection option.

argo-workflows - Workflow engine for Kubernetes

  •    Go

Argo Workflows is an open source container-native workflow engine for orchestrating parallel jobs on Kubernetes. Argo Workflows is implemented as a Kubernetes CRD (Custom Resource Definition). Argo is a Cloud Native Computing Foundation (CNCF) hosted project.

sidekiq-unique-jobs - The missing unique jobs in sidekiq

  •    Ruby

See https://github.com/mperham/sidekiq#requirements for what is required. Starting from 5.0.0 only sidekiq >= 4 is supported and support for MRI <= 2.1 is dropped. Sidekiq consists of a client and a server. The client is responsible for pushing jobs to the queue and the worker is responsible for popping jobs from the queue. Most of the uniqueness is handled when the client is pushing jobs to the queue. The client checks if it is allowed to put a job on the queue. This is probably the most common way of locking.






We have large collection of open source products. Follow the tags from Tag Cloud >>


Open source products are scattered around the web. Please provide information about the open source projects you own / you use. Add Projects.