We have collection of more than 1 Million open source products ranging from Enterprise product to small libraries in all platforms. We aggregate information from all open source repositories. Search and find the best for your needs. Check out projects section.
taskinator - A simple orchestration library for running complex processes or workflows in Ruby
- 95
A simple orchestration library for running complex processes or workflows in Ruby. Processes are defined using a simple DSL, where the sequences and tasks are defined. Processes can then be queued for execution. Sequences can be synchronous or asynchronous, and the overall process can be monitored for completion or failure. The configuration and state of each process and their respective tasks is stored using Redis key/values.
https://github.com/virtualstaticvoid/taskinator| Tags | workers workflow sidekiq distributed-computing cloud resque delayed-jobs |
| Implementation | Ruby |
| License | MIT |
| Platform | Windows Linux |
Related Projects
goflow - A Golang based high performance, scalable and distributed workflow framework
A Golang based high performance, scalable and distributed workflow framework
sidekiq framework workflow-engine resque resque-workers workload-automationhirefire
HireFire automatically "hires" and "fires" (aka "scales") Delayed Job and Resque workers on Heroku. When there are no queue jobs, HireFire will fire (shut down) all workers. If there are queued jobs, then it'll hire (spin up) workers. The amount of workers that get hired depends on the amount of queued jobs (the ratio can be configured by you). HireFire is great for both high, mid and low traffic applications. It can save you a lot of money by only hiring workers when there are pending jobs, and
meskyanichi-hirefire
HireFire automatically "hires" and "fires" (aka "scales") Delayed Job and Resque workers on Heroku. When there are no queue jobs, HireFire will fire (shut down) all workers. If there are queued jobs, then it'll hire (spin up) workers. The amount of workers that get hired depends on the amount of queued jobs (the ratio can be configured by you). HireFire is great for both high, mid and low traffic applications. It can save you a lot of money by only hiring workers when there are pending jobs, and
gush - Fast and distributed workflow runner using only Sidekiq and Redis
Gush is a parallel workflow runner using only Redis as storage and ActiveJob for scheduling and executing jobs. Gush relies on directed acyclic graphs to store dependencies, see Parallelizing Operations With Dependencies by Stephen Toub to learn more about this method.
workflows sidekiq graph parallel parallelization activejob queues redis workflow workerssidekiq-cron - Scheduler / Cron for Sidekiq jobs
A scheduling add-on for Sidekiq. Runs a thread alongside Sidekiq workers to schedule jobs at specified times (using cron notation * * * * * parsed by Rufus-Scheduler, more about cron notation.
sidekiq-cron sidekiq-jobs cron-jobs sidekiq-workers sidekiq scheduled-jobs scheduler ruby-on-rails ruby-gemHangfire - An easy way to perform background job processing in your
Incredibly easy way to perform fire-and-forget, delayed and recurring jobs inside ASP.NET applications. CPU and I/O intensive, long-running and short-running jobs are supported. No Windows Service / Task Scheduler required. Backed by Redis, SQL Server, SQL Azure and MSMQ.Hangfire is a .NET Framework alternative to Resque, Sidekiq, delayed_job, Celery.
dotnet-core background-jobs background-thread background-worker scheduled-jobs hangfirecarrierwave_backgrounder - Offload CarrierWave's image processing and storage to a background process using Delayed Job, Resque, Sidekiq, Qu, Queue Classic or Girl Friday
I like CarrierWave. That being said, I don't like tying up app instances waiting for images to process. This gem addresses that by offloading processing or storage/processing to a background task. We currently support Delayed Job, Resque, Sidekiq, SuckerPunch, Girl Friday, Qu, and Queue Classic.
Faktory - "Takin' care of business, workin' overtime"
At a high level, Faktory is a work server. It is the repository for background jobs within your application. Jobs have a type and a set of arguments and are placed into queues for workers to fetch and execute.You can use this server to distribute jobs to one or hundreds of machines. Jobs can be executed with any language by clients using the Faktory API to fetch a job from a queue.
message-queue job-queue task-queue background-taskSidekiq - Simple, efficient background processing for Ruby
Simple, efficient background processing for Ruby. Sidekiq uses threads to handle many jobs at the same time in the same process. It does not require Rails but will integrate tightly with Rails 3/4 to make background processing dead simple. Sidekiq uses multithreading so it is much more memory efficient than Resque (which forks a new process for every job).
job-queue job task queue workflow worker worker-queue background-taskrecommendable - :+1::-1: A recommendation engine using Likes and Dislikes for your Ruby app
Recommendable is a gem that allows you to quickly add a recommendation engine for Likes and Dislikes to your Ruby application using my version of Jaccardian similarity and memory-based collaborative filtering. Bundling one of the queueing systems above is highly recommended to avoid having to manually refresh users' recommendations. If you bundle Sidekiq, you should also include 'sidekiq-middleware' in your Gemfile to ensure that a user will not get enqueued more than once at a time. If bundling Resque, you should include 'resque-loner' for this. As far as I know, there is no current way to avoid duplicate jobs in DelayedJob. Queueing for Torquebox is also supported.
redis sidekiq recommendations railssidekiq-statistic - See statistic about your workers
Improved display of statistics for your sidekiq workers and jobs. Open Statistic tab on your sidekiq page.
sidekiq statistic-inside-middlewaresidekiq-failures - Keep track of Sidekiq failed jobs
Keeps track of Sidekiq failed jobs and adds a tab to the Web UI to let you browse them. Makes use of Sidekiq's custom tabs and middleware chain. It mimics the way Resque keeps track of failures.
sidekiq-status - an extension to the sidekiq message processing to track your jobs
An extension to Sidekiq message processing to track your jobs. Inspired by resque-status and mostly copying its features, using Sidekiq's middleware. Fully compatible with ActiveJob.
goworker - Go-based background worker
goworker is a Go-based background worker that runs 10 to 100,000* times faster than Ruby-based workers. goworker is compatible with Resque, so you can push your jobs with Rails and Resque, and consume them with Go in the background
job-queue job task queue workflow worker worker-queue background-taskphp-resque - PHP port of resque (Workers and Queueing)
Resque is a Redis-backed library for creating background jobs, placing those jobs on one or more queues, and processing them later. Resque was pioneered and is developed by the fine folks at GitHub (yes, I am a kiss-ass), and written in Ruby. What you're seeing here is an almost direct port of the Resque worker and enqueue system to PHP.
node-resque - It's Resque... For node! Background jobs using redis.
new queue requires only the "queue" variable to be set. You can also pass the jobs hash to it.The configuration hash passed to new NodeResque.Worker, new NodeResque.Scheduler or new NodeResque.Queue can also take a connection option.
delayed queue resque redis work worker background job taskargo-workflows - Workflow engine for Kubernetes
Argo Workflows is an open source container-native workflow engine for orchestrating parallel jobs on Kubernetes. Argo Workflows is implemented as a Kubernetes CRD (Custom Resource Definition). Argo is a Cloud Native Computing Foundation (CNCF) hosted project.
kubernetes workflow machine-learning airflow workflow-engine cncf argo k8s cloud-native hacktoberfest dag knative argo-workflowscouler - Unified Interface for Constructing and Managing Workflows on different workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow
Couler aims to provide a unified interface for constructing and managing workflows on different workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow. Couler is included in CNCF Cloud Native Landscape and LF AI Landscape.
kubernetes machine-learning workflow-engine scheduler distributed-computing cloud-native workflow-management unified-interface workflow-automation unified-api apache-airflow kubeflow argo-workflows tekton-pipelinessidekiq-unique-jobs - The missing unique jobs in sidekiq
See https://github.com/mperham/sidekiq#requirements for what is required. Starting from 5.0.0 only sidekiq >= 4 is supported and support for MRI <= 2.1 is dropped. Sidekiq consists of a client and a server. The client is responsible for pushing jobs to the queue and the worker is responsible for popping jobs from the queue. Most of the uniqueness is handled when the client is pushing jobs to the queue. The client checks if it is allowed to put a job on the queue. This is probably the most common way of locking.
sidekiq redis
We have large collection of open source products. Follow the tags from
Tag Cloud >>
Open source products are scattered around the web. Please provide information
about the open source projects you own / you use.
Add Projects.
