sidecloq - Recurring / Periodic / Scheduled / Cron job extension for Sidekiq

  •        27

There are several options for running periodic tasks with Sidekiq, including sidekiq-scheduler, sidekiq-cron, as well as Sidekiq Pro. Each tackles the problem slightly differently. Sidecloq is inspired by various facets of these projects, as well as resque-scheduler. I urge you to take a look at all of these options to see what works best for you. Tested on MRI > 2, JRuby and Rubinius. Basically, if you can run Sidekiq, you can run Sidecloq. Note that Sidekiq >= 5 does not support MRI ruby < 2.2.2.

https://github.com/mattyr/sidecloq

Tags
Implementation
License
Platform

   




Related Projects

sidekiq-cron - Scheduler / Cron for Sidekiq jobs

  •    Ruby

A scheduling add-on for Sidekiq. Runs a thread alongside Sidekiq workers to schedule jobs at specified times (using cron notation * * * * * parsed by Rufus-Scheduler, more about cron notation.

later - A javascript library for defining recurring schedules and calculating future (or past) occurrences for them

  •    HTML

Later is a library for describing recurring schedules and calculating their future occurrences. It supports a very flexible schedule definition including support for composite schedules and schedule exceptions. Create new schedules manually, via Cron expression, via text expressions, or using a fully chainable API. For more information on SemVer, please visit http://semver.org/.

pg_cron - Run periodic jobs in PostgreSQL

  •    C

pg_cron can run multiple jobs in parallel, but it runs at most one instance of a job at a time. If a second run is supposed to start before the first one finishes, then the second run is queued and started as soon as the first run completes. An easy way to create a cron schedule is: crontab.guru.

django-cron - Write cron business logic as a Python class and let this app do the rest! It enables Django projects to schedule cron tasks, tracks their success / failures, manages contention (via a cache) etc

  •    Python

Django-cron lets you run Django/Python code on a recurring basis providing basic plumbing to track and execute tasks. The 2 most common ways in which most people go about this is either writing custom python scripts or a management command per cron (leads to too many management commands!). Along with that some mechanism to track success, failure etc. is also usually necesary. This app solves both issues to a reasonable extent. This is by no means a replacement for queues like Celery ( http://celeryproject.org/ ) etc.

sidetiq - NO LONGER MAINTAINED Recurring jobs for Sidekiq

  •    Ruby

Recurring jobs for Sidekiq. Due to changes in Sidekiq (which would necessitate an almost complete rewrite) and the move to the ActiveJob abstraction this library is no longer maintained. Please refer to alternatives instead.


sidekiq-scheduler - Lightweight job scheduler extension for Sidekiq

  •    Ruby

sidekiq-scheduler is an extension to Sidekiq that pushes jobs in a scheduled way, mimicking cron utility. Note: If you are looking for version 2.2.*, go to 2.2-stable branch.

crono - A time-based background job scheduler daemon (just like Cron) for Rails

  •    Ruby

Crono is a time-based background job scheduler daemon (just like Cron) for Ruby on Rails. Currently, there is no such thing as Ruby Cron for Rails. Well, there's Whenever but it works on top of Unix Cron, so you can't manage it from Ruby. Crono is pure Ruby. It doesn't use Unix Cron and other platform-dependent things. So you can use it on all platforms supported by Ruby. It persists job states to your database using Active Record. You have full control of jobs performing process. It's Ruby, so you can understand and modify it to fit your needs.

node-cron - Cron for NodeJS.

  •    Javascript

Cron is a tool that allows you to execute something on a schedule. This is typically done using the cron syntax. We allow you to execute a function whenever your scheduled job triggers. We also allow you to execute a job external to the javascript process using child_process. Additionally, this library goes beyond the basic cron syntax and allows you to supply a Date object. This will be used as the trigger for your callback. Cron syntax is still an acceptable CronTime format. Although the Cron patterns supported here extend on the standard Unix format to support seconds digits, leaving it off will default to 0 and match the Unix behavior. Because we can't magically know what you are doing to expose an issue, it is best if you provide a snippet of code. This snippet need not include your secret sauce, but it must replicate the issue you are describing. The issues that get closed without resolution tend to be the ones without code examples. Thanks.

Atomix - Scalable, fault-tolerant distributed systems protocols and primitives for the JVM

  •    Java

Atomix is an event-driven framework for coordinating fault-tolerant distributed systems built on the Raft consensus algorithm. It provides the building blocks that solve many common distributed systems problems including group membership, leader election, distributed concurrency control, partitioning, and replication.

dagobah - Simple DAG-based job scheduler in Python

  •    CSS

Dagobah is a simple dependency-based job scheduler written in Python. Dagobah allows you to schedule periodic jobs using Cron syntax. Each job then kicks off a series of tasks (subprocesses) in an order defined by a dependency graph you can easily draw with click-and-drag in the web interface. Dagobah lets you retry individual tasks from failure, sends you helpful email reports on job completion and failure, keeps track of your tasks' stdout and stderr, and persists its information in various backends so you don't have to worry about losing your data.

Exq - Job processing library for Elixir - compatible with Resque / Sidekiq

  •    Elixir

Exq is a job processing library compatible with Resque / Sidekiq for the Elixir language.While you may reach for Sidekiq / Resque / Celery by default when writing apps in other languages, in Elixir there are some good options to consider that are already provided by the language and platform. So before adding Exq or any Redis backed queueing library to your application, make sure to get familiar with OTP and see if that is enough for your needs. Redis backed queueing libraries do add additional infrastructure complexity and also overhead due to serialization / marshalling, so make sure to evaluate whether or it is an actual need.

jobber - An alternative to cron, with sophisticated status-reporting and error-handling

  •    Go

A replacement for cron, with sophisticated status-reporting and error-handling. Jobber is a lightweight utility for Unix-like systems that can run arbitrary commands, or "jobs", according to a schedule. It is meant to be a replacement for the classic Unix utility cron.

django-celery-beat - Celery Periodic Tasks backed by the Django ORM

  •    Python

This extension enables you to store the periodic task schedule in the database. The periodic tasks can be managed from the Django Admin interface, where you can create, edit and delete periodic tasks and how often they should run.

kafka-monitor - Monitor the availability of Kakfa clusters with generated messages.

  •    Java

Kafka Monitor is a framework to implement and execute long-running kafka system tests in a real cluster. It complements Kafka’s existing system tests by capturing potential bugs or regressions that are only likely to occur after prolonged period of time or with low probability. Moreover, it allows you to monitor Kafka cluster using end-to-end pipelines to obtain a number of derived vital stats such as end-to-end latency, service availability and message loss rate. You can easily deploy Kafka Monitor to test and monitor your Kafka cluster without requiring any change to your application.Kafka Monitor can automatically create the monitor topic with the specified config and increase partition count of the monitor topic to ensure partition# >= broker#. It can also reassign partition and trigger preferred leader election to ensure that each broker acts as leader of at least one partition of the monitor topic. This allows Kafka Monitor to detect performance issue on every broker without requiring users to manually manage the partition assignment of the monitor topic.

Sidekiq - Simple, efficient background processing for Ruby

  •    Ruby

Simple, efficient background processing for Ruby. Sidekiq uses threads to handle many jobs at the same time in the same process. It does not require Rails but will integrate tightly with Rails 3/4 to make background processing dead simple. Sidekiq uses multithreading so it is much more memory efficient than Resque (which forks a new process for every job).

recommendable - :+1::-1: A recommendation engine using Likes and Dislikes for your Ruby app

  •    Ruby

Recommendable is a gem that allows you to quickly add a recommendation engine for Likes and Dislikes to your Ruby application using my version of Jaccardian similarity and memory-based collaborative filtering. Bundling one of the queueing systems above is highly recommended to avoid having to manually refresh users' recommendations. If you bundle Sidekiq, you should also include 'sidekiq-middleware' in your Gemfile to ensure that a user will not get enqueued more than once at a time. If bundling Resque, you should include 'resque-loner' for this. As far as I know, there is no current way to avoid duplicate jobs in DelayedJob. Queueing for Torquebox is also supported.

chronos - Fault tolerant job scheduler for Mesos which handles dependencies and ISO8601 based schedules

  •    Scala

Chronos is a replacement for cron. It is a distributed and fault-tolerant scheduler that runs on top of Apache Mesos that can be used for job orchestration. It supports custom Mesos executors as well as the default command executor. Thus by default, Chronos executes sh (on most systems bash) scripts. Chronos can be used to interact with systems such as Hadoop (incl. EMR), even if the Mesos agents on which execution happens do not have Hadoop installed. Included wrapper scripts allow transfering files and executing them on a remote machine in the background and using asynchronous callbacks to notify Chronos of job completion or failures. Chronos is also natively able to schedule jobs that run inside Docker containers.

minicron - 👨🏻‍💻 Monitor your cron jobs

  •    Ruby

minicron makes it simple to monitor your cron jobs and ensure they are running both correctly and on schedule. Latest stable release in 0.9.7 tag but 0.9.x is not being actively developed/supported. This branch is under active development for 1.0.0.

Hangfire - An easy way to perform background job processing in your

  •    CSharp

Incredibly easy way to perform fire-and-forget, delayed and recurring jobs inside ASP.NET applications. CPU and I/O intensive, long-running and short-running jobs are supported. No Windows Service / Task Scheduler required. Backed by Redis, SQL Server, SQL Azure and MSMQ.Hangfire is a .NET Framework alternative to Resque, Sidekiq, delayed_job, Celery.