sidekiq_profiling_middleware - Profile Sidekiq with StackProf & MemoryProfiler

  •        3

Profile Sidekiq with StackProf & MemoryProfiler with optional support for S3 exports. The hooks provided by Ruby that are used by StackProf & MemoryProfiler are not able to distinguish threads well, so you will need to set your Sidekiq concurrency to 1. This will ensure that only one job operates at a time ensuring other jobs don't confuse your memory profile or stacktraces.

https://github.com/callumj/sidekiq_profiling_middleware

Tags
Implementation
License
Platform

   




Related Projects

sidekiq-statistic - See statistic about your workers

  •    Ruby

Improved display of statistics for your sidekiq workers and jobs. Open Statistic tab on your sidekiq page.

sidekiq-failures - Keep track of Sidekiq failed jobs

  •    Ruby

Keeps track of Sidekiq failed jobs and adds a tab to the Web UI to let you browse them. Makes use of Sidekiq's custom tabs and middleware chain. It mimics the way Resque keeps track of failures.

recommendable - :+1::-1: A recommendation engine using Likes and Dislikes for your Ruby app

  •    Ruby

Recommendable is a gem that allows you to quickly add a recommendation engine for Likes and Dislikes to your Ruby application using my version of Jaccardian similarity and memory-based collaborative filtering. Bundling one of the queueing systems above is highly recommended to avoid having to manually refresh users' recommendations. If you bundle Sidekiq, you should also include 'sidekiq-middleware' in your Gemfile to ensure that a user will not get enqueued more than once at a time. If bundling Resque, you should include 'resque-loner' for this. As far as I know, there is no current way to avoid duplicate jobs in DelayedJob. Queueing for Torquebox is also supported.

sidekiq-status - an extension to the sidekiq message processing to track your jobs

  •    Ruby

An extension to Sidekiq message processing to track your jobs. Inspired by resque-status and mostly copying its features, using Sidekiq's middleware. Fully compatible with ActiveJob.

rack-perftools_profiler - Middleware for profiling Rack-compatible apps using perftools.rb

  •    Ruby

Middleware for profiling Rack-compatible apps using perftools.rb


sidekiq-cron - Scheduler / Cron for Sidekiq jobs

  •    Ruby

A scheduling add-on for Sidekiq. Runs a thread alongside Sidekiq workers to schedule jobs at specified times (using cron notation * * * * * parsed by Rufus-Scheduler, more about cron notation.

django-silk - Silky smooth profiling for Django

  •    Python

Note: The middleware placement is sensitive. If the middleware before silk.middleware.SilkyMiddleware returns from process_request then SilkyMiddleware will never get the chance to execute. Therefore you must ensure that any middleware placed before never returns anything from process_request. See the django docs for more information on this. Note: If you are using django.middleware.gzip.GZipMiddleware, place that before silk.middleware.SilkyMiddleware, otherwise you will get an encoding error.

sidekiq-unique-jobs - The missing unique jobs in sidekiq

  •    Ruby

See https://github.com/mperham/sidekiq#requirements for what is required. Starting from 5.0.0 only sidekiq >= 4 is supported and support for MRI <= 2.1 is dropped. Sidekiq consists of a client and a server. The client is responsible for pushing jobs to the queue and the worker is responsible for popping jobs from the queue. Most of the uniqueness is handled when the client is pushing jobs to the queue. The client checks if it is allowed to put a job on the queue. This is probably the most common way of locking.

rspec-sidekiq - RSpec for Sidekiq

  •    Ruby

rspec-sidekiq requires sidekiq/testing by default so there is no need to include the line require "sidekiq/testing" inside your spec_helper.rb. IMPORTANT! This has the effect of not pushing enqueued jobs to Redis but to a job array to enable testing (see the FAQ & Troubleshooting Wiki page). Thus, only include gem "rspec-sidekiq" in environments where this behaviour is required, such as the test group.

sidekiq-scheduler - Lightweight job scheduler extension for Sidekiq

  •    Ruby

sidekiq-scheduler is an extension to Sidekiq that pushes jobs in a scheduled way, mimicking cron utility. Note: If you are looking for version 2.2.*, go to 2.2-stable branch.

sidekiq-limit_fetch - Sidekiq strategy to support an advanced queue control – limiting, pausing, blocking, querying

  •    Ruby

Sidekiq strategy to support a granular queue control – limiting, pausing, blocking, querying. You must require 'sidekiq-limit_fetch' if it isn't already. It will not work until then.

capistrano-sidekiq - Sidekiq integration for Capistrano

  •    Ruby

There is a known bug that prevents sidekiq from starting when pty is true on Capistrano 3. You can configure sidekiq to start with multiple processes. Just set the proper amount in sidekiq_processes.

Sidekiq - Simple, efficient background processing for Ruby

  •    Ruby

Simple, efficient background processing for Ruby. Sidekiq uses threads to handle many jobs at the same time in the same process. It does not require Rails but will integrate tightly with Rails 3/4 to make background processing dead simple. Sidekiq uses multithreading so it is much more memory efficient than Resque (which forks a new process for every job).

Exq - Job processing library for Elixir - compatible with Resque / Sidekiq

  •    Elixir

Exq is a job processing library compatible with Resque / Sidekiq for the Elixir language.While you may reach for Sidekiq / Resque / Celery by default when writing apps in other languages, in Elixir there are some good options to consider that are already provided by the language and platform. So before adding Exq or any Redis backed queueing library to your application, make sure to get familiar with OTP and see if that is enough for your needs. Redis backed queueing libraries do add additional infrastructure complexity and also overhead due to serialization / marshalling, so make sure to evaluate whether or it is an actual need.

sidetiq - NO LONGER MAINTAINED Recurring jobs for Sidekiq

  •    Ruby

Recurring jobs for Sidekiq. Due to changes in Sidekiq (which would necessitate an almost complete rewrite) and the move to the ActiveJob abstraction this library is no longer maintained. Please refer to alternatives instead.

pyflame - Pyflame: A Ptracing Profiler For Python

  •    C++

Pyflame is a high performance profiling tool that generates flame graphs for Python. Pyflame is implemented in C++, and uses the Linux ptrace(2) system call to collect profiling information. It can take snapshots of the Python call stack without explicit instrumentation, meaning you can profile a program without modifying its source code. Pyflame is capable of profiling embedded Python interpreters like uWSGI. It fully supports profiling multi-threaded Python programs.Pyflame usually introduces significantly less overhead than the builtin profile (or cProfile) modules, and emits richer profiling data. The profiling overhead is low enough that you can use it to profile live processes in production.

Home - Project Glimpse: Node Edition - Spend less time debugging and more time developing.

  •    

Glimpse is an experimental npm package that gives you in-depth insights about the client and server sides of your Node.js apps. More efficient debugging means faster development. Best of all, it’s free.Full details and documentation available at http://node.getglimpse.com.

coz - Coz: Causal Profiling

  •    C++

Coz is a new kind of profiler that unlocks optimization opportunities missed by traditional profilers. Coz employs a novel technique we call causal profiling that measures optimization potential. This measurement matches developers' assumptions about profilers: that optimizing highly-ranked code will have the greatest impact on performance. Causal profiling measures optimization potential for serial, parallel, and asynchronous programs without instrumentation of special handling for library calls and concurrency primitives. Instead, a causal profiler uses performance experiments to predict the effect of optimizations. This allows the profiler to establish causality: "optimizing function X will have effect Y," exactly the measurement developers had assumed they were getting all along. Full details of Coz are available in our paper, Coz: Finding Code that Counts with Causal Profiling (pdf), SOSP 2015, October 2015 (recipient of a Best Paper Award).

Faktory - "Takin' care of business, workin' overtime"

  •    Go

At a high level, Faktory is a work server. It is the repository for background jobs within your application. Jobs have a type and a set of arguments and are placed into queues for workers to fetch and execute.You can use this server to distribute jobs to one or hundreds of machines. Jobs can be executed with any language by clients using the Faktory API to fetch a job from a queue.