Displaying 1 to 4 from 4 results

Celery - Distributed Task Queue

  •    Python

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well. The execution units, called tasks, are executed concurrently on a single or more worker servers using multiprocessing, Eventlet, or gevent. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).

Gearman - Application Framework to farm out work to other Machines

  •    C

Gearman provides a generic application framework to farm out work to other machines or processes that are better suited to do the work. It allows you to do work in parallel by doing load balancing. It also supports to call functions between languages. It is the nervous system for how distributed processing communicates. It is fault tolerant.

Bistro - Flexible distributed scheduler, a high-performance framework supporting multiple paradigms while retaining ease of configuration, management, and monitoring

  •    C++

Bistro is a toolkit for making services that schedule and execute tasks. Bistro is a toolkit for making distributed computation systems. It can schedule and run distributed tasks, including data-parallel jobs. It enforces resource constraints for worker hosts and data-access bottlenecks. It supports remote worker pools, low-latency batch scheduling, dynamic shards, and a variety of other possibilities. It has command-line and web UIs.

node-dequeue - A simple double ended queue datastructure

  •    Javascript

Dequeue is implemented as a doubly linked circular list with a titular head node. By "titular head node", I mean an empty node to designate the beginning and end of the circularly linked list. I first saw this construction in the linux kernel source and it seem simple and elegant. I added the .length property to use it like I was using an Array. I was using a javascript Array as a FIFO. Somewhere between 100,000 and 200,000 entries the program performance went to hell (dev host is a MBP w/8GB RAM). 15 minutes later, I implemented a simple dequeue and my FIFO scales up to millions of entries.