huststore - High-performance Distributed Storage

  •        27

huststore is a open source high performance distributed database system. It not only provides key-value storage service with extremely high performance, up to 100 thousand QPS, but also supports data structures like hash, set, sorted set, etc. Also, it can store binary data as value from a key-value pair, and thus can be used as an alternative of Redis.In addtion, huststore implements a distributed message queue by integrating a special HA module, features including message Push Stream, and message Publish-SubScribe, these features can be used as replacements of the corresponding features in rabbitmq and gearman.



Related Projects

disque - Disque is a distributed message broker

  •    C

Disque is an ongoing experiment to build a distributed, in-memory, message broker. Its goal is to capture the essence of the "Redis as a jobs queue" use case, which is usually implemented using blocking list operations, and move it into an ad-hoc, self-contained, scalable, and fault tolerant design, with simple to understand properties and guarantees, but still resembling Redis in terms of simplicity, performance, and implementation as a C non-blocking networked server.Currently (2 Jan 2016) the project is in release candidate state. People are encouraged to start evaluating it and report bugs and experiences.

RocketMQ - Distributed messaging and streaming data platform

  •    Java

Apache RocketMQ is a distributed messaging and streaming platform with low latency, high performance and reliability, trillion-level capacity and flexible scalability.

SwiftQ - Distributed Task Queue

  •    Swift

SwiftQ is a distributed task queue for server side swift applications. Task queues are used as a mechanism to distribute work across machines. SwiftQ uses messages to communicate between clients and workers. In this case the message broker is Redis. SwiftQ uses the reliable queue pattern. This ensures that all tasks get processed even in the event of networking problems or consumer crashes. SwiftQ can be used for real time operations as well as delayed execution of tasks. SwiftQ can consist of multiple producers and consumers allowing for high availability and horizontal scaling. It is recommended that a separate Redis database is used to avoid conflicting name space.

Bee Queue - A simple, fast, robust job/task queue for Node.js, backed by Redis

  •    Javascript

A simple, fast, robust job/task queue for Node.js, backed by Redis.Bee-Queue is meant to power a distributed worker pool and was built with short, real-time jobs in mind. A web server can enqueue a job, wait for a worker process to complete it, and return its results within an HTTP request. Scaling is as simple as running more workers.

redismq - a durable message queue system for go based on redis, see also https://github

  •    Go

Note: This project is no longer actively maintained. Please refer to its spiritual successor rmq. This is a fast, persistent, atomic message queue implementation that uses redis as its storage engine written in go. It uses atomic list commands to ensure that messages are delivered only once in the right order without being lost by crashing consumers.

Redisson - Redis based In-Memory Data Grid for Java

  •    Java

Redisson - distributed Java objects and services (Set, Multimap, SortedSet, Map, List, Queue, BlockingQueue, Deque, BlockingDeque, Semaphore, Lock, AtomicLong, Map Reduce, Publish / Subscribe, Bloom filter, Spring Cache, Executor service, Tomcat Session Manager, Scheduler service, JCache API) on top of Redis server. Rich Redis client.

tidis - Distributed transactional NoSQL database, Redis protocol compatible using tikv as backend

  •    Go

Tidis is a Distributed NoSQL database, providing a Redis protocol API (string, list, hash, set, sorted set), written in Go. Tidis is like TiDB layer, providing protocol transform and data structure compute, powered by TiKV backend distributed storage which use Raft for data replication and 2PC for distributed transaction.

MassTransit - Distributed Application Framework for .NET

  •    CSharp

MassTransit is a lightweight message bus for creating distributed applications using the .NET framework. MassTransit provides an extensive set of features on top existing message transports, resulting in a developer friendly way to asynchronously connect services using message-based conversation patterns. Message-based communication is a reliable and scalable way to implement a service oriented architecture.

siberite - Siberite is a simple, lightweight, leveldb backed message queue written in Go.

  •    Go

Siberite is a simple leveldb backed message queue server (twitter/kestrel, wavii/darner rewritten in Go). Siberite is a very simple message queue server. Unlike in-memory servers such as redis, Siberite is designed to handle queues much larger than what can be held in RAM. And unlike enterprise queue servers such as RabbitMQ, Siberite keeps all messages out of process, using goleveldb as a persistent storage.

rsmq - Redis Simple Message Queue

  •    CoffeeScript

A lightweight message queue for Node.js that requires no dedicated queue server. Just a Redis server. tl;dr: If you run a Redis server and currently use Amazon SQS or a similar message queue you might as well use this fast little replacement. Using a shared Redis server multiple Node.js processes can send / receive messages.

darner - simple, lightweight message queue

  •    C++

Darner is a very simple message queue server. Unlike in-memory servers such as redis, Darner is designed to handle queues much larger than what can be held in RAM. And unlike enterprise queue servers such as RabbitMQ, Darner keeps all messages out of process, relying instead on the kernel's virtual memory manager via log-structured storage. The result is a durable queue server that uses a small amount of in-resident memory regardless of queue size, while still achieving remarkable performance.

rmq - Message queue system written in Go and backed by Redis

  •    Go

rmq is short for Redis message queue. It's a message queue system written in Go and backed by Redis. It's similar to redismq, but implemented independently with a different interface in mind. Lets take a look at how to use rmq.

Dramatiq - Simple distributed task processing for Python 3.

  •    Python

dramatiq is a distributed task processing library for Python with a focus on simplicity, reliability and performance.

OfflineSampleApp - Sample Offline-First MVVM app that uses Android Priority Job Queue, Room, Retrofit2, LiveData, LifecycleObserver, RxJava2, Dagger Android

  •    Java

Offline App (or Offline-First App) enables user to seamlessly interact with it by using local device storage and then synchronizing the data with some remote storage (cloud database, etc) later via a background process. This app is a working sample that showcases one way of implementing offline commenting capability on Android platform. Users' comments are stored in local Room database first. Then a background job is spawned to synchronize local data with remote database if and when Internet connection is available.

ardb - A redis protocol compatible nosql, it support multiple storage engines as backend like Google's LevelDB, Facebook's RocksDB, OpenLDAP's LMDB, PerconaFT, WiredTiger, ForestDB

  •    C++

Ardb is a BSD licensed, redis-protocol compatible persistent nosql, it support multiple storage engines as backend like Google's LevelDB, Facebook's RocksDB, OpenLDAP's LMDB, WiredTiger, PerconaFT,Couchbase's ForestDB the default backend is Facebook's RocksDB. Rocksdb is the default storage engine, to compile with rocksdb, just type make to compile server & lib & tests.

Celery - Distributed Task Queue

  •    Python

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well. The execution units, called tasks, are executed concurrently on a single or more worker servers using multiprocessing, Eventlet, or gevent. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).

django-websocket-redis - Websockets for Django applications using Redis as message queue

  •    Python

Detailed documentation on ReadTheDocs. This module implements websockets on top of Django without requiring any additional framework. For messaging it uses the Redis datastore and in a production environment, it is intended to work under uWSGI and behind NGiNX or Apache version 2.4.5 or later.

zimg - A lightweight and high performance image storage and processing system.

  •    C

Project zimg is a lightweight image storage and processing system. It's written in C and it has high performance in image field. The zimg is designed for high concurrency image server. It supports many features for storing and processing images.The concurrent I/O, distributed storage and in time processing ability of zimg is excellent. You needn't nginx in your image server any more. In the benchmark test, zimg can deal with 3000+ image downloading task per second and 90000+ HTTP echo request per second on a high concurrency level. The performance is higher than PHP or other image processing server. More infomation of zimg is in the documents below.