Redpanda is a streaming platform for mission critical workloads. It is Kafka API compatible so any client that works with Kafka should work out of the box with Redpanda. It does not have dependency on ZooKeeper, JVM. Redpanda is a new storage engine, optimized for streaming data, using a thread-per-core architecture focused on delivering stable tail-latencies. It transforms data inline with embedded WASM engine.
Its features include:
|Tags||message-queue streaming kafka cpp containers realtime kubernetes queue kafka-protocol|
|Platform||Windows Linux MacOS|
KQ (Kafka Queue) is a lightweight Python library which lets you queue and execute jobs asynchronously using Apache Kafka. It uses kafka-python under the hood. You may need to use sudo depending on your environment.kafka job-queue async kafka-client kafka-consumer kafka-producer python3 python2 python-library python-3 python-2 asynchronous queueing jobqueue apache-kafka worker-queue producer-consumer serialization python-3-5
Liftbridge provides lightweight, fault-tolerant message streams by implementing a durable stream augmentation for the NATS messaging system. It extends NATS with a Kafka-like publish-subscribe log API that is highly available and horizontally scalable. Use Liftbridge as a simpler and lighter alternative to systems like Kafka and Pulsar or use it to add streaming semantics to an existing NATS deployment. See the introduction post on Liftbridge and this post for more context and some of the inspiration behind it.nats streaming messaging cloud-native grpc distributed-systems liftbridge message-queue pubsub
NSQ is a realtime distributed messaging platform designed to operate at scale, handling billions of messages per day. It promotes distributed and decentralized topologies without single points of failure, enabling fault tolerance and high availability coupled with a reliable message delivery guarantee. It scales horizontally, without any centralized brokers. Built-in discovery simplifies the addition of nodes to the cluster.message-queue messaging distributed publish-subscribe pubsub queue
This is an app to monitor your kafka consumers and their position (offset) in the queue. You can see the current consumer groups, for each group the topics that they are consuming and the position of the group in each topic queue. This is useful to understand how quick you are consuming from a queue and how fast the queue is growing. It allows for debuging kafka producers and consumers or just to have an idea of what is going on in your system.
Hermes is an asynchronous message broker built on top of Kafka. We provide reliable, fault tolerant REST interface for message publishing and adaptive push mechanisms for message sending.Questions? We are on gitter.hermes kafka pubsub messaging message-queue
Plumber is a CLI devtool for inspecting, piping, massaging and redirecting data in message systems like Kafka, RabbitMQ , GCP PubSub and many more. It helps to safely view the contents of your data streams, Decode protobuf/avro/thrift/JSON data in real-time, Route data from one place to another, Replay events into a message system on your local network and lot more.kafka protobuf rabbitmq message-bus message-queue event-driven event-bus cli command-line
Strimzi provides a way to run an Apache Kafka cluster on Kubernetes or OpenShift in various deployment configurations. See our website for more details about the project. Documentation to the current master branch as well as all releases can be found on our website.kafka kubernetes openshift docker messaging enmasse kafka-connect kafka-streams data-streaming data-stream data-streams kubernetes-operator kubernetes-controller
Enqueue is production ready, battle-tested messaging solution for PHP. Provides a common way for programs to create, send, read messages. This is a main development repository. It provides a friendly environment for productive development and testing of all Enqueue related features&packages.message-queue message-bus rabbitmq amqp stomp job-queue amazon-sqs redis doctrine-dbal rabbitmq-client message-passing beanstalkd gearman kafka symfony google-pubsub
Jafka mq is a distributed publish-subscribe messaging system cloned from Apache Kafka.publish-subscribe message-queue pub-sub distributed kafka-alternative kafka-clone
Kafka/distributed commit log service in Go. docker build -t travisjeffery/jocko:latest .kafka distributed-systems queue messaging streaming
Lenses offers SQL (for data browsing and Kafka Streams), Kafka Connect connector management, cluster monitoring and more. A collection of components to build a real time ingestion pipeline.kafka kafka-connect connector streaming cassandra hazelcast redis elasticsearch ftp influxdb coap mqtt kudu jms hbase mongodb rethinkdb documentdb cosmosdb kubernetes
MeiliES is an Event Sourcing database that uses the RESP (REdis Serialization Protocol) to communicate. This way it is possible to create clients by reusing the already available protocol. For example, it is possible to use the official Redis command line interface program to communicate with MeiliES. There is a release blog post if you want to know more. An event store is like a Kafka or a Rabbit MQ but it stores events on disk indefinitely. The first purpose of the server is to publish events of a stream to all subscribed clients, note that events are saved in reception order. A client can also specify from which event number (incrementing) it wants to read, therefore it is possible to recover from crashing by reading and reconstructing a state with only new events. Keep in mind that a message queue is not made for event-sourcing.event-store redis-protocol
Nakadi is a distributed event bus broker that implements a RESTful API abstraction on top of Kafka-like queues. It provides abstract event delivery via a secured RESTful API, Enable convenient development of event-driven applications and asynchronous microservices, Efficient low latency event delivery.apis event-bus restful microservices postgresql kafka messaging message-queue
Apache RocketMQ is a distributed messaging and streaming platform with low latency, high performance and reliability, trillion-level capacity and flexible scalability.message-queue messaging queue pub-sub distributed
Kafdrop is a web UI for viewing Kafka topics and browsing consumer groups. The tool displays information such as brokers, topics, partitions, consumers, and lets you view messages.docker kubernetes kafka topic web-ui event-sourcing zookeeper pub-sub kafka-utils consumer-producer consumer-group event-streaming kafka-tools kafka-ui kafka-management
Kafka UI is a UI for Apache Kafka to monitor and manage Apache Kafka clusters. It is a simple tool that makes your data flows observable, helps find and troubleshoot issues faster and deliver optimal performance. Its lightweight dashboard makes it easy to track key metrics of your Kafka clusters - Brokers, Topics, Partitions, Production, and Consumption.kafka big-data web-ui streams kafka-connect apache-kafka-ui kafka-producer kafka-client kafka-streams hacktoberfest streaming-data kafka-manager kafka-cluster event-streaming cluster-management kafka-brokers
Copyright (c) 2012-2016, Magnus Edenhill.librdkafka is a C library implementation of the Apache Kafka protocol, containing both Producer and Consumer support. It was designed with message delivery reliability and high performance in mind, current figures exceed 1 million msgs/second for the producer and 3 million msgs/second for the consumer.kafka kafka-consumer apache-kafka high-performance librdkafka kafka-producer c-plus-plus consumer kafka-client kafka-library
Native C# client for Apache Kafka.Copyright 2014, James Roland under Apache License, V2.0. See LICENSE file.kafka-client kafka-library kafka
Apache kafka is yet another precious gem from Apache Software Foundation. Kafka was originally developed at Linkedin and later on became a member of Apache project. Apache Kafka is a distributed publish-subscribe messaging system. Kafka differs from traditional messaging system as it is designed as distributed system, persists messages on disk and supports multiple subscribers. Kafka-Message-Server is an sample application for demonstrating kafka usage as message-server. Please follow the below instructions for productive use of the sample application.
Koperator (formerly called Banzai Cloud Kafka Operator) is an operator for managing Apache Kafka on Kubernetes that automates the provisioning, management, and autoscaling of Apache Kafka clusters deployed to K8s. We took a different approach to what's out there - we believe for a good reason - please read on to understand more about our design motivations and some of the scenarios which were driving us to create Koperator.kubernetes kafka cruisecontrol kafka-operator
We have large collection of open source products. Follow the tags from
Tag Cloud >>
Open source products are scattered around the web. Please provide information
about the open source projects you own / you use.