karafka-example-app - Karafka and WaterDrop example application

  •        560

This is an example application which uses Karafka framework to receive messages from Apache Kafka server and WaterDrop gem to send messages to Kafka server.To run this application locally you should run Apache Kafka on those hosts and ports which are set in app.rb file.

https://github.com/karafka/karafka-example-app

Tags
Implementation
License
Platform

   




Related Projects

Kafka-Message-Server - Example application based on Apache Kafka framework to show it usage as distributed message server

  •    Java

Apache kafka is yet another precious gem from Apache Software Foundation. Kafka was originally developed at Linkedin and later on became a member of Apache project. Apache Kafka is a distributed publish-subscribe messaging system. Kafka differs from traditional messaging system as it is designed as distributed system, persists messages on disk and supports multiple subscribers. Kafka-Message-Server is an sample application for demonstrating kafka usage as message-server. Please follow the below instructions for productive use of the sample application.

kafka-python - Python client for Apache Kafka

  •    Python

Python client for the Apache Kafka distributed stream processing system. kafka-python is designed to function much like the official java client, with a sprinkling of pythonic interfaces (e.g., consumer iterators).kafka-python is best used with newer brokers (0.9+), but is backwards-compatible with older versions (to 0.8.0). Some features will only be enabled on newer brokers. For example, fully coordinated consumer groups -- i.e., dynamic partition assignment to multiple consumers in the same group -- requires use of 0.9+ kafka brokers. Supporting this feature for earlier broker releases would require writing and maintaining custom leadership election and membership / health check code (perhaps using zookeeper or consul). For older brokers, you can achieve something similar by manually assigning different partitions to each consumer instance with config management tools like chef, ansible, etc. This approach will work fine, though it does not support rebalancing on failures. See <https://kafka-python.readthedocs.io/en/master/compatibility.html> for more details.

kafka-node - Node.js client for Apache Kafka 0.8 and later.

  •    Javascript

Kafka-node is a Node.js client with Zookeeper integration for Apache Kafka 0.8.1 and later. Follow the instructions on the Kafka wiki to build Kafka 0.8 and get a test broker up and running.

kafka-stack-docker-compose - docker compose files to create a fully working kafka stack

  •    Shell

This replicates as well as possible real deployment configurations, where you have your zookeeper servers and kafka servers actually all distinct from each other. This solves all the networking hurdles that comes with Docker and docker-compose, and is compatible cross platform. This configuration fits most development requirements.


kafka - Load-balancing, resuming Kafka consumer for go, backed by Zookeeper.

  •    Go

Kafka libraries, tools and example applications built on top of the sarama package.The following tools can be useful for discovery, testing, and benchmarking. They also serve as examples of how to use Sarama.

Redpanda - A Kafka API compatible streaming platform for mission-critical workloads

  •    C++

Redpanda is a streaming platform for mission critical workloads. It is Kafka API compatible so any client that works with Kafka should work out of the box with Redpanda. It does not have dependency on ZooKeeper, JVM. Redpanda is a new storage engine, optimized for streaming data, using a thread-per-core architecture focused on delivering stable tail-latencies. It transforms data inline with embedded WASM engine.

Kafdrop – Kafka Web UI

  •    Java

Kafdrop is a web UI for viewing Kafka topics and browsing consumer groups. The tool displays information such as brokers, topics, partitions, consumers, and lets you view messages.

docker-kafka - Kafka (and Zookeeper) in Docker

  •    Shell

This repository provides everything you need to run Kafka in Docker.For convenience also contains a packaged proxy that can be used to get data from a legacy Kafka 7 cluster into a dockerized Kafka 8.

DCMonitor - Data Center monitor, included zookeeper, kafka, druid

  •    Java

A simple, lightweight Data Center monitor, currently includes Zookeeper, Kafka, Druid(in progress). Motivated by KafkaOffsetMonitor, but faster and more stable. It is written in java, and use Prometheus as historical metrics storage.

kafka-monitor - Monitor the availability of Kakfa clusters with generated messages.

  •    Java

Kafka Monitor is a framework to implement and execute long-running kafka system tests in a real cluster. It complements Kafka’s existing system tests by capturing potential bugs or regressions that are only likely to occur after prolonged period of time or with low probability. Moreover, it allows you to monitor Kafka cluster using end-to-end pipelines to obtain a number of derived vital stats such as end-to-end latency, service availability and message loss rate. You can easily deploy Kafka Monitor to test and monitor your Kafka cluster without requiring any change to your application.Kafka Monitor can automatically create the monitor topic with the specified config and increase partition count of the monitor topic to ensure partition# >= broker#. It can also reassign partition and trigger preferred leader election to ensure that each broker acts as leader of at least one partition of the monitor topic. This allows Kafka Monitor to detect performance issue on every broker without requiring users to manually manage the partition assignment of the monitor topic.

kafka-websocket - Websocket server interface for Kafka distributed message broker

  •    Java

kafka-websocket is a simple websocket server interface to the kafka distributed message broker. It supports clients subscribing to topics, including multiple topics at once, and sending messages to topics. Messages may be either text or binary, the format for each is described below. A client may produce and consume messages on the same connection.

confluent-kafka-dotnet - Confluent's Apache Kafka .NET client

  •    CSharp

confluent-kafka-dotnet is Confluent's .NET client for Apache Kafka and the Confluent Platform.High performance - confluent-kafka-dotnet is a lightweight wrapper around librdkafka, a finely tuned C client.

confluent-kafka-python - Confluent's Apache Kafka Python client

  •    Python

confluent-kafka-python is Confluent's Python client for Apache Kafka and the Confluent Platform.High performance - confluent-kafka-python is a lightweight wrapper around librdkafka, a finely tuned C client.

kafka-rest - Confluent REST Proxy for Kafka

  •    Java

The Kafka REST Proxy provides a RESTful interface to a Kafka cluster, making it easy to produce and consume messages, view the state of the cluster, and perform administrative actions without using the native Kafka protocol or clients.

ruby-kafka - A Ruby client library for Apache Kafka

  •    Ruby

A Ruby client library for Apache Kafka, a distributed log and message bus. The focus of this library will be operational simplicity, with good logging and metrics that can make debugging issues easier.Although parts of this library work with Kafka 0.8 – specifically, the Producer API – it's being tested and developed against Kafka 0.9. The Consumer API is Kafka 0.9+ only.

kq - Kafka-based Job Queue for Python

  •    Python

KQ (Kafka Queue) is a lightweight Python library which lets you queue and execute jobs asynchronously using Apache Kafka. It uses kafka-python under the hood. You may need to use sudo depending on your environment.






We have large collection of open source products. Follow the tags from Tag Cloud >>


Open source products are scattered around the web. Please provide information about the open source projects you own / you use. Add Projects.