librdkafka - The Apache Kafka C/C++ library

•    C

---

Copyright (c) 2012-2016, Magnus Edenhill.librdkafka is a C library implementation of the Apache Kafka protocol, containing both Producer and Consumer support. It was designed with message delivery reliability and high performance in mind, current figures exceed 1 million msgs/second for the producer and 3 million msgs/second for the consumer.

kafka kafka-consumer apache-kafka high-performance librdkafka kafka-producer c-plus-plus consumer kafka-client kafka-library

Debezium - Stream changes from your databases.

•    Java

---

Debezium is a distributed platform that turns your existing databases into event streams, so applications can see and respond immediately to each row-level change in the databases. Debezium is built on top of Apache Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, from where your application consumes them. This makes it possible for your application to easily consume all of the events correctly and completely.

change-data-capture kafka-connect apache-kafka debezium cdc database kafka kafka-producer database-migration events

karafka - Framework used to simplify Apache Kafka based Ruby and Rails applications development.

•    Ruby

---

Framework used to simplify Apache Kafka based Ruby applications development.It allows programmers to use approach similar to standard HTTP conventions (params and params_batch) when working with asynchronous Kafka messages.

karafka-framework kafka-topic kafka kafka-client kafka-clients kafka-producer kafka-consumer apache-kafka kafka-message karafka-application sidekiq rails kafka-ruby ruby-on-rails rubygems rubygem ruby-library kafka-library

kq - Kafka-based Job Queue for Python

•    Python

---

KQ (Kafka Queue) is a lightweight Python library which lets you queue and execute jobs asynchronously using Apache Kafka. It uses kafka-python under the hood. You may need to use sudo depending on your environment.

kafka job-queue async kafka-client kafka-consumer kafka-producer python3 python2 python-library python-3 python-2 asynchronous queueing jobqueue apache-kafka worker-queue producer-consumer serialization python-3-5

delivery_boy - A simple way to publish messages to Kafka from Ruby applications

•    Ruby

---

In addition to improving response time, delivering messages asynchronously also protects your application against Kafka availability issues -- if messages cannot be delivered, they'll be buffered for later and retried automatically. Regarding partition and partition_key: if none are specified, DeliveryBoy will pick a partition at random. If you want to ensure that e.g. all messages related to a user always get written to the same partition, you can pass the user id to partition_key. Don't use partition directly unless you know what you're doing, since it requires you to know exactly how many partitions each topic has, which can change and cause you pain and misery. Just use partition_key or let DeliveryBoy choose at random.

kafka kafka-producer

rafka - Interact with Kafka in a simple way

•    Go

---

Rafka is a gateway service that exposes Kafka using simple semantics. It implements the Redis protocol.

kafka kafka-consumer kafka-producer librdkafka kafka-client

neo4j-kafka - Neo4j Kafka Integrations

•    Kotlin

---

Ideas based on a now removed repository from workday. You can set the following configuration values in your neo4j.conf, here are the defaults.

neo4j kafka kafka-producer cdc change-data-capture graph-database

spark-kafka-sink - A Kafka metric sink for Apache Spark

•    Scala

---

A Kafka metric sink for Apache Spark

apache-spark apache-kafka metric-sink metrics-gathering metrics kafka kafka-producer spark

reactive-interaction-gateway - Your UI deserves an API, too!

•    Elixir

---

In short: handling asynchronous events. Instead, let RIG handle those connections for you. RIG is designed for scalability and allows you to concentrate on the actual business logic. Back-end (micro)services no longer have to care about connection state, which means they can be stateless, making it very easy to roll out updates to them. Sending notifications to all online devices of a certain user becomes as easy as POSTing a message to an HTTP endpoint.

elixir elixir-phoenix elixir-apps api-gateway kafka-consumer kafka-producer server-sent-events unidirectional-data-flow reactive frontend polling architecture

strimzi-kafka-bridge - Apache Kafka bridge

•    Java

---

Apache Kafka bridge

amqp bridge kafka kafka-bridge http kafka-consumer kafka-producer kafka-rest-proxy kafka-client amqp-client amqp10

kafka-0.11-examples - Code snippets that demonstrate how to leverage the new Kafka 0.11 APIs

•    Scala

---

This repository contains code snippets in Scala, using the Java clients, that demonstrate how to use the Kafka 0.11 new APIs. The code is incomplete, and that's intended: the goal is to bring the focus on the new APIs only.

kafka kafka-client kafka-producer kafka-consumer

neo4j-streams - Neo4j Kafka Integrations

•    Kotlin

---

You can set the following configuration values in your neo4j.conf, here are the defaults. See the Apache Kafka documentation for details on these settings.

neo4j kafka kafka-producer cdc change-data-capture graph-database

kafkabridge - The Apache Kafka Client SDK

•    C++

---

Navigate to the cxx/c installation directory，and run build.sh -release, you will get a new file named libqbus.so in the "./lib/release" directory. Navigate to the Go installation directory，and run build.sh, you will get the new files qbus.go and libQBus_go.so in the directory "gopath/src/qbus".

kafka consumer producer kafkaclient kafkasdk cplusplus kafkabridge kafka-producer kafka-consumer

prometheus-kafka-adapter - Use Kafka as a remote storage database for Prometheus (remote write only)

•    Go

---

Prometheus-kafka-adapter is a service which receives Prometheus metrics through remote_write, marshal into JSON and sends them into Kafka. We use prometheus-kafka-adapter internally at Telefonica for dumping Prometheus metrics into an object storage in diferent clouds, through Kafka and Kafka-Connect.

prometheus kafka prometheus-adapter kafka-producer