kafka-storm-starter - Code examples that show to integrate Apache Kafka 0

•    Scala

---

Code examples that show how to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark 1.1+ while using Apache Avro as the data serialization format. Take a look at the Kafka Streams code examples at https://github.com/confluentinc/examples.

apache-kafka kafka apache-storm storm spark apache-spark integration avro apache-avro

winton-kafka-streams - A Python implementation of Apache Kafka Streams

•    Python

---

Implementation of Apache Kafka's Streams API in Python. Apache Kafka is an open-source stream processing platform developed by the Apache Software Foundation written in Scala and Java. Kafka has Streams API added for building stream processing applications using Apache Kafka. Applications built with Kafka's Streams API do not require any setup beyond the provision of a Kafka cluster.

kafka-streams streaming-api librdkafka

confluent-kafka-dotnet - Confluent's Apache Kafka .NET client

•    CSharp

---

confluent-kafka-dotnet is Confluent's .NET client for Apache Kafka and the Confluent Platform.High performance - confluent-kafka-dotnet is a lightweight wrapper around librdkafka, a finely tuned C client.

kafka nuget c-sharp confluent kafka-client kafka-clients kafka-library

kafka-rest - Confluent REST Proxy for Kafka

•    Java

---

The Kafka REST Proxy provides a RESTful interface to a Kafka cluster, making it easy to produce and consume messages, view the state of the cluster, and perform administrative actions without using the native Kafka protocol or clients.

confluent kafka-rest-proxy kafka schema-registry rest-api kafka-client kafka-library

kafka-streams-course - Learn Kafka Streams with several examples!

•    Java

---

Learn Kafka Streams with several examples!

kafka kafka-streams learning udemy

confluent-kafka-python - Confluent's Apache Kafka Python client

•    Python

---

confluent-kafka-python is Confluent's Python client for Apache Kafka and the Confluent Platform.High performance - confluent-kafka-python is a lightweight wrapper around librdkafka, a finely tuned C client.

confluent kafka-client librdkafka python-client kafka-protocol kafka-library kafka

confluent-kafka-go - Confluent's Apache Kafka Golang client

•    Go

---

confluent-kafka-go is Confluent's Golang client for Apache Kafka and the Confluent Platform.High performance - confluent-kafka-go is a lightweight wrapper around librdkafka, a finely tuned C client.

confluent golang-library golang-bindings librdkafka kafka-client consumer producer kafka-library kafka

faust - Python Stream Processing

•    Python

---

Faust is a stream processing library, porting the ideas from Kafka Streams to Python. It is used at Robinhood to build high performance distributed systems and real-time data pipelines that process billions of events every day.

kafka-streams kafka asyncio distributed-systems stream-processing

kafka-streams-examples - Demo applications and code examples for Apache Kafka's Streams API.

•    Java

---

Kafka Streams examples

Alpakka Kafka connector - Alpakka is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Akka

•    Scala

---

The Alpakka project is an open source initiative to implement stream-aware, reactive, integration pipelines for Java and Scala. It is built on top of Akka Streams, and has been designed from the ground up to understand streaming natively and provide a DSL for reactive and stream-oriented programming, with built-in support for backpressure. Akka Streams is a Reactive Streams and JDK 9+ java.util.concurrent.Flow-compliant implementation and therefore fully interoperable with other implementations.

kafka reactive-streams akka akka-streams reactive kafka-client kafka-connector

CSharpClient-for-Kafka -

•    CSharp

---

This library was created for Kafka 0.8 with an intention to have a native library built from scratch. With Kafka protocol getting updated frequently with new features (which is expected until it reaches version 1.0), it doesn't seem beneficial to maintain a library built from scratch. The right approach (and as suggested by Confluent) for now would be to use a C# wrapper around the librdkafka C-Library, which the confluent-kafka-dotnet client is doing.So, if you are using Kafka 0.9 or higher, please move to using the confluent-kafka-dotnet client library.

goka - Goka is a compact yet powerful distributed stream processing library for Apache Kafka written in Go

•    Go

---

Goka is a compact yet powerful distributed stream processing library for Apache Kafka written in Go. Goka aims to reduce the complexity of building highly scalable and highly available microservices. Goka extends the concept of Kafka consumer groups by binding a state table to them and persisting them in Kafka. Goka provides sane defaults and a pluggable architecture.

goka stream processing kafka microservices

kafka-python - Python client for Apache Kafka

•    Python

---

Python client for the Apache Kafka distributed stream processing system. kafka-python is designed to function much like the official java client, with a sprinkling of pythonic interfaces (e.g., consumer iterators).kafka-python is best used with newer brokers (0.9+), but is backwards-compatible with older versions (to 0.8.0). Some features will only be enabled on newer brokers. For example, fully coordinated consumer groups -- i.e., dynamic partition assignment to multiple consumers in the same group -- requires use of 0.9+ kafka brokers. Supporting this feature for earlier broker releases would require writing and maintaining custom leadership election and membership / health check code (perhaps using zookeeper or consul). For older brokers, you can achieve something similar by manually assigning different partitions to each consumer instance with config management tools like chef, ansible, etc. This approach will work fine, though it does not support rebalancing on failures. See <https://kafka-python.readthedocs.io/en/master/compatibility.html> for more details.

kafka-topics-ui - Web Tool for Kafka Topics using Kafka Rest |

•    Javascript

---

Browse Kafka topics and understand what's happening on your cluster. Find topics / view topic metadata / browse topic data (kafka messages) / view topic configuration / download data. This is a web tool for the confluentinc/kafka-rest proxy. Config: If you don't use our docker image, keep in mind that Kafka-REST-Proxy CORS support can be a bit buggy, so if you have trouble setting it up, you may need to provide CORS headers through a proxy (i.e. nginx).

schema registry kafka topics avro

strimzi-kafka-operator - Apache Kafka running on Kubernetes and OpenShift

•    Java

---

Strimzi provides a way to run an Apache Kafka cluster on Kubernetes or OpenShift in various deployment configurations. See our website for more details about the project. Documentation to the current master branch as well as all releases can be found on our website.

kafka kubernetes openshift docker messaging enmasse kafka-connect kafka-streams data-streaming data-stream data-streams kubernetes-operator kubernetes-controller

Kafka UI - Open-Source Web UI for Apache Kafka Management

•    TypeScript

---

Kafka UI is a UI for Apache Kafka to monitor and manage Apache Kafka clusters. It is a simple tool that makes your data flows observable, helps find and troubleshoot issues faster and deliver optimal performance. Its lightweight dashboard makes it easy to track key metrics of your Kafka clusters - Brokers, Topics, Partitions, Production, and Consumption.

kafka big-data web-ui streams kafka-connect apache-kafka-ui kafka-producer kafka-client kafka-streams hacktoberfest streaming-data kafka-manager kafka-cluster event-streaming cluster-management kafka-brokers

Debezium - Stream changes from your databases.

•    Java

---

Debezium is a distributed platform that turns your existing databases into event streams, so applications can see and respond immediately to each row-level change in the databases. Debezium is built on top of Apache Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, from where your application consumes them. This makes it possible for your application to easily consume all of the events correctly and completely.

change-data-capture kafka-connect apache-kafka debezium cdc database kafka kafka-producer database-migration events

stream-reactor - Streaming reference architecture for ETL with Kafka and Kafka-Connect

•    Scala

---

Lenses offers SQL (for data browsing and Kafka Streams), Kafka Connect connector management, cluster monitoring and more. A collection of components to build a real time ingestion pipeline.

kafka kafka-connect connector streaming cassandra hazelcast redis elasticsearch ftp influxdb coap mqtt kudu jms hbase mongodb rethinkdb documentdb cosmosdb kubernetes

Memgraph - Build modern, graph-based applications on top of your streaming data in minutes

•    C++

---

Memgraph is a streaming graph application platform that helps you wrangle your streaming data, build sophisticated models that you can query in real-time, and develop graph applications.

kafka graph graph-algorithms nosql stream-processing graph-database kafka-streams cypher graph-analysis streaming-data opencypher

kafka-connect-jdbc - Kafka Connect connector for JDBC-compatible databases

•    Java

---

A Kafka Connect JDBC connector for copying data between databases and Kafka.

confluent kafka kafka-connector jdbc streaming