kafka-connect-storage-common - Shared software among connectors that target distributed filesystems and cloud storage

  •        60

Shared software modules among Kafka Connectors that target distributed filesystems and cloud storage.To build a development version you'll need a recent version of Kafka. You can build kafka-connect-storage-common with Maven using the standard lifecycle phases.

https://github.com/confluentinc/kafka-connect-storage-common

Tags
Implementation
License
Platform

   




Related Projects

kafka-connect-jdbc - Kafka Connect connector for JDBC-compatible databases

  •    Java

A Kafka Connect JDBC connector for copying data between databases and Kafka.

confluent-kafka-dotnet - Confluent's Apache Kafka .NET client

  •    CSharp

confluent-kafka-dotnet is Confluent's .NET client for Apache Kafka and the Confluent Platform.High performance - confluent-kafka-dotnet is a lightweight wrapper around librdkafka, a finely tuned C client.

confluent-kafka-python - Confluent's Apache Kafka Python client

  •    Python

confluent-kafka-python is Confluent's Python client for Apache Kafka and the Confluent Platform.High performance - confluent-kafka-python is a lightweight wrapper around librdkafka, a finely tuned C client.

confluent-kafka-go - Confluent's Apache Kafka Golang client

  •    Go

confluent-kafka-go is Confluent's Golang client for Apache Kafka and the Confluent Platform.High performance - confluent-kafka-go is a lightweight wrapper around librdkafka, a finely tuned C client.

stream-reactor - Streaming reference architecture for ETL with Kafka and Kafka-Connect

  •    Scala

Lenses offers SQL (for data browsing and Kafka Streams), Kafka Connect connector management, cluster monitoring and more. A collection of components to build a real time ingestion pipeline.


kafka-rest - Confluent REST Proxy for Kafka

  •    Java

The Kafka REST Proxy provides a RESTful interface to a Kafka cluster, making it easy to produce and consume messages, view the state of the cluster, and perform administrative actions without using the native Kafka protocol or clients.

CSharpClient-for-Kafka -

  •    CSharp

This library was created for Kafka 0.8 with an intention to have a native library built from scratch. With Kafka protocol getting updated frequently with new features (which is expected until it reaches version 1.0), it doesn't seem beneficial to maintain a library built from scratch. The right approach (and as suggested by Confluent) for now would be to use a C# wrapper around the librdkafka C-Library, which the confluent-kafka-dotnet client is doing.So, if you are using Kafka 0.9 or higher, please move to using the confluent-kafka-dotnet client library.

Debezium - Stream changes from your databases.

  •    Java

Debezium is a distributed platform that turns your existing databases into event streams, so applications can see and respond immediately to each row-level change in the databases. Debezium is built on top of Apache Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, from where your application consumes them. This makes it possible for your application to easily consume all of the events correctly and completely.

kafka-topics-ui - Web Tool for Kafka Topics using Kafka Rest |

  •    Javascript

Browse Kafka topics and understand what's happening on your cluster. Find topics / view topic metadata / browse topic data (kafka messages) / view topic configuration / download data. This is a web tool for the confluentinc/kafka-rest proxy. Config: If you don't use our docker image, keep in mind that Kafka-REST-Proxy CORS support can be a bit buggy, so if you have trouble setting it up, you may need to provide CORS headers through a proxy (i.e. nginx).

kafka-storm-starter - Code examples that show to integrate Apache Kafka 0

  •    Scala

Code examples that show how to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark 1.1+ while using Apache Avro as the data serialization format. Take a look at the Kafka Streams code examples at https://github.com/confluentinc/examples.

fast-data-dev - Kafka Docker for development

  •    Shell

Apache Kafka docker image for developers; with Landoop Lenses (landoop/kafka-lenses-dev) or Landoop's open source UI tools (landoop/fast-data-dev). Have a full fledged Kafka installation up and running in seconds and top it off with a modern streaming platform (only for kafka-lenses-dev), intuitive UIs and extra goodies. Also includes Kafka Connect, Schema Registry, Landoop Stream Reactor 25+ Connectors and more.

strimzi-kafka-operator - Apache Kafka running on Kubernetes and OpenShift

  •    Java

Strimzi provides a way to run an Apache Kafka cluster on Kubernetes or OpenShift in various deployment configurations. See our website for more details about the project. Documentation to the current master branch as well as all releases can be found on our website.

kafka-go - Kafka library in Go

  •    Go

sarama, which is by far the most popular but is quite difficult to work with. It is poorly documented, the API exposes low level concepts of the Kafka protocol, and it doesn't support recent Go features like contexts. It also passes all values as pointers which causes large numbers of dynamic memory allocations, more frequent garbage collections, and higher memory usage. confluent-kafka-go is a cgo based wrapper around librdkafka, which means it introduces a dependency to a C library on all Go code that uses the package. It has much better documentation than sarama but still lacks support for Go contexts.

schema-registry - Schema registry for Kafka

  •    Java

Schema Registry provides a RESTful interface for storing and retrieving versioned Avro schemas for use with Kafka.

ruby-kafka - A Ruby client library for Apache Kafka

  •    Ruby

A Ruby client library for Apache Kafka, a distributed log and message bus. The focus of this library will be operational simplicity, with good logging and metrics that can make debugging issues easier.Although parts of this library work with Kafka 0.8 – specifically, the Producer API – it's being tested and developed against Kafka 0.9. The Consumer API is Kafka 0.9+ only.

kq - Kafka-based Job Queue for Python

  •    Python

KQ (Kafka Queue) is a lightweight Python library which lets you queue and execute jobs asynchronously using Apache Kafka. It uses kafka-python under the hood. You may need to use sudo depending on your environment.

pykafka - Apache Kafka client for Python; high-level & low-level consumer/producer, with great performance

  •    Python

PyKafka is a cluster-aware Kafka>=0.8.2 client for Python. It includes Python implementations of Kafka producers and consumers, which are optionally backed by a C extension built on librdkafka, and runs under Python 2.7+, Python 3.4+, and PyPy.PyKafka's primary goal is to provide a similar level of abstraction to the JVM Kafka client using idioms familiar to Python programmers and exposing the most Pythonic API possible.

librdkafka - The Apache Kafka C/C++ library

  •    C

Copyright (c) 2012-2016, Magnus Edenhill.librdkafka is a C library implementation of the Apache Kafka protocol, containing both Producer and Consumer support. It was designed with message delivery reliability and high performance in mind, current figures exceed 1 million msgs/second for the producer and 3 million msgs/second for the consumer.