Sarama - Go library for Apache Kafka 0.8, 0.9, and 0.10.

  •        87

Package sarama is a pure Go client library for dealing with Apache Kafka (versions 0.8 and later). It includes a high-level API for easily producing and consuming messages, and a low-level API for controlling bytes on the wire when the high-level API is insufficient.



Related Projects

ruby-kafka - A Ruby client library for Apache Kafka

A Ruby client library for Apache Kafka, a distributed log and message bus. The focus of this library will be operational simplicity, with good logging and metrics that can make debugging issues easier.Although parts of this library work with Kafka 0.8 – specifically, the Producer API – it's being tested and developed against Kafka 0.9. The Consumer API is Kafka 0.9+ only.

kafka - Load-balancing, resuming Kafka consumer for go, backed by Zookeeper.

Kafka libraries, tools and example applications built on top of the sarama package.The following tools can be useful for discovery, testing, and benchmarking. They also serve as examples of how to use Sarama.

kafka-rust - Rust client for Apache Kafka

To build kafka-rust the usual cargo build should suffice. The crate supports various features which can be turned off at compile time. See kafka-rust's Cargo.toml and cargo's documentation.kafka-rust is tested against Kafka 0.8.2.x and regularly used against Kafka 0.9 servers. However, efforts to implement support for new features from the Kafka 0.9 release are just in their beginnings.

confluent-kafka-go - Confluent's Apache Kafka Golang client

confluent-kafka-go is Confluent's Golang client for Apache Kafka and the Confluent Platform.High performance - confluent-kafka-go is a lightweight wrapper around librdkafka, a finely tuned C client.

confluent-kafka-python - Confluent's Apache Kafka Python client

confluent-kafka-python is Confluent's Python client for Apache Kafka and the Confluent Platform.High performance - confluent-kafka-python is a lightweight wrapper around librdkafka, a finely tuned C client.

dory - Producer daemon for Apache Kafka

Dory decouples message sources from the Kafka cluster. A client is not forced to wait for an ACK after sending a message, since Dory handles the details of waiting for ACKs from Kafka and resending messages when necessary. Likewise, a client is not burdened with holding onto messages until it has a reasonable-sized batch to send to Kafka. If a client crashes immediately after sending a message to Dory, the message is safe with Dory. However, if the client assumes responsibility for interacting with Kafka, a crash will cause the loss of all batched messages, and possibly sent messages for which an ACK is pending.Dory provides uniformity of mechanism for status monitoring and data quality reporting through its web interface. Likewise, it provides a unified configuration mechanism for settings related to batching, compression, and other aspects of interaction with Kafka. This simplifies system administration, as compared to a multitude of producer mechanisms for various programming languages and applications, each with its own status monitoring, data quality reporting, and configuration mechanisms or lack thereof.

confluent-kafka-dotnet - Confluent's Apache Kafka .NET client

confluent-kafka-dotnet is Confluent's .NET client for Apache Kafka and the Confluent Platform.High performance - confluent-kafka-dotnet is a lightweight wrapper around librdkafka, a finely tuned C client.

kafka-pixy - gRPC/REST proxy for Kafka

Kafka-Pixy is a dual API (gRPC and REST) proxy for Kafka with automatic consumer group control. It is designed to hide the complexity of the Kafka client protocol and provide a stupid simple API that is trivial to implement in any language.Kafka-Pixy supports Kafka versions form 0.8.2.x to 0.10.1.x. It uses the Kafka Offset Commit/Fetch API to keep track of consumer offsets. However Group Membership API is not yet implemented, therefore it needs to talk to Zookeeper directly to manage consumer group membership.

pykafka - Apache Kafka client for Python; high-level & low-level consumer/producer, with great performance

PyKafka is a cluster-aware Kafka>=0.8.2 client for Python. It includes Python implementations of Kafka producers and consumers, which are optionally backed by a C extension built on librdkafka, and runs under Python 2.7+, Python 3.4+, and PyPy.PyKafka's primary goal is to provide a similar level of abstraction to the JVM Kafka client using idioms familiar to Python programmers and exposing the most Pythonic API possible.

CSharpClient-for-Kafka -

This library was created for Kafka 0.8 with an intention to have a native library built from scratch. With Kafka protocol getting updated frequently with new features (which is expected until it reaches version 1.0), it doesn't seem beneficial to maintain a library built from scratch. The right approach (and as suggested by Confluent) for now would be to use a C# wrapper around the librdkafka C-Library, which the confluent-kafka-dotnet client is doing.So, if you are using Kafka 0.9 or higher, please move to using the confluent-kafka-dotnet client library.

librdkafka - The Apache Kafka C/C++ library

Copyright (c) 2012-2016, Magnus Edenhill.librdkafka is a C library implementation of the Apache Kafka protocol, containing both Producer and Consumer support. It was designed with message delivery reliability and high performance in mind, current figures exceed 1 million msgs/second for the producer and 3 million msgs/second for the consumer.

kafka - Go driver for Kafka

Kafka is Go client library for Apache Kafka server, released under MIT license.Kafka provides minimal abstraction over wire protocol, support for transparent failover and easy to use blocking API.

kafka-php - PHP client library for Apache Kafka

kafka-php allows you to produce messages to the Apache Kafka distributed publish/subscribe messaging service.Add the lib directory to the PHP include_path and use an autoloader like the one in the examples directory (the code follows the PEAR/Zend one-class-per-file convention).

brod - Apache Kafka client library for Erlang/Elixir

Brod is an erlang implementation of the Apache Kafka protocol, providing support for both producers and consumers.Assuming kafka is running at localhost:9092 and there is a topic named brod-test.

kafka-rest - Confluent REST Proxy for Kafka

The Kafka REST Proxy provides a RESTful interface to a Kafka cluster, making it easy to produce and consume messages, view the state of the cluster, and perform administrative actions without using the native Kafka protocol or clients.

Krackle - A Low Overhead Kafka Client

While the standard Java Kafka client is easy to use, it does tend to have a high level of overhead. A lot of objects are created, only to be garbage collected very quickly, often within milliseconds on a heavily loaded producer.Krackle is a Kafka 0.8 client designed to minimize the number of objects created, and therefore to reduce garbage collection overhead. In my tests this has reduced the CPU usage by 50% under heavy load.

kafka-cpp - CPP client for Apache Kafka

This library allows you to produce messages to the Kafka distributed publish/subscribe messaging service.Run this to generate the makefile for your system. Do this first.