Displaying 1 to 20 from 33 results

librdkafka - The Apache Kafka C/C++ library

  •    C

Copyright (c) 2012-2016, Magnus Edenhill.librdkafka is a C library implementation of the Apache Kafka protocol, containing both Producer and Consumer support. It was designed with message delivery reliability and high performance in mind, current figures exceed 1 million msgs/second for the producer and 3 million msgs/second for the consumer.

kafka-storm-starter - Code examples that show to integrate Apache Kafka 0

  •    Scala

Code examples that show how to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark 1.1+ while using Apache Avro as the data serialization format. Take a look at the Kafka Streams code examples at https://github.com/confluentinc/examples.

pykafka - Apache Kafka client for Python; high-level & low-level consumer/producer, with great performance

  •    Python

PyKafka is a cluster-aware Kafka>=0.8.2 client for Python. It includes Python implementations of Kafka producers and consumers, which are optionally backed by a C extension built on librdkafka, and runs under Python 2.7+, Python 3.4+, and PyPy.PyKafka's primary goal is to provide a similar level of abstraction to the JVM Kafka client using idioms familiar to Python programmers and exposing the most Pythonic API possible.

Debezium - Stream changes from your databases.

  •    Java

Debezium is a distributed platform that turns your existing databases into event streams, so applications can see and respond immediately to each row-level change in the databases. Debezium is built on top of Apache Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, from where your application consumes them. This makes it possible for your application to easily consume all of the events correctly and completely.

kt - Kafka command line tool

  •    Go

You can download kt via the Releases section.

wirbelsturm - Wirbelsturm is a Vagrant and Puppet based tool to perform 1-click local and remote deployments, with a focus on big data related infrastructure

  •    Shell

Wirbelsturm is a Vagrant and Puppet based tool to perform 1-click local and remote deployments, with a focus on big data related infrastructure. Wirbelsturm's goal is to make tasks such as "I want to deploy a multi-node Storm cluster" simple, easy, and fun.

kq - Kafka-based Job Queue for Python

  •    Python

KQ (Kafka Queue) is a lightweight Python library which lets you queue and execute jobs asynchronously using Apache Kafka. It uses kafka-python under the hood. You may need to use sudo depending on your environment.

Agile_Data_Code_2 - Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition

  •    Jupyter

Like my work? I am Principal Consultant at Data Syndrome, a consultancy offering assistance and training with building full-stack analytics products, applications and systems. Find us on the web at datasyndrome.com. There is now a video course using code from chapter 8, Realtime Predictive Analytics with Kafka, PySpark, Spark MLlib and Spark Streaming. Check it out now at datasyndrome.com/video.

waterdrop - Ruby-Kafka based aspect oriented library for Kafka messages generating

  •    Ruby

Gem used to send messages to Kafka in an easy way.message that you want to send should be either binary or stringified (to_s, to_json, etc).

kafka-connect-hdfs - Kafka Connect HDFS connector

  •    Java

kafka-connect-hdfs is a Kafka Connector for copying data between Kafka and Hadoop HDFS.Documentation for this connector can be found here.

akka-stream-eventsourcing - Event sourcing for Akka Streams

  •    Scala

Methods emit and respond themselves are side-effect free. They only generate Emission values that are interpreted by the EventSourcing stage.

awesome-kafka - A list about Apache Kafka


This list is for anyone wishing to learn about Apache Kafka, but do not have a starting point. You can help by sending Pull Requests to add more information.

cppkafka - Modern C++ Apache Kafka client library (wrapper for librdkafka)

  •    C++

cppkafka allows C++ applications to consume and produce messages using the Apache Kafka protocol. The library is built on top of librdkafka, and provides a high level API that uses modern C++ features to make it easier to write code while keeping the wrapper's performance overhead to a minimum. cppkafka is a high level C++ wrapper for rdkafka, aiming to allow using rdkafka in a simple, less error prone way.

docker-kafka-zookeeper - Docker image for Kafka (0

  •    Shell

Docker image for Kafka (0.10.x - 0.11.x - 1.0.x) message broker including Zookeeper


  •    Scala

This repository contains code examples for Kafka Streams API.

docker-kafka-alpine - Alpine Linux based Kafka Docker Image

  •    Shell

This will create a single-node kafka broker (listening on localhost:9092), a local zookeeper instance and create the topic test-topic with 1 replication-factor and 1 partition.

We have large collection of open source products. Follow the tags from Tag Cloud >>

Open source products are scattered around the web. Please provide information about the open source projects you own / you use. Add Projects.