prometheus-kafka-adapter - Use Kafka as a remote storage database for Prometheus (remote write only)

  •        1851

Prometheus-kafka-adapter is a service which receives Prometheus metrics through remote_write, marshal into JSON and sends them into Kafka. We use prometheus-kafka-adapter internally at Telefonica for dumping Prometheus metrics into an object storage in diferent clouds, through Kafka and Kafka-Connect.

https://github.com/Telefonica/prometheus-kafka-adapter

Tags
Implementation
License
Platform

   




Related Projects

kafka_exporter - Kafka exporter for Prometheus

  •    Go

Kafka exporter for Prometheus. For other metrics from Kafka, have a look at the JMX exporter. Support Apache Kafka version 0.10.1.0 (and later).

Redpanda - A Kafka API compatible streaming platform for mission-critical workloads

  •    C++

Redpanda is a streaming platform for mission critical workloads. It is Kafka API compatible so any client that works with Kafka should work out of the box with Redpanda. It does not have dependency on ZooKeeper, JVM. Redpanda is a new storage engine, optimized for streaming data, using a thread-per-core architecture focused on delivering stable tail-latencies. It transforms data inline with embedded WASM engine.

Signoz - Open-source Observability platform and an alternative to DataDog, NewRelic

  •    Javascript

SigNoz is an opensource observability platform. SigNoz uses distributed tracing to gain visibility into your systems and powers data using Kafka (to handle high ingestion rate and backpressure) and Apache Druid (Apache Druid is a high performance real-time analytics database), both proven in the industry to handle scale.

librdkafka - The Apache Kafka C/C++ library

  •    C

Copyright (c) 2012-2016, Magnus Edenhill.librdkafka is a C library implementation of the Apache Kafka protocol, containing both Producer and Consumer support. It was designed with message delivery reliability and high performance in mind, current figures exceed 1 million msgs/second for the producer and 3 million msgs/second for the consumer.

kq - Kafka-based Job Queue for Python

  •    Python

KQ (Kafka Queue) is a lightweight Python library which lets you queue and execute jobs asynchronously using Apache Kafka. It uses kafka-python under the hood. You may need to use sudo depending on your environment.


kafkacat - Generic command line non-JVM Apache Kafka producer and consumer

  •    C

kafkacat is a generic non-JVM producer and consumer for Apache Kafka >=0.8, think of it as a netcat for Kafka.In producer mode kafkacat reads messages from stdin, delimited with a configurable delimiter (-D, defaults to newline), and produces them to the provided Kafka cluster (-b), topic (-t) and partition (-p).

prometheus-adapter - An implementation of the custom.metrics.k8s.io API using Prometheus

  •    Go

This repository contains an implementation of the Kubernetes resource metrics, custom metrics, and external metrics APIs. This adapter is therefore suitable for use with the autoscaling/v2 Horizontal Pod Autoscaler in Kubernetes 1.6+. It can also replace the metrics server on clusters that already run Prometheus and collect the appropriate metrics.

DCMonitor - Data Center monitor, included zookeeper, kafka, druid

  •    Java

A simple, lightweight Data Center monitor, currently includes Zookeeper, Kafka, Druid(in progress). Motivated by KafkaOffsetMonitor, but faster and more stable. It is written in java, and use Prometheus as historical metrics storage.

ruby-kafka - A Ruby client library for Apache Kafka

  •    Ruby

A Ruby client library for Apache Kafka, a distributed log and message bus. The focus of this library will be operational simplicity, with good logging and metrics that can make debugging issues easier.Although parts of this library work with Kafka 0.8 – specifically, the Producer API – it's being tested and developed against Kafka 0.9. The Consumer API is Kafka 0.9+ only.

pykafka - Apache Kafka client for Python; high-level & low-level consumer/producer, with great performance

  •    Python

PyKafka is a cluster-aware Kafka>=0.8.2 client for Python. It includes Python implementations of Kafka producers and consumers, which are optionally backed by a C extension built on librdkafka, and runs under Python 2.7+, Python 3.4+, and PyPy.PyKafka's primary goal is to provide a similar level of abstraction to the JVM Kafka client using idioms familiar to Python programmers and exposing the most Pythonic API possible.

confluent-kafka-go - Confluent's Apache Kafka Golang client

  •    Go

confluent-kafka-go is Confluent's Golang client for Apache Kafka and the Confluent Platform.High performance - confluent-kafka-go is a lightweight wrapper around librdkafka, a finely tuned C client.

spring-integration-kafka

  •    Java

The Spring Integration Kafka extension project provides inbound and outbound channel adapters for Apache Kafka. Apache Kafka is a distributed publish-subscribe messaging system that is designed for high throughput (terabytes of data) and low latency (milliseconds). For more information on Kafka and its design goals, see the Kafka main page.Starting from version 2.0 version this project is a complete rewrite based on the new spring-kafka project which uses the pure java Producer and Consumer clients provided by Kafka 0.9.x.x and 0.10.x.x..

kafka-node - Node.js client for Apache Kafka 0.8 and later.

  •    Javascript

Kafka-node is a Node.js client with Zookeeper integration for Apache Kafka 0.8.1 and later. Follow the instructions on the Kafka wiki to build Kafka 0.8 and get a test broker up and running.

Debezium - Stream changes from your databases.

  •    Java

Debezium is a distributed platform that turns your existing databases into event streams, so applications can see and respond immediately to each row-level change in the databases. Debezium is built on top of Apache Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, from where your application consumes them. This makes it possible for your application to easily consume all of the events correctly and completely.

Kafdrop – Kafka Web UI

  •    Java

Kafdrop is a web UI for viewing Kafka topics and browsing consumer groups. The tool displays information such as brokers, topics, partitions, consumers, and lets you view messages.

php-rdkafka - Kafka client for PHP

  •    C

PHP-rdkafka is a thin librdkafka binding providing a working PHP 5 / PHP 7 Kafka 0.8 / 0.9 / 0.10 client.It supports the high level and low level consumers, producer, and metadata APIs.

kafkajs - A modern Apache Kafka client for node.js

  •    Javascript

KafkaJS is a modern Apache Kafka client for Node.js. It is compatible with Kafka 0.10+ and offers native support for 0.11 features. It has support for Producer, Consumer groups with pause, resume, and seek, GZIP compression, Admin client.

tempo - Grafana Tempo is a high volume, minimal dependency distributed tracing backend.

  •    Go

Grafana Tempo is an open source, easy-to-use and high-scale distributed tracing backend. Tempo is cost-efficient, requiring only object storage to operate, and is deeply integrated with Grafana, Prometheus, and Loki. Tempo can be used with any of the open source tracing protocols, including Jaeger, Zipkin, OpenCensus, Kafka, and OpenTelemetry. It supports key/value lookup only and is designed to work in concert with logs and metrics (exemplars) for discovery. Check out the Integration Guides to see examples of OpenTelemetry instrumentation with Tempo.

shell-operator - Shell-operator is a tool for running event-driven scripts in a Kubernetes cluster

  •    Go

Shell-operator is a tool for running event-driven scripts in a Kubernetes cluster. This operator is not an operator for a particular software product such as prometheus-operator or kafka-operator. Shell-operator provides an integration layer between Kubernetes cluster events and shell scripts by treating scripts as hooks triggered by events. Think of it as an operator-sdk but for scripts.






We have large collection of open source products. Follow the tags from Tag Cloud >>


Open source products are scattered around the web. Please provide information about the open source projects you own / you use. Add Projects.