kafkabeat - Kafka event forwarder build on top of Elastic Beats platform

  •        58

Kafkabeat is an Elastic Beat that read events from Kafka topics and forward them to any supported output. Kafkabeat is supporting two event processing modes via so-called codecs: plain and json.

https://github.com/arkady-emelyanov/kafkabeat

Tags
Implementation
License
Platform

   




Related Projects

beats - :tropical_fish: Beats - Lightweight shippers for Elasticsearch & Logstash

  •    Go

The Beats are lightweight data shippers, written in Go, that you install on your servers to capture all sorts of operational data (think of logs, metrics, or network packet data). The Beats send the operational data to Elasticsearch, either directly or via Logstash, so it can be visualized with Kibana.By "lightweight", we mean that Beats have a small installation footprint, use limited system resources, and have no runtime dependencies.

logstash - Logstash - transport and process your logs, events, or other data

  •    Ruby

Logstash is part of the Elastic Stack along with Beats, Elasticsearch and Kibana. Logstash is an open source, server-side data processing pipeline that ingests data from a multitude of sources simultaneously, transforms it, and then sends it to your favorite "stash." (Ours is Elasticsearch, naturally.). Logstash has over 200 plugins, and you can write your own very easily as well.The license is Apache 2.0, meaning you are pretty much free to use it however you want in whatever way.

hangout - 用java实现一下Logstash的几个常用input/filter/output, 希望能有效率上面的大提升

  •    Java

模仿logstash做的一个应用. 现在我们迁移到了 https://github.com/childe/gohangout , 这个项目基本上停止更新了. 我们一直用logstash从Kafka消费数据进ES, 随着数据量的越来越大, 需要的logstash实例和机器也越来越多.

elastiflow - Network flow Monitoring (Netflow, sFlow and IPFIX) with the Elastic Stack

  •    Shell

ElastiFlow™ provides network flow data collection and visualization using the Elastic Stack (Elasticsearch, Logstash and Kibana). It supports Netflow v5/v9, sFlow and IPFIX flow types (1.x versions support only Netflow v5/v9). The following dashboards are provided.

Search Guard - Elasticsearch plugin that offers encryption, authentication, and authorisation

  •    Java

Search Guard is an Elasticsearch plugin that offers encryption, authentication, and authorization. It builds on Search Guard SSL and provides pluggable authentication and authorization modules in addition. Search Guard is fully compatible with Kibana, Logstash and Beats.


puppet-logstash - Puppet module to manage Logstash

  •    Puppet

A Puppet module for managing and configuring Logstash.This module, "elastic/logstash" supports only Logstash 5.x. For earlier Logstash versions, support is provided by the legacy module, "elasticsearch/logstash".

HELK - The Incredible HELK

  •    Shell

A Hunting ELK (Elasticsearch, Logstash, Kibana) with advanced analytic capabilities.At the end of the HELK installation, you will have a similar output with the information you need to access the primary HELK components. Remember that the default username and password for the HELK are helk:hunting.

kibana2 - Kibana was acquired by Elastic in 2013. See elastic/kibana. More info at http://kibana.org

  •    Javascript

Kibana is a browser based interface for Logstash and ElasticSearch that allows you to efficiently search, visualize, analyze and otherwise make sense of your logs. To run Kibana with JRuby, e.g. if you have to run in on a windows machine, you can create a (executable) WAR archive.

docker-elk - The ELK stack powered by Docker and Compose.

  •    Dockerfile

Run the latest version of the Elastic stack with Docker and Docker Compose. It will give you the ability to analyze any data set by using the searching/aggregation capabilities of Elasticsearch and the visualization power of Kibana.

Flummi - Elastic Search HTTP REST Client

  •    Java

Flummi is a client library for Elastic Search. It has been successfully tested with Elastic Search versions 2.3, 2.4 and 5.1. It provides a comprehensive Java query DSL API and communicates with the Elastic Search Cluster via HTTP/JSON. Flummi uses HTTP and JSON for communication with Elastic Search. Its only dependencies are Gson and AsyncHttpClient, so it is good for you if you don't want to have your application depend on the full ElasticSearch JAR.

elastic - Elasticsearch client for Go.

  •    Go

Elastic is an Elasticsearch client for the Go programming language.See the wiki for additional information about Elastic.

elastic-builder - A Node.js implementation of the elasticsearch Query DSL :construction_worker:

  •    Javascript

A Node.js implementation of the Elasticsearch DSL for use with the official elasticsearch javascript client with builder syntax. Check out the API reference documentation.

elk-docker - Elasticsearch, Logstash, Kibana (ELK) Docker image

  •    Shell

This Docker image provides a convenient centralised log server and log management web interface, by packaging Elasticsearch, Logstash, and Kibana, collectively known as ELK. See the ELK Docker image documentation web page for complete instructions on how to use this image.

VulnWhisperer - Create actionable data from your Vulnerability Scans

  •    Python

VulnWhisperer is a vulnerability data and report aggregator. VulnWhisperer will pull all the reports and create a file with a unique filename which is then fed into logstash. Logstash extracts data from the filename and tags all of the information inside the report (see logstash_vulnwhisp.conf file). Data is then shipped to elasticsearch to be indexed. The following instructions should be utilized as a Sample Guide in the absence of an existing ELK Cluster/Node. This will cover a Debian example install guide of a stand-alone node of Elasticsearch & Kibana.

journalbeat - Journalbeat is a log shipper from systemd/journald to Logstash/Elasticsearch

  •    Go

Journalbeat is the Beat used for log shipping from systemd/journald based Linux systems. It follows the system journal very much like journalctl -f and sends the data to Logstash/Elasticsearch (or whatever you configured for your beat). Journalbeat is targeting pure systemd distributions like CoreOS, Atomic Host, or others. There are no intentions to add support for older systems that do not use journald.

elasticsearch-logstash-index-mgmt - Bash scripts for managing backup, delete, and restore of elasticsearch indexes created by logstash

  •    Shell

Please note that Elasticsearch provides the python based Curator which manages closing/deleting and maintenance with lots of tuning capabilities. It is worth investigating Curator as an elasticsearch-maintained solution for your cluster's time-based index maintenance needs. If you are using Curator with Elasticsearch >= 1.0.0 (and Hubot) and you want a way to restore old indices, try hubot-elk-restore.

ElasticHD - Elasticsearch 可视化DashBoard, 支持Es监控、实时搜索,Index template快捷替换修改,索引列表信息查看, SQL converts to DSL等

  •    Go

Precompiled binaries for supported operating systems are available.ElasticHD does not require any software. It works in your web browser, allowing you to manage and monitor your ElasticSearch clusters from anywhere at any time. Built on responsive CSS design, ElasticHD adjusts itself to any screen size on any device.

Gimel - PayPal's Big Data Processing Framework

  •    Scala

Gimel provides unified Data API to access data from any storage like HDFS, GS, Alluxio, Hbase, Aerospike, BigQuery, Druid, Elastic, Teradata, Oracle, MySQL, etc.