cernan - telemetry aggregation and shipping, last up the ladder

  •        25

Cernan is a telemetry and logging aggregation server. It exposes multiple interfaces for ingestion and can emit to multiple aggregation sources while doing in-flight manipulation of data. Cernan has minimal CPU and memory requirements and is intended to service bursty telemetry without load shedding. Cernan aims to be reliable and convenient to use, both for application engineers and operations staff. If you'd like to learn more, please do have a look in our wiki.

https://github.com/postmates/cernan

Tags
Implementation
License
Platform

   




Related Projects

graphios - A program to send nagios perf data to graphite (carbon) / statsd / librato / influxdb

  •    Python

Graphios is a script to emit nagios perfdata to various upstream metrics processing and time-series (graphing) systems. It's currently compatible with [graphite], [statsd], [Librato] and [InfluxDB], with possibly [Heka], and [RRDTool] support coming soon. Graphios can emit Nagios metrics to any number of supported upstream metrics systems simultaenously. Graphios is released under the GPL v2.

Snap - A powerful open telemetry framework

  •    Go

Snap is an open telemetry framework designed to simplify the collection, processing and publishing of system data through a single API. The goals of this project are to Empower systems to expose a consistent set of telemetry data, Simplify telemetry ingestion across ubiquitous storage systems, Provide powerful clustered control of telemetry workflows across small or large clusters and lot more.

statsite - C implementation of statsd

  •    C

Statsite is a metrics aggregation server. Statsite is based heavily on Etsy's StatsD https://github.com/etsy/statsd, and is wire compatible. Statsite is designed to be both highly performant, and very flexible. To achieve this, it implements the stats collection and aggregation in pure C, using an event loop to be extremely fast. This allows it to handle hundreds of connections, and millions of metrics. After each flush interval expires, statsite performs a fork/exec to start a new stream handler invoking a specified application. Statsite then streams the aggregated metrics over stdin to the application, which is free to handle the metrics as it sees fit.

opensoc - OpenSOC Apache Hadoop Code

  •    

OpenSOC integrates a variety of open source big data technologies in order to offer a centralized tool for security monitoring and analysis. OpenSOC provides capabilities for log aggregation, full packet capture indexing, storage, advanced behavioral analytics and data enrichment, while applying the most current threat intelligence information to security telemetry within a single platform. A mechanism to capture, store, and normalize any type of security telemetry at extremely high rates. Because security telemetry is constantly being generated, it requires a method for ingesting the data at high speeds and pushing it to various processing units for advanced computation and analytics.

Apache Skywalking - A distributed tracing system, and APM ( Application Performance Monitoring )

  •    Java

SkyWalking is an APM(application performance monitor) system, especially designed for microservices, cloud native and container-based (Docker, Kubernetes, Mesos) architectures. It provides distributed tracing, service mesh telemetry analysis, metric aggregation and visualization all-in-one solution.


telemetry - Dynamic dispatching library for metrics and instrumentations.

  •    Erlang

Telemetry is a dynamic dispatching library for metrics and instrumentations. It is lightweight, small and can be used in any Erlang or Elixir project. In a nutshell, you register a custom module and function to be invoked for certain events, which are executed whenever there is such event. Event name is a list of atoms. Each event is composed of a numeric value and can have metadata attached to it. Let's see an example.

Grafana - The leading graph and dashboard builder for visualizing time series metrics

  •    Go

Grafana is an open source, feature rich metrics dashboard and graph editor for Graphite, Elasticsearch, OpenTSDB, Prometheus and InfluxDB. It is most commonly used for visualizing time series data for Internet infrastructure and application analytics but many use it in other domains including industrial sensors, home automation, weather, and process control.

brubeck - A Statsd-compatible metrics aggregator

  •    C

Brubeck is a statsd-compatible stats aggregator written in C.Statsd is a metrics aggregator for Graphite (and other data storage backends). This technical documentation assumes working knowledge of what statsd is and how it works; please read the statsd documentation for more details.

pushgateway - push acceptor for ephemeral and batch jobs

  •    Go

The Prometheus Pushgateway exists to allow ephemeral and batch jobs to expose their metrics to Prometheus. Since these kinds of jobs may not exist long enough to be scraped, they can instead push their metrics to a Pushgateway. The Pushgateway then exposes these metrics to Prometheus.The Pushgateway is explicitly not an aggregator or distributed counter but rather a metrics cache. It does not have statsd-like semantics. The metrics pushed are exactly the same as you would present for scraping in a permanently running program.

statsd - Daemon for easy but powerful stats aggregation

  •    Javascript

A network daemon that runs on the Node.js platform and listens for statistics, like counters and timers, sent over UDP or TCP and sends aggregates to one or more pluggable backend services (e.g., Graphite). values Each stat will have a value. How it is interpreted depends on modifiers. In general values should be integers.

AppMetrics - App Metrics is an open-source and cross-platform

  •    CSharp

App Metrics is an open-source and cross-platform .NET library used to record metrics within an application. App Metrics can run on .NET Core or on the full .NET framework also supporting .NET 4.5.2. App Metrics abstracts away the underlaying repository of your Metrics for example InfluxDB, Graphite, Elasticsearch etc, by sampling and aggregating in memory and providing extensibility points to flush metrics to a repository at a specified interval.App Metrics provides various metric types to measure things such as the rate of requests, counting the number of user logins over time, measure the time taken to execute a database query, measure the amount of free memory and so on. Metrics types supported are Gauges, Counters, Meters, Histograms and Timers and Application Performance Indexes Apdex.

Disable-Nvidia-Telemetry - Windows utility to disable Nvidia's telemetry services

  •    CSharp

Disable Nvidia Telemetry is a utility that allows you to disable the telemetry services Nvidia bundles with their drivers.

carbon-c-relay - Enhanced C implementation of Carbon relay, aggregator and rewriter

  •    C

carbon-c-relay accepts, cleanses, matches, rewrites, forwards and aggregates graphite metrics by listening for incoming connections and relaying the messages to other servers defined in its configuration. The core functionality is to route messages via flexible rules to the desired destinations. carbon-c-relay is a simple program that reads its routing information from a file. The command line arguments allow to set the location for this file, as well as the amount of dispatchers (worker threads) to use for reading the data from incoming connections and passing them onto the right destination(s). The route file supports two main constructs: clusters and matches. The first define groups of hosts data metrics can be sent to, the latter define which metrics should be sent to which cluster. Aggregation rules are seen as matches.

loki - Like Prometheus, but for logs.

  •    Go

Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus. It is designed to be very cost effective and easy to operate, as it does not index the contents of the logs, but rather a set of labels for each log stream. Loki is like Prometheus, but for logs: we prefer a multidimensional label-based approach to indexing, and want a single-binary, easy to operate system with no dependencies. Loki differs from Prometheus by focussing on logs instead of metrics, and delivering logs via push, instead of pull.

browser-perf - Performance Metrics for Web Browsers

  •    Javascript

Read more on why browser-perf here. Please see the wiki pages for more information. You can find information about supported browsers, getting started, command line usage, reference for the Node API etc.

Stagemonitor - Application performance monitoring for java server applications

  •    Java

Stagemonitor is a Java monitoring agent that tightly integrates with time series databases like Elasticsearch, Graphite and InfluxDB to analyze graphed metrics and Kibana to analyze requests and call stacks. It includes preconfigured Grafana and Kibana dashboards that can be customized. It supports Real Time Metrics, Call tree of the current request including SQL statements to identify the bottlenecks of a request, Execution time and count of database interactions, Analyze the HTTP headers and lot more.

Apache Metron - Real-time Big Data Security

  •    Java

Metron integrates a variety of open source big data technologies in order to offer a centralized tool for security monitoring and analysis. Metron provides capabilities for log aggregation, full packet capture indexing, storage, advanced behavioral analytics and data enrichment, while applying the most current threat intelligence information to security telemetry within a single platform.

kube-state-metrics - Add-on agent to generate and expose cluster-level metrics.

  •    Go

kube-state-metrics is a simple service that listens to the Kubernetes API server and generates metrics about the state of the objects. (See examples in the Metrics section below.) It is not focused on the health of the individual Kubernetes components, but rather on the health of the various objects inside, such as deployments, nodes and pods.The metrics are exported through the Prometheus golang client on the HTTP endpoint /metrics on the listening port (default 8080). They are served either as plaintext or protobuf depending on the Accept header. They are designed to be consumed either by Prometheus itself or by a scraper that is compatible with scraping a Prometheus client endpoint. You can also open /metrics in a browser to see the raw metrics.

influxdb - Scalable datastore for metrics, events, and real-time analytics

  •    Go

InfluxDB is an open source time series database with no external dependencies. It's useful for recording metrics, events, and performing analytics. If you're feeling adventurous and want to contribute to InfluxDB, see our contributing doc for info on how to make feature requests, build from source, and run tests.

statsd-php - a PHP client for statsd

  •    PHP

A PHP client library for the statistics daemon (statsd) intended to send metrics from PHP applications. When establishing the connection to statsd and sending metrics, errors will be suppressed to prevent your application from crashing.