mtail - extract whitebox monitoring data from application logs for collection in a timeseries database

•    Go

---

mtail is a tool for extracting metrics from application logs to be exported into a timeseries database or timeseries calculator for alerting and dashboarding.It aims to fill a niche between applications that do not export their own internal state, and existing monitoring systems, without patching those applications or rewriting the same framework for custom extraction glue code.

monitoring logs observability prometheus collector proxy metrics extraction mtail calculator vm compiler bytecode

rtail - rtail(1) - Terminal output to the browser in seconds, using UNIX pipes.

•    CSS

---

rtail is a command line utility that grabs every line in stdin and broadcasts it over UDP. That's it. Nothing fancy. Nothing complicated. Tail log files, app output, or whatever you wish, using rtail broadcasting to an rtail-server – See multiple streams in the browser, in realtime.Whether you deploy your code on remote servers using multiple environments or simply have multiple projects, you must ssh to each machine running your code, in order to monitor the logs in realtime.

log logging logio logs tail udp

Log.io - Real-time log monitoring in your browser

•    CoffeeScript

---

Harvesters watch log files for changes, send new log messages to the server via TCP, which broadcasts to web clients via socket.io. Log streams are defined by mapping file paths to a stream name in harvester configuration.

logs monitoring realtime socket.io node.js ajax

vector - A high-performance, highly reliable, observability data pipeline.

•    Rust

---

Vector is a high-performance, end-to-end (agent & aggregator) observability data pipeline that puts you in control of your observability data. Collect, transform, and route all your logs, metrics, and traces to any vendors you want today and any other vendors you may want tomorrow. Vector enables dramatic cost reduction, novel data enrichment, and data security where you need it, not where is most convenient for your vendors. Additionally, it is open source and up to 10x faster than every alternative in the space. To get started, follow our quickstart guide or install Vector.

parser events router pipeline metrics vector logs stream-processing forwarder observability

Loki - Multi-tenant log aggregation system inspired by Prometheus

•    Go

---

Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus. It is designed to be very cost effective and easy to operate. It does not index the contents of the logs, but rather a set of labels for each log stream. It is especially good fit for storing Kubernetes Pod logs. Metadata such as Pod labels is automatically scraped and indexed. By storing compressed, unstructured logs and only indexing metadata, Loki is simpler to operate and cheaper to run.

grafana prometheus logging cloudnative log-collector log-aggregation logs

vector - A high-performance observability data pipeline.

•    Rust

---

Vector is a high-performance, end-to-end (agent & aggregator) observability data pipeline that puts you in control of your observability data. Collect, transform, and route all your logs, metrics, and traces to any vendors you want today and any other vendors you may want tomorrow. Vector enables dramatic cost reduction, novel data enrichment, and data security where you need it, not where it is most convenient for your vendors. Additionally, it is open source and up to 10x faster than every alternative in the space. To get started, follow our quickstart guide or install Vector.

parser events router pipeline metrics vector logs stream-processing forwarder observability

gollum - An n:m message multiplexer written in Go

•    Go

---

Gollum is an n:m multiplexer that gathers messages from different sources and broadcasts them to a set of destinations.Gollum originally started as a tool to MUL-tiplex LOG-files (read it backwards to get the name). It quickly evolved to a one-way router for all kinds of messages, not limited to just logs. Gollum is written in Go to make it scalable and easy to extend without the need to use a scripting language.

gollum logging logger stream log logs message-bus multiplexer pipeline messaging pub-sub message-queue

Meniscus - The Python Event Logging Service

•    Python

---

Meniscus is a Python based system for event collection, transit and processing in the large. It's primary use case is for large-scale Cloud logging, but can be used in many other scenarios including usage reporting and API tracing. Its components include Collection, Transport, Storage, Event Processing & Enhancement, Complex Event Processing, Analytics.

log-analysis event-processing events logs syslog

rita - Real Intelligence Threat Analytics

•    Go

---

Brought to you by Active Countermeasures. RITA is an open source framework for network traffic analysis.

rita network-traffic threat scanning offensive-countermeasures bro-ids blueteam security logs analytics analysis bhis beacon beacon-sniffer dns dns-tunneling dga

ekanite - The Syslog server with built-in search

•    Go

---

For detailed look at the goals, design, and implementation of this project, check out these blog posts. Ekanite is a high-performance syslog server with built-in text search. Its goal is to do a couple of things, and do them well -- accept log messages over the network, and make it easy to search the messages. What it lacks in feature, it makes up for in focus. Built in Go, it has no external dependencies, which makes deployment easy.

rsyslog syslog-server syslog-ng search logs syslog

kail - kubernetes log viewer

•    Go

---

Kubernetes tail. Streams logs from all containers of all matched pods. Match pods by service, replicaset, deployment, and others. Adjusts to a changing cluster - pods are added and removed from logging as they fall in or out of the selection.With no arguments, kail matches all pods in the cluster. You can control the matching pods with arguments which select pods based on various criteria.

kubernetes devtools tail logging logs log-viewer

Performance Analysis of Logs (PAL) Tool

•    

---

Ever have a performance problem, but don't know what performance counters to collect or how to analyze them? The PAL (Performance Analysis of Logs) tool is a powerful tool that reads in a performance monitor counter log and analyzes it using known thresholds.

performance pal performance-analysis performance-testing logs analysis log

kinesalite - An implementation of Amazon's Kinesis built on LevelDB

•    Javascript

---

An implementation of Amazon's Kinesis, focussed on correctness and performance, and built on LevelDB (well, @rvagg's awesome LevelUP to be precise).The Kinesis equivalent of dynalite.

kinesis mock kinesilite kinesis-mock test event stream streaming processing aws logs logging amazon-kinesis

Apache log4cxx - Logging Services for C++.

•    C

---

Apache log4cxx provides logging services for C++.

logging logs log-service log-framework

saw - Fast, multi-purpose tool for AWS CloudWatch Logs

•    Go

---

Fast, multi-purpose tool for AWS CloudWatch Logs

aws cloudwatch-logs logs streaming streamer color json cli tail watch

SharePoint Log Reader

•    

---

SharePoint Log Reader is a utility to open and filter the SharePoint 2007/2010 log files typically found in 12hive\LOGS or 14hive\LOGS. It's very user friendly and allows opening multiple log files at once.

logs moss sharepoint sharepoint-2007 sharepoint-2010 uls wss

aws4 - Signs and prepares node.js (and browser) requests using AWS Signature Version 4

•    Javascript

---

A small utility to sign vanilla node.js http(s) request options using Amazon's AWS Signature Version 4.Can also be used in the browser.

amazon aws signature s3 ec2 autoscaling cloudformation elasticloadbalancing elb elasticbeanstalk cloudsearch dynamodb kinesis lambda glacier sqs sns iam sts ses swf storagegateway datapipeline directconnect redshift opsworks rds monitoring cloudtrail cloudfront codedeploy elasticache elasticmapreduce elastictranscoder emr cloudwatch mobileanalytics cognitoidentity cognitosync cognito containerservice ecs appstream keymanagementservice kms config cloudhsm route53 route53domains logs

earhorn - JavaScript Execution Logs

•    HTML

---

JavaScript execution logs. Earhorn instruments your JavaScript and shows you a detailed, reversible, line-by-line log of JavaScript execution, sort of like console.log's crazy uncle.

log logs execution instrumentation esprima ast editor

stream - Stream plugin for WordPress

•    PHP

---

Stream tracks logged-in user activity so you can monitor every change made on your WordPress site in beautifully organized detail. All activity is organized by context, action and IP address for easy filtering. Developers can extend Stream with custom connectors to log any kind of action.

wp stream activity logs track

Honeypot - Low interaction honeypot that displays real time attacks

•    Javascript

---

Low interaction honeypot application that displays real time attacks in the web-interface. Made just for fun and it is not production ready. Written in Node.js the application listens on 128 most common TCP ports and saves results to the MySQL Database for further analysis.

nodejs honeypot data-visualization ssh telnet security logs tail