nxlog - Multi platform Log management

•    C

---

nxlog is a modular, multi-threaded, high-performance log management solution with multi-platform support. In concept it is similar to syslog-ng or rsyslog but is not limited to unix/syslog only. It can collect logs from files in various formats, receive logs from the network remotely over UDP, TCP or TLS/SSL . It supports platform specific sources such as the Windows Eventlog, Linux kernel logs, Android logs, local syslog etc.

log syslog log-management log-analysis log-server

Graylog2 - Open Source Log Management

•    Java

---

Graylog2 is an open source log management solution that stores your logs in ElasticSearch. It consists of a server written in Java that accepts your syslog messages via TCP, UDP or AMQP and stores it in the database. The second part is a web interface that allows you to manage the log messages from your web browser. Take a look at the screenshots or the latest release info page to get a feeling of what you can do with Graylog2.

log log-analysis log-search log-management syslog

X-Itools: Enterprise Collaboration

•    Javascript

---

Enterprise Collaboration modules and strong Log Analysis modules

Logsandra - log management using Cassandra

•    Python

---

Logsandra is a log management application written in Python and using Cassandra as back-end. It is written as demo for cassandra but it is worth to take a look. It provides support to create your own parser.

log log-analysis log-management

GoAccess - Real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems or through your browser

•    C

---

GoAccess is an open source real-time web log analyzer and interactive viewer that runs in a terminal on *nix systems or through your browser. It provides fast and valuable HTTP statistics for system administrators that require a visual server report on the fly. It supports nearly all web log formats (Apache, Nginx, Amazon S3, Elastic Load Balancing, CloudFront, etc)

data-analysis analytics nginx apache webserver web-analytics monitoring dashboard command-line log-management log-analysis log-search

Flume - Log management using HDFS

•    Java

---

Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. It has a simple and flexible architecture based on streaming data flows. It is robust and fault tolerant with tunable reliability mechanisms and many failover and recovery mechanisms. It uses a simple extensible data model that allows for online analytic application.

log-collector syslog log-analysis log log-management distributed

Chainsaw - log viewer and analysis tool

•    Java

---

Chainsaw is a companion application to Log4j written by members of the Log4j development community. Chainsaw can read log files formatted in Log4j's XMLLayout, receive events from remote locations, read events from a DB, it can even work with the JDK 1.4 logging events.

log log-analysis log-management

Clarity - Web interface for the grep

•    Ruby

---

Clarity is a Splunk like web interface for your server log files. It supports searching (using grep) as well as trailing log files in realtime. It has been written using the event based architecture based on EventMachine and so allows real-time search of very large log files.

log log-analysis log-management log-search

Fluentd - Unified Logging Layer

•    Ruby

---

Fluentd collects events from various data sources and writes them to files, RDBMS, NoSQL, IaaS, SaaS, Hadoop and so on. Fluentd helps you unify your logging infrastructure. Fluentd can be used to tail access/error logs and transport them reliably to remote systems. It can "grep" for events and send out alerts. It can function as middleware to enable asynchronous, scalable logging for user action events.

log-collector syslog log-analysis log log-management log-aggregator log-search data-collector

rrdtool-1.x - RRDtool 1.x - Round Robin Database

•    C

---

RRDtool is a little program for easily maintaining a database of time-series data. It comes with a charting program for drawing pretty graphs based on the data stored. It is pretty easy to gather status information from all sorts of things, ranging from the temperature in your office to the number of octets which have passed through the FDDI interface of your router. But it is not so trivial to store this data in a efficient and systematic manner. This is where RRDtool kicks in. It lets you log and analyze the data you gather from all kinds of data-sources (DS). The data analysis part of RRDtool is based on the ability to quickly generate graphical representations of the data values collected over a definable time period.

White-elephant - Hadoop log aggregator and dashboard

•    Java

---

White Elephant is a Hadoop log aggregator and dashboard which enables visualization of Hadoop cluster utilization across users. The server is a JRuby web application. In a production environment it can be deployed to tomcat and reads aggregated usage data directly from Hadoop. This data is stored in an in-memory database provided by HyperSQL. Charting is provided by Rickshaw. This project is developed by LinkedIn.

log-analysis log-aggregation hadoop-logs visualization log-management

Sentry - Realtime Platform-Agnostic Error Logging and Aggregation platform

•    Python

---

Sentry is a realtime event logging and aggregation platform. It specializes in monitoring errors and extracting all the information needed to do a proper post-mortem without any of the hassle of the standard user feedback loop.

log log-analysis log-search log-management aggregation event-logging

Kafka - A high-throughput distributed messaging system

•    Java

---

Kafka provides a publish-subscribe solution that can handle all activity stream data and processing on a consumer-scale web site. This kind of activity (page views, searches, and other user actions) are a key ingredient in many of the social feature on the modern web. This data is typically handled by "logging" and ad hoc log aggregation solutions due to the throughput requirements. This kind of ad hoc solution is a viable solution to providing logging data to Hadoop.

log-analysis log-aggregation messaging-system publish-subscribe messaging log-management

Epylog - a Syslog parser

•    Python

---

Epylog is a syslog parser which runs periodically, looks at your logs, processes some of the entries in order to present them in a more comprehensible format, and then mails you the output. It is written specifically for large network clusters where a lot of machines (around 50 and upwards) log to the same loghost using syslog or syslog-ng.

log log-analysis log-management

Webalizer - fast web server log file analysis

•    C

---

The Webalizer is a fast web server log file analysis program. It produces highly detailed, easily configurable usage reports in HTML format, for viewing with a standard web browser. It handles standard Common logfile format (CLF) server logs, several variations of the NCSA Combined logfile format, wu-ftpd/proftpd xferlog (FTP) format logs, Squid proxy server native format, and W3C Extended log formats.

web-analytics web-traffic google-analytics-alternative log log-analysis

Awstats - Advanced web, streaming, ftp and mail server statistics

•    CGI

---

AWStats is a powerful tool that generates advanced web, streaming, ftp or mail server statistics graphically. It can analyze log files from all major server tools like Apache log files, WebStar, IIS and a lot of other web, proxy, wap, streaming servers, mail servers and some ftp servers. This log analyzer works as a CGI or from command line and shows you all possible information your log contains, in few graphical web pages.

web-analytics web-traffic google-analytics-alternative log log-analysis

oklog - A distributed and coördination-free log management system

•    Go

---

I hoped to find the opportunity to continue developing OK Log after the spike of its creation. Unfortunately, despite effort, no such opportunity presented itself. Please look at OK Log for inspiration, and consider using the (maintained!) projects that came from it, ulid and run. OK Log is a distributed and coördination-free log management system for big ol' clusters. It's an on-prem solution that's designed to be a sort of building block: easy to understand, easy to operate, and easy to extend.

log logging distributed distributed-systems metallica

Wazuh - Host and endpoint security

•    C

---

Wazuh helps you to gain deeper security visibility into your infrastructure by monitoring hosts at an operating system and application level. This solution, based on lightweight multi-platform agents, provides the capabilities like Log management and analysis, File integrity monitoring, Intrusion and anomaly detection, Policy and compliance monitoring.

ossec security loganalyzer compliance monitoring intrusion-detection policy-monitoring openscap security-hardening ids pci-dss file-integrity-management log-analysis vulnerability-detection incident-response threat-detection

Indihiang - IIS and Apache log analyzing tool

•    CSharp

---

Indihiang Project is a web log analyzing tool. This tool analyzes IIS and Apache Web logs and generates real time reports. It has Web Log Viewer and analyzer. It is capable to analyze the trend from the logs. This tool also integrate with windows Explorer so you can attach a log file in to indihiang tool via context menu.

web-analytics web-traffic google-analytics-alternative log log-analysis

SCRUBS: SQL Reporting Services audit, log, management & optimization analysis

•    

---

SCRUBS your SQL Reporting Services Logs to provide management, auditing & optimization reporting. SSRS provides robust logging in the Execution Log, but no management metrics in the box. You'd have to develop your own DW, SSIS procedures and metrics for reporting. Not anymore!