Sentry - Realtime Platform-Agnostic Error Logging and Aggregation platform

•    Python

---

Sentry is a realtime event logging and aggregation platform. It specializes in monitoring errors and extracting all the information needed to do a proper post-mortem without any of the hassle of the standard user feedback loop.

log log-analysis log-search log-management aggregation event-logging

GoAccess - Real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems or through your browser

•    C

---

GoAccess is an open source real-time web log analyzer and interactive viewer that runs in a terminal on *nix systems or through your browser. It provides fast and valuable HTTP statistics for system administrators that require a visual server report on the fly. It supports nearly all web log formats (Apache, Nginx, Amazon S3, Elastic Load Balancing, CloudFront, etc)

data-analysis analytics nginx apache webserver web-analytics monitoring dashboard command-line log-management log-analysis log-search

Graylog2 - Open Source Log Management

•    Java

---

Graylog2 is an open source log management solution that stores your logs in ElasticSearch. It consists of a server written in Java that accepts your syslog messages via TCP, UDP or AMQP and stores it in the database. The second part is a web interface that allows you to manage the log messages from your web browser. Take a look at the screenshots or the latest release info page to get a feeling of what you can do with Graylog2.

log log-analysis log-search log-management syslog

Fluentd - Unified Logging Layer

•    Ruby

---

Fluentd collects events from various data sources and writes them to files, RDBMS, NoSQL, IaaS, SaaS, Hadoop and so on. Fluentd helps you unify your logging infrastructure. Fluentd can be used to tail access/error logs and transport them reliably to remote systems. It can "grep" for events and send out alerts. It can function as middleware to enable asynchronous, scalable logging for user action events.

log-collector syslog log-analysis log log-management log-aggregator log-search data-collector

Clarity - Web interface for the grep

•    Ruby

---

Clarity is a Splunk like web interface for your server log files. It supports searching (using grep) as well as trailing log files in realtime. It has been written using the event based architecture based on EventMachine and so allows real-time search of very large log files.

log log-analysis log-management log-search

terraform-aws-lambda-elasticsearch-cleanup - Terraform module to provision a scheduled Lambda function which will delete old AWS ElasticSearch indices

•    HCL

---

This project is part of our comprehensive "SweetOps" approach towards DevOps. It's 100% Open Source and licensed under the APACHE2.

elasticsearch lambda terraform logs purge elasticsearch-curator vacuum retention indexes delete terraform-modules log-search lambda-function