Meniscus - The Python Event Logging Service

  •        3837

Meniscus is a Python based system for event collection, transit and processing in the large. It's primary use case is for large-scale Cloud logging, but can be used in many other scenarios including usage reporting and API tracing. Its components include Collection, Transport, Storage, Event Processing & Enhancement, Complex Event Processing, Analytics.

http://projectmeniscus.org/
https://github.com/ProjectMeniscus/meniscus

Tags
Implementation
License
Platform

   




Related Projects

Fluentd - Unified Logging Layer

  •    Ruby

Fluentd collects events from various data sources and writes them to files, RDBMS, NoSQL, IaaS, SaaS, Hadoop and so on. Fluentd helps you unify your logging infrastructure. Fluentd can be used to tail access/error logs and transport them reliably to remote systems. It can "grep" for events and send out alerts. It can function as middleware to enable asynchronous, scalable logging for user action events.

nxlog - Multi platform Log management

  •    C

nxlog is a modular, multi-threaded, high-performance log management solution with multi-platform support. In concept it is similar to syslog-ng or rsyslog but is not limited to unix/syslog only. It can collect logs from files in various formats, receive logs from the network remotely over UDP, TCP or TLS/SSL . It supports platform specific sources such as the Windows Eventlog, Linux kernel logs, Android logs, local syslog etc.

LogTrail - Log Viewer plugin for Kibana

  •    Javascript

LogTrail is a plugin for Kibana to view, analyze, search and tail log events from multiple hosts in realtime with devops friendly interface inspired by Papertrail.

APT-Hunter - APT-Hunter is Threat Hunting tool for windows event logs which made by purple team mindset to provide detect APT movements hidden in the sea of windows event logs to decrease the time to uncover suspicious activity

  •    Python

APT-Hunter is Threat Hunting tool for windows event logs which made by purple team mindset to provide detect APT movements hidden in the sea of windows event logs to decrease the time to uncover suspicious activity . this tool will make a good use of the windows event logs collected and make sure to not miss critical events configured to be detected. If you are a Threat Hunter , Incident Responder or forensic investigator , i assure you will enjoy using this tool , why ? i will discuss the reason in this article and how it will make your life easy just it made mine . Kindly note this tool is heavily tested but still a beta version and may contain bugs . The first thing to do is to collect the logs if you didn’t and with powershell log collectors its easy to collect the needed logs automatically you just run the powershell scripts as administrator .

nxlog

  •    C

A multi-platform universal log collector and forwarder


Zenoss - Open Source IT Management

  •    Python

Zenoss Core is an open source IT monitoring product that delivers the functionality to effectively manage the configuration, health, performance of networks, servers and applications through a single, integrated software package.

Epylog - a Syslog parser

  •    Python

Epylog is a syslog parser which runs periodically, looks at your logs, processes some of the entries in order to present them in a more comprehensible format, and then mails you the output. It is written specifically for large network clusters where a lot of machines (around 50 and upwards) log to the same loghost using syslog or syslog-ng.

Event Log Analyzer

  •    WPF

Event Log Analyzer is a simple yet powerful tool to analyze event logs in Windows. It has features to group similar events together, give graphical view of distribution of events over time and in similarity. It is developed in C# and WPF and uses MVVM framework.

Event Log to SysLog

  •    CSharp

el2sl mean Event Lot to SysLog. This program start as windows service and send windows event logs to syslog server over network. Source code avialable on github: https://github.com/Sheridan/el2sl

Graylog2 - Open Source Log Management

  •    Java

Graylog2 is an open source log management solution that stores your logs in ElasticSearch. It consists of a server written in Java that accepts your syslog messages via TCP, UDP or AMQP and stores it in the database. The second part is a web interface that allows you to manage the log messages from your web browser. Take a look at the screenshots or the latest release info page to get a feeling of what you can do with Graylog2.

Tremor - An early-stage event processing system for unstructured data with rich support for structural pattern-matching, filtering and transformation

  •    Rust

Tremor is an event processing system. It was originally designed as a replacement for software such as Logstash or Telegraf. However tremor has outgrown this singular use case by supporting more complex workflows such as aggregation, rollups, an ETL language, and a query language.

LogonTracer - Investigate malicious Windows logon by visualizing and analyzing Windows event log

  •    Python

LogonTracer is a tool to investigate malicious logon by visualizing and analyzing Windows Active Directory event logs. This tool associates a host name (or an IP address) and account name found in logon-related events and displays it as a graph. This way, it is possible to see in which account login attempt occurs and which host is used. This tool can visualize the following event id related to Windows logon based on this research. LogonTracer uses PageRank, Hidden Markov model and ChangeFinder to detect malicious hosts and accounts from event log. With LogonTracer, it is also possible to display event logs in a chronological order.

Event-Forwarding-Guidance - Configuration guidance for implementing collection of security relevant Windows Event Log events by using Windows Event Forwarding

  •    PowerShell

This repository hosts content for aiding administrators in collecting security relevant Windows event logs using Windows Event Forwarding (WEF). This repository is a companion to Spotting the Adversary with Windows Event Log Monitoring paper. The list of events in this repository are more up to date than those in the paper. See LICENSE.

Octopussy - Perl/XML Logs Analyzer, Alerter & Reporter

  •    Perl

Octopussy is a Log analyzer tool. It analyzes the log, generates reports and alerts the admin. It has LDAP support to maintain users list. It exports report by Email, FTP & SCP. Scheduled reports could be generated. RRD tool to generate graphs.

liblogfaf - A library that logs messages using non-blocking UDP datagrams.

  •    C

liblogfaf (faf stands for fire-and-forget) is a dynamic library that is designed to be LD_PRELOAD-ed while starting a process that uses openlog() & syslog() functions to send syslog messages. It overrides logging functions to make log messages sent as UDP datagrams instead of getting written to /dev/log (which can block). This is useful for processes that call syslog() as part of their main execution flow and can therefore be easily broken when /dev/log buffer gets full, for example when the process that is expected to read from it (usually system syslog daemon like rsyslog or syslog-ng) stops doing that.Please note that liblogfaf should not be used in an environment where reliable log message delivery is required.

liblogfaf - A library that logs messages using non-blocking UDP datagrams.

  •    C

liblogfaf (faf stands for fire-and-forget) is a dynamic library that is designed to be LD_PRELOAD-ed while starting a process that uses openlog() & syslog() functions to send syslog messages. It overrides logging functions to make log messages sent as UDP datagrams instead of getting written to /dev/log (which can block). This is useful for processes that call syslog() as part of their main execution flow and can therefore be easily broken when /dev/log buffer gets full, for example when the process that is expected to read from it (usually system syslog daemon like rsyslog or syslog-ng) stops doing that. Please note that liblogfaf should not be used in an environment where reliable log message delivery is required.

Windows Event Log Manager

  •    

Windows Event Log Manager is a tool for creating, editing, and viewing windows event logs.

awslogs - AWS CloudWatch logs for Humans™

  •    Python

awslogs is a simple command line tool for querying groups, streams and events from Amazon CloudWatch logs. Running: awslogs get /var/logs/syslog ALL -s1d will return you events from any stream in the /var/logs/syslog group generated in the last day.

Fluent Bit - Fast and Lightweight Logs and Metrics processor

  •    C

Fluent Bit is a fast Log Processor and Forwarder, it allows to collect log events or metrics from different sources, process them and deliver them to different backends such as Fluentd, Elasticsearch, Splunk, DataDog, Kafka, New Relic, Azure services, AWS services, Google services, NATS, InfluxDB or any custom HTTP end-point. It also comes with full SQL Stream Processing capabilities: data manipulation and analytics using SQL queries.

Webalizer - fast web server log file analysis

  •    C

The Webalizer is a fast web server log file analysis program. It produces highly detailed, easily configurable usage reports in HTML format, for viewing with a standard web browser. It handles standard Common logfile format (CLF) server logs, several variations of the NCSA Combined logfile format, wu-ftpd/proftpd xferlog (FTP) format logs, Squid proxy server native format, and W3C Extended log formats.






We have large collection of open source products. Follow the tags from Tag Cloud >>


Open source products are scattered around the web. Please provide information about the open source projects you own / you use. Add Projects.