Apache Metron - Real-time Big Data Security

  •        752

Metron integrates a variety of open source big data technologies in order to offer a centralized tool for security monitoring and analysis. Metron provides capabilities for log aggregation, full packet capture indexing, storage, advanced behavioral analytics and data enrichment, while applying the most current threat intelligence information to security telemetry within a single platform.

Metron will provide next generation defense techniques that consists of using a class of anomaly detection and machine learning algorithms that can be applied in real-time as events are streaming in.

http://metron.apache.org/
https://github.com/apache/metron

Tags
Implementation
License
Platform

   




Related Projects

OSSEC - Host-based Intrusion Detection System


OSSEC is a full platform to monitor and control your systems. It mixes together all the aspects of HIDS (host-based intrusion detection), log monitoring and SIM/SIEM together in a simple, powerful and open source solution.

Apache Spot - A Community Approach to Fighting Cyber Threats


Apache Spot is a community-driven cybersecurity project, built from the ground up, to bring advanced analytics to all IT Telemetry data on an open, scalable platform. pot expedites threat detection, investigation, and remediation via machine learning and consolidates all enterprise security data into a comprehensive IT telemetry hub based on open data models.

Proactive Investigator: advanced analytics for threat detection


Proactive Investigator is a solution created for information security threat detection. It is an end-to-end solution, currently built on SQL Server 2008

Suricata IDS - Network threat detection engine


The Suricata engine is capable of real time intrusion detection (IDS), inline intrusion prevention (IPS), network security monitoring (NSM) and offline pcap processing. Suricata inspects the network traffic using a powerful and extensive rules and signature language, and has powerful Lua scripting support for detection of complex threats.

DetectionLab - Vagrant & Packer scripts to build a lab environment complete with security tooling and logging best practices


This lab has been designed with defenders in mind. Its primary purpose is to allow the user to quickly build a Windows domain that comes pre-loaded with security tooling and some best practices when it comes to system logging configurations. It can easily be modified to fit most needs or expanded to include additional hosts.NOTE: This lab has not been hardened in any way and runs with default vagrant credentials. Please do not connect or bridge it to any networks you care about. This lab is deliberately designed to be insecure; the primary purpose of it is to provide visibility and introspection into each host.


osquery - SQL powered operating system instrumentation, monitoring, and analytics.


osquery is an operating system instrumentation framework for OS X/macOS, Windows, and Linux. The tools make low-level operating system analytics and monitoring both performant and intuitive.There are many additional continuous build jobs that perform dynamic and static analysis, test the package build process, rebuild dependencies from source, assure deterministic build on macOS and Linux, fuzz test the virtual tables, and build on several other platforms not included above. Code safety, testing rigor, data integrity, and a friendly development community are our primary goals.

awesome-threat-detection - A curated list of awesome threat detection and hunting resources


Contributions welcome! Read the contribution guidelines first. To the extent possible under law, Adel "0x4D31" Karimi has waived all copyright and related or neighboring rights to this work.

packetpig - Packetpig - Open Source Big Data Security Analytics


Packetpig - Open Source Big Data Security Analytics

Sguil - The Analyst Console for Network Security Monitoring


Sguil (pronounced sgweel) is built by network security analysts for network security analysts. Sguil's main component is an intuitive GUI that provides access to realtime events, session data, and raw packet captures. Sguil facilitates the practice of Network Security Monitoring and event driven analysis.

MozDef - MozDef: The Mozilla Defense Platform


The inspiration for MozDef comes from the large arsenal of tools available to attackers. Suites like metasploit, armitage, lair, dradis and others are readily available to help attackers coordinate, share intelligence and finely tune their attacks in real time. Defenders are usually limited to wikis, ticketing systems and manual tracking databases attached to the end of a Security Information Event Management (SIEM) system.The Mozilla Defense Platform (MozDef) seeks to automate the security incident handling process and facilitate the real-time activities of incident handlers.

binaryalert - BinaryAlert: Serverless, Real-time & Retroactive Malware Detection


BinaryAlert is an open-source serverless AWS pipeline where any file uploaded to an S3 bucket is immediately scanned with a configurable set of YARA rules. An alert will fire as soon as any match is found, giving an incident response team the ability to quickly contain the threat before it spreads.

Ranger - Manage Data Security across the Hadoop Platform


Ranger is a framework to enable, monitor and manage comprehensive data security across the Hadoop platform. It provides centralized security administration to manage all security related tasks in a central UI or using REST APIs, Fine grained authorization, Centralize auditing of user access within Apache Hadoop, Apache Hive, Apache HBase and other Apache components.

Bro - Network Security Monitor


Bro is a powerful network analysis framework that is much different from the typical intrusion detection system you may know. Bro provides a comprehensive platform for more general network traffic analysis as well.

eagle - Mirror of Apache Eagle


Apache® Eagle™ is an open source analytics solution for identifying security and performance issues instantly on big data platforms e.g. Apache Hadoop, Apache Spark, NoSQL etc.

sigma - Generic Signature Format for SIEM Systems


Sigma is a generic and open signature format that allows you to describe relevant log events in a straight forward manner. The rule format is very flexible, easy to write and applicable to any type of log file. The main purpose of this project is to provide a structured form in which researchers or analysts can describe their once developed detection methods and make them shareable with others. Sigma is for log files what Snort is for network traffic and YARA is for files.

Acra - Database protection suite with selective encryption and intrusion detection


Acra helps you to easily secure your databases in distributed, microservice-rich environments. It allows you to selectively encrypt sensitive records with strong multi-layer cryptography, detect potential intrusions and SQL injections and cryptographically compartment data stored in large sharded schemes. It's security model guarantees that compromising the database or your application does not leak sensitive data, or keys to decrypt it.

atomic-red-team - Small and highly portable detection tests.


Small and highly portable detection tests mapped to the Mitre ATT&CK Framework.Our Atomic Red Team tests are small, highly portable detection tests mapped to the MITRE ATT&CK Framework. Each test is designed to map back to a particular tactic. We hope that this gives defenders a highly actionable way to immediately start testing their defenses against a broad spectrum of attacks.

ThreatExchange - Share threat information with vetted partners


ThreatExchange is a set of RESTful APIs on the Facebook Platform for querying, publishing, and sharing security threat information. It's a light-weight way for exchanging details on malware, phishing pages, and other threats with either specific members of the community or the ThreatExchange community at large.This repository contains example code for using the API.

StratoSphere - Cloud Computing Framework for Big Data Analytics


The Stratosphere System is an open-source cluster/cloud computing framework for Big Data analytics. It comprises of An extensible higher level language (Meteor) to quickly compose queries for common and recurring use cases, A parallel programming model (PACT, an extension of MapReduce) to run user-defined operations, An efficient massively parallel runtime (Nephele) for fault tolerant execution of acyclic data flows.

Endian Firewall Community


Endian Firewall Community (EFW) is a "turn-key" linux security distribution that makes your system a full featured security appliance with Unified Threat Management (UTM) functionalities. The software has been designed for the best usability: very easy to install, use and manage and still greatly flexible. The feature suite includes stateful packet inspection firewall, application-level proxies for various protocols (HTTP, FTP, POP3, SMTP) with antivirus support, virus and spam-filtering f