Apache Metron - Real-time Big Data Security

  •        794

Metron integrates a variety of open source big data technologies in order to offer a centralized tool for security monitoring and analysis. Metron provides capabilities for log aggregation, full packet capture indexing, storage, advanced behavioral analytics and data enrichment, while applying the most current threat intelligence information to security telemetry within a single platform.

Metron will provide next generation defense techniques that consists of using a class of anomaly detection and machine learning algorithms that can be applied in real-time as events are streaming in.

http://metron.apache.org/
https://github.com/apache/metron

Tags
Implementation
License
Platform

   




Related Projects

OSSEC - Host-based Intrusion Detection System

  •    C

OSSEC is a full platform to monitor and control your systems. It mixes together all the aspects of HIDS (host-based intrusion detection), log monitoring and SIM/SIEM together in a simple, powerful and open source solution.

Apache Spot - A Community Approach to Fighting Cyber Threats

  •    Java

Apache Spot is a community-driven cybersecurity project, built from the ground up, to bring advanced analytics to all IT Telemetry data on an open, scalable platform. pot expedites threat detection, investigation, and remediation via machine learning and consolidates all enterprise security data into a comprehensive IT telemetry hub based on open data models.

MISP - MISP (core software) - Open Source Threat Intelligence Platform (formely known as Malware Information Sharing Platform)

  •    PHP

MISP, is an open source software solution for collecting, storing, distributing and sharing cyber security indicators and threat about cyber security incidents analysis and malware analysis. MISP is designed by and for incident analysts, security and ICT professionals or malware reverser to support their day-to-day operations to share structured informations efficiently. The objective of MISP is to foster the sharing of structured information within the security community and abroad. MISP provides functionalities to support the exchange of information but also the consumption of the information by Network Intrusion Detection System (NIDS), LIDS but also log analysis tools, SIEMs.

Proactive Investigator: advanced analytics for threat detection

  •    CSharp

Proactive Investigator is a solution created for information security threat detection. It is an end-to-end solution, currently built on SQL Server 2008

h4cker - This repository is primarily maintained by Omar Santos and includes resources related to ethical hacking / penetration testing, digital forensics and incident response (DFIR), vulnerability research, exploit development, reverse engineering, and more

  •    Java

This repository includes thousands of cybersecurity-related references and resources and it is maintained by Omar Santos. This GitHub repository has been created to provide supplemental material to several books, video courses, and live training created by Omar Santos and other co-authors. It provides over 6,000 references, scripts, tools, code, and other resources that help offensive and defensive security professionals learn and develop new skills. This GitHub repository provides guidance on how build your own hacking environment, learn about offensive security (ethical hacking) techniques, vulnerability research, exploit development, reverse engineering, malware analysis, threat intelligence, threat hunting, digital forensics and incident response (DFIR), includes examples of real-life penetration testing reports, and more. These courses serve as comprehensive guide for any network and security professional who is starting a career in ethical hacking and penetration testing. It also can help individuals preparing for the Offensive Security Certified Professional (OSCP), the Certified Ethical Hacker (CEH), CompTIA PenTest+ and any other ethical hacking certification. This course helps any cyber security professional that want to learn the skills required to becoming a professional ethical hacker or that want to learn more about general hacking methodologies and concepts.


ngx-graph - Graph visualization library for angular

  •    TypeScript

This library is focused on handling graph data (anything with nodes and edges) rather than chart data. Currently the only visualization uses the Dagre layout, which is specialized for directed graphs. The plan is to implement multiple visualisations for graph data within this same library. Eventually, ngx-charts-force-directed-graph may be imported into this library as another option to visualize your graph data. ngx-graph is a Swimlane open-source project; we believe in giving back to the open-source community by sharing some of the projects we build for our application. Swimlane is an automated cyber security operations and incident response platform that enables cyber security teams to leverage threat intelligence, speed up incident response and automate security operations.

Suricata IDS - Network threat detection engine

  •    C

The Suricata engine is capable of real time intrusion detection (IDS), inline intrusion prevention (IPS), network security monitoring (NSM) and offline pcap processing. Suricata inspects the network traffic using a powerful and extensive rules and signature language, and has powerful Lua scripting support for detection of complex threats.

DetectionLab - Vagrant & Packer scripts to build a lab environment complete with security tooling and logging best practices

  •    HTML

This lab has been designed with defenders in mind. Its primary purpose is to allow the user to quickly build a Windows domain that comes pre-loaded with security tooling and some best practices when it comes to system logging configurations. It can easily be modified to fit most needs or expanded to include additional hosts.NOTE: This lab has not been hardened in any way and runs with default vagrant credentials. Please do not connect or bridge it to any networks you care about. This lab is deliberately designed to be insecure; the primary purpose of it is to provide visibility and introspection into each host.

awesome-threat-intelligence - A curated list of Awesome Threat Intelligence resources

  •    

A concise definition of Threat Intelligence: evidence-based knowledge, including context, mechanisms, indicators, implications and actionable advice, about an existing or emerging menace or hazard to assets that can be used to inform decisions regarding the subject’s response to that menace or hazard. Feel free to contribute.

osquery - SQL powered operating system instrumentation, monitoring, and analytics.

  •    C++

osquery is an operating system instrumentation framework for OS X/macOS, Windows, and Linux. The tools make low-level operating system analytics and monitoring both performant and intuitive.There are many additional continuous build jobs that perform dynamic and static analysis, test the package build process, rebuild dependencies from source, assure deterministic build on macOS and Linux, fuzz test the virtual tables, and build on several other platforms not included above. Code safety, testing rigor, data integrity, and a friendly development community are our primary goals.

awesome-threat-detection - A curated list of awesome threat detection and hunting resources

  •    

Contributions welcome! Read the contribution guidelines first. To the extent possible under law, Adel "0x4D31" Karimi has waived all copyright and related or neighboring rights to this work.

caldera - An automated adversary emulation system

  •    Python

CALDERA is an automated adversary emulation system that performs post-compromise adversarial behavior within Windows Enterprise networks. It generates plans during operation using a planning system and a pre-configured adversary model based on the Adversarial Tactics, Techniques & Common Knowledge (ATT&CK™) project. These features allow CALDERA to dynamically operate over a set of systems using variable behavior, which better represents how human adversaries perform operations than systems that follow prescribed sequences of actions. CALDERA is useful for defenders who want to generate real data that represents how an adversary would typically behave within their networks. Since CALDERA's knowledge about a network is gathered during its operation and is used to drive its use of techniques to reach a goal, defenders can get a glimpse into how the intrinsic security dependencies of their network allow an adversary to be successful. CALDERA is useful for identifying new data sources, creating and refining behavioral-based intrusion detection analytics, testing defenses and security configurations, and generating experience for training.

packetpig - Packetpig - Open Source Big Data Security Analytics

  •    Python

Packetpig - Open Source Big Data Security Analytics

joy - A package for capturing and analyzing network flow data and intraflow data, for network research, forensics, and security monitoring

  •    C

Joy is a BSD-licensed libpcap-based software package for extracting data features from live network traffic or packet capture (pcap) files, using a flow-oriented model similar to that of IPFIX or Netflow, and then representing these data features in JSON. It also contains analysis tools that can be applied to these data files. Joy can be used to explore data at scale, especially security and threat-relevant data. JSON is used in order to make the output easily consumable by data analysis tools. While the JSON output files are somewhat verbose, they are reasonably small, and they respond well to compression.

Sguil - The Analyst Console for Network Security Monitoring

  •    Tcl

Sguil (pronounced sgweel) is built by network security analysts for network security analysts. Sguil's main component is an intuitive GUI that provides access to realtime events, session data, and raw packet captures. Sguil facilitates the practice of Network Security Monitoring and event driven analysis.

MozDef - MozDef: The Mozilla Defense Platform

  •    Javascript

The inspiration for MozDef comes from the large arsenal of tools available to attackers. Suites like metasploit, armitage, lair, dradis and others are readily available to help attackers coordinate, share intelligence and finely tune their attacks in real time. Defenders are usually limited to wikis, ticketing systems and manual tracking databases attached to the end of a Security Information Event Management (SIEM) system.The Mozilla Defense Platform (MozDef) seeks to automate the security incident handling process and facilitate the real-time activities of incident handlers.

binaryalert - BinaryAlert: Serverless, Real-time & Retroactive Malware Detection

  •    Python

BinaryAlert is an open-source serverless AWS pipeline where any file uploaded to an S3 bucket is immediately scanned with a configurable set of YARA rules. An alert will fire as soon as any match is found, giving an incident response team the ability to quickly contain the threat before it spreads.

Ranger - Manage Data Security across the Hadoop Platform

  •    Java

Ranger is a framework to enable, monitor and manage comprehensive data security across the Hadoop platform. It provides centralized security administration to manage all security related tasks in a central UI or using REST APIs, Fine grained authorization, Centralize auditing of user access within Apache Hadoop, Apache Hive, Apache HBase and other Apache components.

arachni - Web Application Security Scanner Framework

  •    Ruby

Arachni is a feature-full, modular, high-performance Ruby framework aimed towards helping penetration testers and administrators evaluate the security of web applications. It is smart, it trains itself by monitoring and learning from the web application's behavior during the scan process and is able to perform meta-analysis using a number of factors in order to correctly assess the trustworthiness of results and intelligently identify (or avoid) false-positives.

Bro - Network Security Monitor

  •    C++

Bro is a powerful network analysis framework that is much different from the typical intrusion detection system you may know. Bro provides a comprehensive platform for more general network traffic analysis as well.