anomaly-detection-models - some anomaly detection models and experiments.

  •        28

some anomaly detection models and experiments. This is simply a blog for learning and experimentation for now...

https://github.com/yunstanford/anomaly-detection-models

Tags
Implementation
License
Platform

   




Related Projects

Pyod - A Python Toolkit for Scalable Outlier Detection (Anomaly Detection)

  •    Python

Important Notes: PyOD contains some neural network based models, e.g., AutoEncoders, which are implemented in keras. However, PyOD would NOT install keras and tensorflow automatically. This would reduce the risk of damaging your local installations. You are responsible for installing keras and tensorflow if you want to use neural net based models. An instruction is provided here. Anomaly detection resources, e.g., courses, books, papers and videos.

NAB - The Numenta Anomaly Benchmark

  •    Python

Welcome. This repository contains the data and scripts comprising the Numenta Anomaly Benchmark (NAB). NAB is a novel benchmark for evaluating algorithms for anomaly detection in streaming, real-time applications. It is comprised of over 50 labeled real-world and artificial timeseries data files plus a novel scoring mechanism designed for real-time applications. Included are the tools to allow you to easily run NAB on your own anomaly detection algorithms; see the NAB entry points info. Competitive results tied to open source code will be posted in the wiki on the Scoreboard. Let us know about your work by emailing us at nab@numenta.org or submitting a pull request.

Apache Metron - Real-time Big Data Security

  •    Java

Metron integrates a variety of open source big data technologies in order to offer a centralized tool for security monitoring and analysis. Metron provides capabilities for log aggregation, full packet capture indexing, storage, advanced behavioral analytics and data enrichment, while applying the most current threat intelligence information to security telemetry within a single platform.

AnomalyDetection - Anomaly Detection with R

  •    R

AnomalyDetection is an open-source R package to detect anomalies which is robust, from a statistical standpoint, in the presence of seasonality and an underlying trend. The AnomalyDetection package can be used in wide variety of contexts. For example, detecting anomalies in system metrics after a new software release, user engagement post an A/B test, or for problems in econometrics, financial engineering, political and social sciences.

morgoth - Metric anomaly detection

  •    Go

Morgoth provides a framework for implementing the smaller pieces of an anomaly detection problem. The basic framework is that Morgoth maintains a dictionary of normal behaviors and compares new windows of data to the normal dictionary. If the new window of data is not found in the dictionary then it is considered anomalous.


Deeplearning4J - Neural Net Platform in Java and Scala

  •    Java

Deeplearning4J is an open source, distributed neural net library written in Java and Scala. It integrates with Hadoop and Spark and runs on several backends that enable use of CPUs and GPUs. It provides versatile n-dimensional array class for Java and Scala.

devialog

  •    Perl

devialog is a behavior/anomaly-based syslog intrusion detection system which detects unknown attacks via anomalies in syslog. It can generate signatures for ease of management, act upon anomalies in a predefined fashion or perform as a standard log parser

vSentinel

  •    C

vSentinel is a customizable 3D mapping of your network monitoring or security data for real-time or trend-based attack and anomaly detection and analysis.

Jubatus - Framework and Library for Distributed Online Machine Learning

  •    C++

Jubatus is a distributed processing framework and streaming machine learning library. Jubatus includes these functionalities: Online Machine Learning Library: Classification, Regression, Recommendation (Nearest Neighbor Search), Graph Mining, Anomaly Detection, Clustering, Feature Vector Converter (fv_converter): Data Preprocess and Feature Extraction, Framework for Distributed Online Machine Learning with Fault Tolerance.

egads - Extendible Generic Anomaly Detection System

  •    Java

EGADS system, consumes time series and outputs anomalies.

bell.js - No longer maintained. Use https://github.com/eleme/banshee instead please.

  •    Javascript

Bell.js is a real-time anomalies(outliers) detection system for periodic time series, built to be able to monitor a large quantity of metrics. It collects metrics form statsd, analyzes them with the 3-sigma, once enough anomalies were found in a short time it alerts us via sms/email etc.We eleme use it to monitor our website/rpc interfaces, including api called frequency, api response time(time cost per call) and exceptions count. Our services send these statistics to statsd, statsd aggregates them every 10 seconds and broadcasts the results to its backends including bell, bell analyzes current stats with history data, calculates the trending, and alerts us if the trending behaves anomalous.

ourmon

  •    C

Ourmon is a network monitoring and anomaly detection system and displays the data for multiple BPF expressions via RRDTOOL-based graphs. It also helps the user identify various kinds of network anomalies using various flow analysis tools and logging.

nupic - Numenta Platform for Intelligent Computing is an implementation of Hierarchical Temporal Memory (HTM), a theory of intelligence based strictly on the neuroscience of the neocortex

  •    Python

The Numenta Platform for Intelligent Computing (NuPIC) is a machine intelligence platform that implements the HTM learning algorithms. HTM is a detailed computational theory of the neocortex. At the core of HTM are time-based continuous learning algorithms that store and recall spatial and temporal patterns. NuPIC is suited to a variety of problems, particularly anomaly detection and prediction of streaming data sources. For more information, see numenta.org or the NuPIC Forum. For usage guides, quick starts, and API documentation, see http://nupic.docs.numenta.org/.

macrobase - MacroBase: A Search Engine for Fast Data

  •    Java

MacroBase is an anomaly detection engine designed to prioritize human attention in large-scale datasets and data streams.

wdbgark - WinDBG Anti-RootKit Extension

  •    C++

WDBGARK is an extension (dynamic library) for the Microsoft Debugging Tools for Windows. It main purpose is to view and analyze anomalies in Windows kernel using kernel debugger. It is possible to view various system callbacks, system tables, object types and so on. For more user-friendly view extension uses DML. For the most of commands kernel-mode connection is required. Feel free to use extension with live kernel-mode debugging or with kernel-mode crash dump analysis (some commands will not work). Public symbols are required, so use them, force to reload them, ignore checksum problems, prepare them before analysis and you'll be happy. Windows BETA/RC is supported by design, but read a few notes. First, i don't care about checked builds. Second, i don't care if you don't have symbols (public or private). IA64/ARM is unsupported (and will not).

Apache Spot - A Community Approach to Fighting Cyber Threats

  •    Java

Apache Spot is a community-driven cybersecurity project, built from the ground up, to bring advanced analytics to all IT Telemetry data on an open, scalable platform. pot expedites threat detection, investigation, and remediation via machine learning and consolidates all enterprise security data into a comprehensive IT telemetry hub based on open data models.

DeepVideoAnalytics - A distributed visual search and visual data analytics platform.

  •    Python

Deep Video Analytics is a platform for indexing and extracting information from videos and images. With latest version of docker installed correctly, you can run Deep Video Analytics in minutes locally (even without a GPU) using a single command. Deep Video Analytics implements a client-server architecture pattern, where clients can access state of the server via a REST API. For uploading, processing data, training models, performing queries, i.e. mutating the state clients can send DVAPQL (Deep Video Analytics Processing and Query Language) formatted as JSON. The query represents a directed acyclic graph of operations.

OSSEC - Host-based Intrusion Detection System

  •    C

OSSEC is a full platform to monitor and control your systems. It mixes together all the aspects of HIDS (host-based intrusion detection), log monitoring and SIM/SIEM together in a simple, powerful and open source solution.

pytorch-segmentation-detection - Image Segmentation and Object Detection in Pytorch

  •    Jupyter

So far, the library contains an implementation of FCN-32s (Long et al.), Resnet-18-8s, Resnet-34-8s (Chen et al.) image segmentation models in Pytorch and Pytorch/Vision library with training routine, reported accuracy, trained models for PASCAL VOC 2012 dataset. To train these models on your data, you will have to write a dataloader for your dataset. Models for Object Detection will be released soon.