PyMC3 - Probabilistic Programming in Python: Bayesian Modeling and Probabilistic Machine Learning with Theano

•    Python

---

PyMC3 is a Python package for Bayesian statistical modeling and Probabilistic Machine Learning which focuses on advanced Markov chain Monte Carlo and variational fitting algorithms. Its flexibility and extensibility make it applicable to a large suite of problems.Note: Running pip install pymc will install PyMC 2.3, not PyMC3, from PyPI.

statistical-analysis bayesian-inference mcmc variational-inference theano probabilistic-programming bayesian

Probabilistic-Programming-and-Bayesian-Methods-for-Hackers - aka "Bayesian Methods for Hackers": An introduction to Bayesian methods + probabilistic programming with a computation/understanding-first, mathematics-second point of view

•    Jupyter

---

The Bayesian method is the natural approach to inference, yet it is hidden from readers behind chapters of slow, mathematical analysis. The typical text on Bayesian inference involves two to three chapters on probability theory, then enters what Bayesian inference is. Unfortunately, due to mathematical intractability of most Bayesian models, the reader is only shown simple, artificial examples. This can leave the user with a so-what feeling about Bayesian inference. In fact, this was the author's own prior opinion. After some recent success of Bayesian methods in machine-learning competitions, I decided to investigate the subject again. Even with my mathematical background, it took me three straight-days of reading examples and trying to put the pieces together to understand the methods. There was simply not enough literature bridging theory to practice. The problem with my misunderstanding was the disconnect between Bayesian mathematics and probabilistic programming. That being said, I suffered then so the reader would not have to now. This book attempts to bridge the gap.

bayesian-methods pymc mathematical-analysis jupyter-notebook data-science statistics

infer - Infer.NET is a framework for running Bayesian inference in graphical models

•    CSharp

---

Infer.NET is a framework for running Bayesian inference in graphical models. It can also be used for probabilistic programming. One can use Infer.NET to solve many different kinds of machine learning problems - from standard problems like classification, recommendation or clustering through to customised solutions to domain-specific problems.

machine-learning bayesian-inference

edward - A probabilistic programming language in TensorFlow

•    Jupyter

---

Edward is a Python library for probabilistic modeling, inference, and criticism. It is a testbed for fast experimentation and research with probabilistic models, ranging from classical hierarchical models on small data sets to complex deep probabilistic models on large data sets. Edward fuses three fields: Bayesian statistics and machine learning, deep learning, and probabilistic programming. Edward is built on top of TensorFlow. It enables features such as computational graphs, distributed training, CPU/GPU integration, automatic differentiation, and visualization with TensorBoard.

bayesian-methods deep-learning machine-learning data-science tensorflow neural-networks statistics probabilistic-programming

probability - Probabilistic reasoning and statistical analysis in TensorFlow

•    Jupyter

---

TensorFlow Probability is a library for probabilistic reasoning and statistical analysis in TensorFlow. As part of the TensorFlow ecosystem, TensorFlow Probability provides integration of probabilistic methods with deep networks, gradient-based inference via automatic differentiation, and scalability to large datasets and models via hardware acceleration (e.g., GPUs) and distributed computation. Our probabilistic machine learning tools are structured as follows.

tensorflow bayesian-methods deep-learning machine-learning data-science neural-networks statistics probabilistic-programming

Turing.jl - The Turing language for probabilistic programming

•    Julia

---

News: Turing.jl is now Julia 1.0 compatible now! Be aware that some things still might fail. Turing was originally created and is now managed by Hong Ge. Current and past Turing team members include Hong Ge, Adam Scibior, Matej Balog, Zoubin Ghahramani, Kai Xu, Emma Smith, Emile Mathieu, Martin Trapp. You can see the full list of on Github: https://github.com/TuringLang/Turing.jl/graphs/contributors.

machine-learning probabilistic-programming mcmc-sampler julia-language artificial-intelligence bayesian-inference

pystan - PyStan, the Python interface to Stan

•    Python

---

PyStan provides a Python interface to Stan, a package for Bayesian inference using the No-U-Turn sampler, a variant of Hamiltonian Monte Carlo. For more information on Stan and its modeling language, see the Stan User's Guide and Reference Manual at http://mc-stan.org/.

stan statistics machine-learning probabilistic-programming

brms - brms R package for Bayesian generalized multivariate non-linear multilevel models using Stan

•    R

---

The brms package provides an interface to fit Bayesian generalized (non-)linear multivariate multilevel models using Stan, which is a C++ package for performing full Bayesian inference (see http://mc-stan.org/). The formula syntax is very similar to that of the package lme4 to provide a familiar and simple interface for performing regression analyses. A wide range of distributions and link functions are supported, allowing users to fit -- among others -- linear, robust linear, count data, survival, response times, ordinal, zero-inflated, hurdle, and even self-defined mixture models all in a multilevel context. Further modeling options include non-linear and smooth terms, auto-correlation structures, censored data, missing value imputation, and quite a few more. In addition, all parameters of the response distribution can be predicted in order to perform distributional regression. Multivariate models (i.e. models with multiple response variables) can be fitted, as well. Prior specifications are flexible and explicitly encourage users to apply prior distributions that actually reflect their beliefs. Model fit can easily be assessed and compared with posterior predictive checks, leave-one-out cross-validation, and Bayes factors. As a simple example, we use poisson regression to model the seizure counts in epileptic patients to investigate whether the treatment (represented by variable Trt) can reduce the seizure counts and whether the effect of the treatment varies with the baseline number of seizures a person had before treatment (variable log_Base4_c). As we have multiple observations per person, a group-level intercept is incorporated to account for the resulting dependency in the data.

brms stan bayesian-inference multilevel-models statistical-models r-package

JProGraM

•    Java

---

JProGraM (PRObabilistic GRAphical Models in Java) is a statistical machine learning library. It supports statistical modeling and data analysis along three main directions: (1) probabilistic graphical models (Bayesian networks, Markov random fields, dependency networks, hybrid random fields); (2) parametric, semiparametric, and nonparametric density estimation (Gaussian models, nonparanormal estimators, Parzen windows, Nadaraya-Watson estimator); (3) generative models for random networks (

pmtk3 - Probabilistic Modeling Toolkit for Matlab/Octave.

•    HTML

---

PMTK is a collection of Matlab/Octave functions, written by Matt Dunham, Kevin Murphy and various other people. The toolkit is primarily designed to accompany Kevin Murphy's textbook Machine learning: a probabilistic perspective, but can also be used independently of this book. The goal is to provide a unified conceptual and software framework encompassing machine learning, graphical models, and Bayesian statistics (hence the logo). (Some methods from frequentist statistics, such as cross validation, are also supported.) Since December 2011, the toolbox is in maintenance mode, meaning that bugs will be fixed, but no new features will be added (at least not by Kevin or Matt). PMTK builds on top of several existing packages, available from pmtksupport, and provides a unified interface to them. In addition, it provides readable "reference" implementations of many common machine learning techniques. The vast majority of the code is written in Matlab. (For a brief discussion of why we chose Matlab, click here. Most of the code also runs on Octave an open-source Matlab clone.) However, in a few cases we also provide wrappers to implementations written in C, for speed reasons. PMTK currently (October 2010) has over 67,000 lines.

bayadera - High-performance Bayesian Data Analysis on the GPU in Clojure

•    Clojure

---

A Clojure Library for Bayesian Data Analysis and Machine Learning on the GPU. Distributed under the Eclipse Public License either version 1.0 or (at your option) any later version.

bayesian-inference bayesian-data-analysis gpu-computing gpu-acceleration statistics machine-learning clojure-library bayesian opencl cuda high-performance-computing gpu mcmc markov-chain-monte-carlo

bayes-nn - Lecture notes on Bayesian deep learning

•    

---

Lecture notes on Bayesian deep learning

deep-learning probability-theory variational-inference

dowhy - DoWhy is a Python library that makes it easy to estimate causal effects

•    Python

---

As computing systems are more frequently and more actively intervening in societally critical domains such as healthcare, education and governance, it is critical to correctly predict and understand the causal effects of these interventions. Without an A/B test, conventional machine learning methods, built on pattern recognition and correlational analyses, are insufficient for causal reasoning. Much like machine learning libraries have done for prediction, "DoWhy" is a Python library that aims to spark causal thinking and analysis. DoWhy provides a unified interface for causal inference methods and automatically tests many assumptions, thus making inference accessible to non-experts.

causal-inference python3 graphical-models

Bayesian Network Tools in Java (BNJ)

•    Java

---

Java/XML toolkit for research using Bayesian networks and other graphical models of probability (exact and approximate inference, structure learning, etc.)

pgmpy - Python Library for Probabilistic Graphical Models

•    Python

---

pgmpy is a python library for working with Probabilistic Graphical Models. Our mailing list is at https://groups.google.com/forum/#!forum/pgmpy .

probabilistic-graphical-models bayesian-networks

bayesian-belief-networks - Pythonic Bayesian Belief Network Package, supporting creation of and exact inference on Bayesian Belief Networks specified as pure python functions

•    Python

---

Pythonic Bayesian Belief Network Package, supporting creation of and exact inference on Bayesian Belief Networks specified as pure python functions.

PNL

•    C++

---

The Open Source Probabilistic Networks Library is a tool for working with graphical models. It supports directed and undirected models, discrete and continuous variables, various inference and learning algorithms.

VIBES

•    

---

VIBES stands for Variational Inference in BayES Nets. It consists of a graphical Bayes Net editor and an inference engine which allows variational inference to be applied automatically using Variational Message Passing.

Bios8366 - Advanced Statistical Computing at Vanderbilt University's Department of Biostatistics

•    Jupyter

---

Course covers numerical optimization, statistical machine learning, Markov Chain Monte Carlo (MCMC), variational inference (VI) algorithms, data augmentation algorithms with applications for model fitting and techniques for dealing with missing data. Prerequisites: Bios 6341 (Fundamentals of Probability), Bios 6342 (Contemporary Statistical Inference), or permission of instructor. Students must be familiar with basic probability, have some formal programming experience, and be comfortable using the Git version control system.

pomegranate - Fast, flexible and easy to use probabilistic modelling in Python.

•    Jupyter

---

JMLR-MLOSS Manuscript Please consider citing it if you used it in your academic work. pomegranate is a package for probabilistic models in Python that is implemented in cython for speed. It's focus is on merging the easy-to-use scikit-learn API with the modularity that comes with probabilistic modeling to allow users to specify complicated models without needing to worry about implementation details. The models are built from the ground up with big data processing in mind and so natively support features like out-of-core learning and parallelism.

machine-learning probabilistic-graphical-models cython parallel