ReverseDiff.jl - Reverse Mode Automatic Differentiation for Julia

  •        31

Note: While ReverseDiff technically supports Julia v0.7/v1.0 and is somewhat maintained, it is currently not actively developed. Instead, ForwardDiff/ReverseDiff's maintainers are focused on the development of a new AD package built on top of Cassette. In the meantime, it might be worth checking out other reverse-mode AD implementations in Nabla.jl, AutoGrad.jl, Flux.jl, or XGrad.jl. ReverseDiff implements methods to take gradients, Jacobians, Hessians, and higher-order derivatives of native Julia functions (or any callable object, really) using reverse mode automatic differentiation (AD).

https://github.com/JuliaDiff/ReverseDiff.jl

Tags
Implementation
License
Platform

   




Related Projects

Knet.jl - Koç University deep learning framework.

  •    Julia

Knet uses dynamic computational graphs generated at runtime for automatic differentiation of (almost) any Julia code. This allows machine learning models to be implemented by defining just the forward calculation (i.e. the computation from parameters and data to loss) using the full power and expressivity of Julia. The implementation can use helper functions, loops, conditionals, recursion, closures, tuples and dictionaries, array indexing, concatenation and other high level language features, some of which are often missing in the restricted modeling languages of static computational graph systems like Theano, Torch, Caffe and Tensorflow. GPU operation is supported by simply using the KnetArray type instead of regular Array for parameters and data. Knet builds a dynamic computational graph by recording primitive operations during forward calculation. Only pointers to inputs and outputs are recorded for efficiency. Therefore array overwriting is not supported during forward and backward passes. This encourages a clean functional programming style. High performance is achieved using custom memory management and efficient GPU kernels. See Under the hood for more details.

Julia.jl - Curated decibans of Julia language.

  •    Julia

Julia.jl aggregates and curates decibans of knowledge resources for programming in Julia, an all-purpose programming language that addresses the needs of high-performance numerical analysis and computational science. For Base packages, check if the package you seek is listed in the built-in package manager on github, or check METADATA for registered Julia packages, then use the built-in package manager to install it after checking the requirements for respective versions. Pkg3.jl is an alpha next-generation package manager for Julia that creates a Manifest.toml file that records the exact versions of each dependency and their transitive dependencies.

ADMC++

  •    C++

ADMC++ -- An Automatic Differentiation Package for MATLAB and C++ ADMC++ is an automatic differentiation package designed for MATLAB and C++. Automatic differentiation is a technique for computing derivatives of functions.

Arraymancer - A fast, ergonomic and portable tensor library in Nim with a deep learning focus for CPU, GPU, OpenCL and embedded devices

  •    Nim

Arraymancer is a tensor (N-dimensional array) project in Nim. The main focus is providing a fast and ergonomic CPU, Cuda and OpenCL ndarray library on which to build a scientific computing and in particular a deep learning ecosystem. The library is inspired by Numpy and PyTorch. The library provides ergonomics very similar to Numpy, Julia and Matlab but is fully parallel and significantly faster than those libraries. It is also faster than C-based Torch.

HDF5.jl - Saving and loading Julia variables

  •    Julia

Saving and loading Julia variables


tangent - Source-to-Source Debuggable Derivatives in Pure Python

  •    Python

Tangent is a new, free, and open-source Python library for automatic differentiation.Existing libraries implement automatic differentiation by tracing a program's execution (at runtime, like PyTorch) or by staging out a dynamic data-flow graph and then differentiating the graph (ahead-of-time, like TensorFlow). In contrast, Tangent performs ahead-of-time autodiff on the Python source code itself, and produces Python source code as its output. Tangent fills a unique location in the space of machine learning tools.

swift - Swift for TensorFlow project home page

  •    

Swift for TensorFlow is a new way to develop machine learning models. It gives you the power of TensorFlow directly integrated into the Swift programming language. With Swift, you can write the following imperative code, and Swift automatically turns it into a single TensorFlow Graph and runs it with the full performance of TensorFlow Sessions on CPU, GPU and TPU. Swift combines the flexibility of Eager Execution with the high performance of Graphs and Sessions. Behind the scenes, Swift analyzes your Tensor code and automatically builds graphs for you. Swift also catches type errors and shape mismatches before running your code, and has Automatic Differentiation built right in. We believe that machine learning tools are so important that they deserve a first-class language and a compiler.

tfjs-core - WebGL-accelerated ML // linear algebra // automatic differentiation for JavaScript.

  •    TypeScript

NOTE: Building on the momentum of deeplearn.js, we have joined the TensorFlow family and we are starting a new ecosystem of libraries and tools for Machine Learning in Javascript, called TensorFlow.js. This repo moved from PAIR-code/deeplearnjs to tensorflow/tfjs-core. A part of the TensorFlow.js ecosystem, this repo hosts @tensorflow/tfjs-core, the TensorFlow.js Core API, which provides low-level, hardware-accelerated linear algebra operations and an eager API for automatic differentiation.

math - Stan Math Library

  •    C++

The Stan Math Library is a C++, reverse-mode automatic differentiation library designed to be usable, extensive and extensible, efficient, scalable, stable, portable, and redistributable in order to facilitate the construction and utilization of algorithms that utilize derivatives. The Stan Math Library is licensed under the new BSD license.

FuncLib

  •    CSharp

C# automatic differentiation and numerical optimization library, transparent use with operator overloading, unlimited order of differentiation and unlimited number of variables, very flexible support for matrix algebra, on-the-fly function compilation to IL code for very fast ...

gorgonia - Gorgonia is a library that helps facilitate machine learning in Go.

  •    Go

Gorgonia is a library that helps facilitate machine learning in Go. Write and evaluate mathematical equations involving multidimensional arrays easily. If this sounds like Theano or TensorFlow, it's because the idea is quite similar. Specifically, the library is pretty low-level, like Theano, but has higher goals like Tensorflow.The main reason to use Gorgonia is developer comfort. If you're using a Go stack extensively, now you have access to the ability to create production-ready machine learning systems in an environment that you are already familiar and comfortable with.

gorgonia - Gorgonia is a library that helps facilitate machine learning in Go.

  •    Go

Gorgonia is a library that helps facilitate machine learning in Go. Write and evaluate mathematical equations involving multidimensional arrays easily. If this sounds like Theano or TensorFlow, it's because the idea is quite similar. Specifically, the library is pretty low-level, like Theano, but has higher goals like Tensorflow. The main reason to use Gorgonia is developer comfort. If you're using a Go stack extensively, now you have access to the ability to create production-ready machine learning systems in an environment that you are already familiar and comfortable with.

PyCall.jl - Package to call Python functions from the Julia language

  •    Julia

This package provides the ability to directly call and fully interoperate with Python from the Julia language. You can import arbitrary Python modules from Julia, call Python functions (with automatic conversion of types between Julia and Python), define Python classes from Julia methods, and share large data structures between Julia and Python without copying them. Within Julia, just use the package manager to run Pkg.add("PyCall") to install the files. Julia 0.5 or later is required.

hell

  •    

Side note My first version of this experiment wasn't good; I never used it. That was based on shell-conduit, but it's a bit OTT and wasn't as cosy to use as bash. I've been working on a more practical version based somewhat on my work with the jl tool which is simply typed lambda calculus for querying JSON data structures. I believe the same mini-language can work well for a shell.

hypergrad - Exploring differentiation with respect to hyperparameters

  •    Python

Tuning hyperparameters of learning algorithms is hard because gradients are usually unavailable. We compute exact gradients of cross-validation performance with respect to all hyperparameters by chaining derivatives backwards through the entire training procedure. These gradients allow us to optimize thousands of hyperparameters, including step-size and momentum schedules, weight initialization distributions, richly parameterized regularization schemes, and neural network architectures. We compute hyperparameter gradients by exactly reversing the dynamics of stochastic gradient descent with momentum. You'll need to install autograd, our automatic differentiation package. However, autograd (aka funkyYak) has changed a lot since we wrote the hypergrad code, and it would take a little bit of work to make them compatible again.

DifferentialEquations.jl - Julia suite for high-performance solvers of differential equations

  •    Julia

The well-optimized DifferentialEquations solvers benchmark as the some of the fastest implementations, using classic algorithms and ones from recent research which routinely outperform the "standard" C/Fortran methods, and include algorithms optimized for high-precision and HPC applications. At the same time, it wraps the classic C/Fortran methods, making it easy to switch over to them whenever necessary. It integrates with the Julia package sphere, for example using Juno's progress meter, automatic plotting, built-in interpolations, and wraps other differential equation solvers so that many different methods for solving the equations can be accessed by simply switching a keyword argument. It utilizes Julia's generality to be able to solve problems specified with arbitrary number types (types with units like Unitful, and arbitrary precision numbers like BigFloats and ArbFloats), arbitrary sized arrays (ODEs on matrices), and more. This gives a powerful mixture of speed and productivity features to help you solve and analyze your differential equations faster. For information on using the package, see the stable documentation. Use the latest documentation for the version of the documentation which contains the un-released features.

FriCAS

  •    

FriCAS is an advanced computer algebra system. Its capabilities range from calculus (integration and differentiation) to abstract algebra. It can plot functions and has integrated help system.

owl - Owl is an OCaml library for scientific and engineering computing.

  •    OCaml

Owl is an emerging numerical library for scientific computing and engineering. The library is developed in the OCaml language and inherits all its powerful features such as static type checking, powerful module system, and superior runtime efficiency. Owl allows you to write succinct type-safe numerical applications in functional language without sacrificing performance, significantly reduces the cost from prototype to production use. Owl's documentation contains a lot of learning materials to help you start. The full documentation consists of two parts: Tutorial Book and API Reference. Both are perfectly synchronised with the code in the repository by the automatic building system. You can access both parts with the following link.

ANGEL

  •    C++

ANGEL-Automatic differentiation Nested Graph Elimination Library is a template library using the Boost Graph Library and the Standard C++ Library; it provides sparse representations of c-graphs their dual line graphs and vertex, edge and face elimina

ADOL-Py

  •    Python

ADOL-Py is a python extension to the ADOL-C automatic differentiation library.