awesome-machine-learning-interpretability - A curated list of awesome machine learning interpretability resources

•    

---

A curated, but probably biased and incomplete, list of awesome machine learning interpretability resources. If you want to contribute to this list (and please do!) read over the contribution guidelines, send a pull request, or contact me @jpatrickhall.

fairness xai interpretability iml fatml accountability transparency machine-learning data-science data-mining r awesome awesome-list machine-learning-interpretability interpretable-machine-learning interpretable-ml interpretable-ai interpretable-deep-learning explainable-ml

aequitas - Fairness regulator and rate limiter

•    Erlang

---

aequitas is a fairness regulator for Erlang/OTP and Elixir, with optional rate limiting capabilities. It intends on allowing fair access to limited external resources, like databases and web services, amongst distinct actors.

fairness rate-limiting outlier-detection

mli-resources - Machine Learning Interpretability Resources

•    Jupyter

---

Machine learning algorithms create potentially more accurate models than linear models, but any increase in accuracy over more traditional, better-understood, and more easily explainable techniques is not practical for those who must explain their models to regulators or customers. For many decades, the models created by machine learning algorithms were generally taken to be black-boxes. However, a recent flurry of research has introduced credible techniques for interpreting complex, machine-learned models. Materials presented here illustrate applications or adaptations of these techniques for practicing data scientists. Want to contribute your own examples? Just make a pull request.

machine-learning jupyter-notebooks interpretability data-science data-mining h2o mli xai fatml transparency accountability fairness xgboost

AIF360 - A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models

•    Python

---

The AI Fairness 360 toolkit is an open-source library to help detect and remove bias in machine learning models. The AI Fairness 360 Python package includes a comprehensive set of metrics for datasets and models to test for biases, explanations for these metrics, and algorithms to mitigate bias in datasets and models. The AI Fairness 360 interactive experience provides a gentle introduction to the concepts and capabilities. The tutorials and other notebooks offer a deeper, data scientist-oriented introduction. The complete API is also available.

ai fairness-ai fairness fairness-testing fairness-awareness-model bias-detection bias bias-correction bias-reduction bias-finder artificial-intelligence discrimination ibm-research-ai ibm-research

interpretable_machine_learning_with_python - Practical techniques for interpreting machine learning models

•    Jupyter

---

Monotonicity constraints can turn opaque, complex models into transparent, and potentially regulator-approved models, by ensuring predictions only increase or only decrease for any change in a given input variable. In this notebook, I will demonstrate how to use monotonicity constraints in the popular open source gradient boosting package XGBoost to train a simple, accurate, nonlinear classifier on the UCI credit card default data. Once we have trained a monotonic XGBoost model, we will use partial dependence plots and individual conditional expectation (ICE) plots to investigate the internal mechanisms of the model and to verify its monotonic behavior. Partial dependence plots show us the way machine-learned response functions change based on the values of one or two input variables of interest, while averaging out the effects of all other input variables. ICE plots can be used to create more localized descriptions of model predictions, and ICE plots pair nicely with partial dependence plots. An example of generating regulator mandated reason codes from high fidelity Shapley explanations for any model prediction is also presented. The combination of monotonic XGBoost, partial dependence, ICE, and Shapley explanations is likely the most direct way to create an interpretable machine learning model today.

machine-learning fatml xai gradient-boosting-machine decision-tree data-science fairness interpretable-machine-learning interpretability machine-learning-interpretability iml accountability transparency data-mining interpretable-ml interpretable interpretable-ai lime h2o

kvmScheduling - :alarm_clock: vCPU scheduler & memory coordinator for KVM through Libvirt

•    C

---

This is just a experiment to play with the libvirt APIs and have a benchmarking tool for scheduling problems. A vCPU scheduler which tries to assign the best pCPU to each vCPU, based on fairness. It triggers only after a fixed usage percentage on one of the CPUs (defined in the source).

vcpus fairness schedule vcpu-scheduler libvirt

aequitas - Bias and Fairness Audit Toolkit

•    Python

---

Aequitas is an open-source bias audit toolkit for machine learning developers, analysts, and policymakers to audit machine learning models for discrimination and bias, and to make informed and equitable decisions around developing and deploying predictive risk-assessment tools. Learn more about the project.

fairness bias machine-bias fairness-testing

wasserstein_fairness - An implementation of Wasserstein Fair Classification, a conference paper submitted to UAI 2019

•    Python

---

An implementation of Wasserstein Fair Classification, a conference paper submitted to UAI 2019.

fairness optimal-transport

ensure-loan-fairness-aif360 - A demonstration of the AI Fairness 360 toolbox for bias metrics, explanations, and remediation

•    Jupyter

---

A machine learning model makes predictions of an outcome for a particular instance. (Given an instance of a loan application, predict if the applicant will repay the loan.) The model makes these predictions based on a training dataset, where many other instances (other loan applications) and actual outcomes (whether they repaid) are provided. Thus, a machine learning algorithm will attempt to find patterns, or generalizations, in the training dataset to use when a prediction for a new instance is needed. (For example, one pattern it might discover is "if a person has salary > USD 40K and has outstanding debt < USD 5, they will repay the loan".) In many domains this technique, called supervised machine learning, has worked very well. However, sometimes the patterns that are found may not be desirable or may even be illegal. For example, a loan repay model may determine that age plays a significant role in the prediction of repayment because the training dataset happened to have better repayment for one age group than for another. This raises two problems: 1) the training dataset may not be representative of the true population of people of all age groups, and 2) even if it is representative, it is illegal to base any decision on a applicant's age, regardless of whether this is a good prediction based on historical data.

ai machine-learning fairness open-source ibm

LiFT - The LinkedIn Fairness Toolkit (LiFT) is a Scala/Spark library that enables the measurement of fairness in large scale machine learning workflows

•    Scala

---

As of version 0.2.2, we are only publishing versions to LinkedIn's Artifactory instance rather than Bintray, which is approaching end of life. The LinkedIn Fairness Toolkit (LiFT) is a Scala/Spark library that enables the measurement of fairness and the mitigation of bias in large-scale machine learning workflows. The measurement module includes measuring biases in training data, evaluating fairness metrics for ML models, and detecting statistically significant differences in their performance across different subgroups. It can also be used for ad-hoc fairness analysis. The mitigation part includes a post-processing method for transforming model scores to ensure the so-called equality of opportunity for rankings (in the presence/absence of position bias). This method can be directly applied to the model-generated scores without changing the existing model training pipeline.

machine-learning spark linkedin fairness fairness-ai fairness-ml