captum - Model interpretability and understanding for PyTorch

•    Python

---

Captum is a model interpretability and understanding library for PyTorch. Captum means comprehension in Latin and contains general purpose implementations of integrated gradients, saliency maps, smoothgrad, vargrad and others for PyTorch models. It has quick integration for models built with domain-specific libraries such as torchvision, torchtext, and others. With the increase in model complexity and the resulting lack of transparency, model interpretability methods have become increasingly important. Model understanding is both an active area of research as well as an area of focus for practical applications across industries using machine learning. Captum provides state-of-the-art algorithms, including Integrated Gradients, to provide researchers and developers with an easy way to understand which features are contributing to a model’s output.

interpretability feature-importance interpretable-ai interpretable-ml feature-attribution

awesome-machine-learning-interpretability - A curated list of awesome machine learning interpretability resources

•    

---

A curated, but probably biased and incomplete, list of awesome machine learning interpretability resources. If you want to contribute to this list (and please do!) read over the contribution guidelines, send a pull request, or contact me @jpatrickhall.

fairness xai interpretability iml fatml accountability transparency machine-learning data-science data-mining r awesome awesome-list machine-learning-interpretability interpretable-machine-learning interpretable-ml interpretable-ai interpretable-deep-learning explainable-ml

diabetes_use_case - Sample use case for Xavier AI in Healthcare conference: https://www

•    Jupyter

---

Recent advances enable practitioners to break open machine learning’s “black box”. From machine learning algorithms guiding analytical tests in drug manufacture, to predictive models recommending courses of treatment, to sophisticated software that can read images better than doctors, machine learning has promised a new world of healthcare where algorithms can assist, or even outperform, professionals in consistency and accuracy, saving money and avoiding potentially life-threatening mistakes. But what if your doctor told you that you were sick but could not tell you why? Imagine a hospital that hospitalized and discharged patients but was unable to provide specific justification for these decisions. For decades, this was a roadblock for the adoption of machine learning algorithms in healthcare: they could make data-driven decisions that helped practitioners, payers, and patients, but they couldn’t tell users why those decisions were made.

xgboost healthcare interpretability xai iml transparency machine-learning data-science data-mining machine-learning-interpretability interpretable-ml interpretable-machine-learning explainable-ml

interpretable_machine_learning_with_python - Practical techniques for interpreting machine learning models

•    Jupyter

---

Monotonicity constraints can turn opaque, complex models into transparent, and potentially regulator-approved models, by ensuring predictions only increase or only decrease for any change in a given input variable. In this notebook, I will demonstrate how to use monotonicity constraints in the popular open source gradient boosting package XGBoost to train a simple, accurate, nonlinear classifier on the UCI credit card default data. Once we have trained a monotonic XGBoost model, we will use partial dependence plots and individual conditional expectation (ICE) plots to investigate the internal mechanisms of the model and to verify its monotonic behavior. Partial dependence plots show us the way machine-learned response functions change based on the values of one or two input variables of interest, while averaging out the effects of all other input variables. ICE plots can be used to create more localized descriptions of model predictions, and ICE plots pair nicely with partial dependence plots. An example of generating regulator mandated reason codes from high fidelity Shapley explanations for any model prediction is also presented. The combination of monotonic XGBoost, partial dependence, ICE, and Shapley explanations is likely the most direct way to create an interpretable machine learning model today.

machine-learning fatml xai gradient-boosting-machine decision-tree data-science fairness interpretable-machine-learning interpretability machine-learning-interpretability iml accountability transparency data-mining interpretable-ml interpretable interpretable-ai lime h2o

cav-keras - Concept activation vectors for Keras

•    Python

---

In this package, we allow a user to explore concept activation vectors CAVs in their Keras models. We provide a simple example using CIFAR data.

interpretability explainable-artificial-intelligence interpretable-deep-learning interpretable-ai interpretable-ml explainable-ai explainable-ml interpretable-machine-learning explainability