Machine learning in credit granting: an analysis of the risks

What it is about

An analysis of the risks of using machine learning models in credit granting.

Session summary

The speaker, a financial innovation researcher at the Bank of Spain, picks up from the previous webinar on the predictive advantages of machine learning in credit and focuses on the other side: greater algorithmic complexity and the new model risk. He proposes a framework that groups risk factors into three families (statistics, technology and market conduct), whose weight changes with the model’s use: lending, regulatory capital or provisions. He summarizes that, compared with parametric models and neural networks, tree-based models such as random forests and gradient boosting seem to offer a good balance between predictive power and complexity.

He devotes most of the session to interpretability. He explains why people and regulation, such as US consumer protection rules, the European data protection regulation and the European artificial intelligence act, require credit decisions to be explained, and reviews the techniques for opening ‘black box’ models: surrogate models, permutation importance, partial dependence plots, SHAP and LIME. He presents his research with synthetic data, according to which these techniques do not always reflect the underlying reality, above all when variables are highly correlated. The questions covered auditability, the acceptable margin of error in interpretability, decision trees, combining models, data quality and the cost of mistakes.

Topics covered

• Algorithmic complexity and model risk in machine learning
• Three risk families: statistics, technology and market conduct
• Why regulation requires credit decisions to be explained
• Interpretability techniques: permutation, partial dependence, SHAP and LIME
• Limits of explanations: correlation between variables and synthetic data
• Model auditability and a probabilistic world for supervision
• Questions: decision trees, combining models, data and the cost of errors