Credit portfolio default in the Dominican Republic: a predictive machine learning approach

What it is about

A study that identifies the variables with the greatest impact on predicting a borrower’s default in the Dominican financial system. It selects 39 variables inspired by the default prediction literature and a machine learning model called CatBoost, and uses SHAP values to assess the interaction between variables and identify the relevance of each.

The model fit, measured by the area under the ROC curve, was 86%, with an F1 score of 50%.

Session summary

The speaker, a data scientist and researcher at the Superintendency of Banks, presents a study that predicts default of consumer-credit borrowers twelve months ahead using credit-bureau information and borrower characteristics, with a gradient boosting model and 39 variables. He explains the methodology: the definition of default (90 days past due or restructuring), hyperparameter tuning, cross-validation, the split between training and test sets, and why metrics such as the F1 score and the area under the ROC curve are better than accuracy when defaults are rare. He clarifies that the views are the author’s and not necessarily the institution’s.

To open the ‘black box’ he uses SHAP values, which show how much each variable contributes to each borrower’s prediction, with examples of true and false positives and negatives. Among the most influential variables are credit card utilization, maximum delinquency in the last year, borrower seniority, monthly spending (with an effect that runs against intuition, which he reads as a signal of income), months since the last disbursement, gender, income and job stability. He shows how the pandemic altered the predictive power of some variables and points to the lack of transactional variables as a limitation. The questions covered the F1 metric, variable selection, categorization and the decision to keep counterintuitive variables.

Topics covered

• Predicting default in consumer credit with gradient boosting
• Definition of default and model validation
• Metrics with imbalanced classes: F1 and area under the ROC curve
• SHAP values to interpret the model borrower by borrower
• Most influential variables in the Dominican market
• Effect of the pandemic on predictive power
• Questions: variable selection and categorization, and counterintuitive variables