Risk predictions for 2020: 2) Bolstered model risk practices
Discover risk best practice at our UK Finance CRO Network events
Risk predictions for 2020: 1) A surge in Machine Learning adoption
29 January 2020
Now is the time to assess budgets and plans for the future. And this year, the focus is on growth. Here’s one of our top predictions.
This year, we anticipate a surge in the number of lenders adopting modern Machine Learning techniques to produce the powerful credit scores they crave.
The credit scoring industry was one of the earliest adopters of Machine Learning, and has deployed it successfully for 30+ years to automate decision making. Machine Learning techniques have moved on since then, with more powerful approaches emerging, but in view of their “black box” nature the industry has taken a responsible approach to adoption, preferring to stick with more traditional approaches which yield more explainable decisions.
In the last few years though, approaches to “Explainable AI” have been developed that now make responsible adoption of ML models a possibility, and we developed our industry-leading Archetype product specifically to solve this. Regulators have also provided welcome guidance in the last year around what practices are and are not appropriate in this regard.
In 2019, we ran over 20 head-to-head tests of our approach finding that it consistently generates 5-20% uplifts in predictive power – typically amounting to a multi-million pound business case with a very strong ROI. We’ve also worked with a number of forward-thinking early-adopters to put these models into production - in a way that’s consistent with best practices outlined by regulators. We expect that over the course of next year, more lenders will follow suit – keen not to be left behind as the industry moves forward.
Views from the market
AI and ML have been at the helm of conference discussions and talks across the industry for quite some time. Fintech Futures recently surveyed lenders to understand what they believe are the main benefits to this technology, and whether it would really shake up the industry in the short to medium term. The potential to improve data analytics (86%) and increased productivity (77%) were seen by respondents as the greatest benefits to the deployment of an AI or ML solution.
When asked whether AI and ML would become a differentiator for financial services firms by 2024, more than two-thirds (69%) of those asked agreed or strongly agreed that it would. And more than three-quarters (83%) of respondents said that the deployment and development of AI and ML systems would become a core part of their business strategy going forward.
What should a lender do?
Advanced, explainable AI approaches present a massive opportunity to improve credit scoring, yielding outcomes that are better for customers and more profitable for lenders. But it’s essential that AI is adopted responsibly. We recommend reading the guidance provided by the FCA and ICO.
Key outputs from recent regulatory guidance
In summary, the recent ICO paper states that within processes associated with credit scoring, these approaches cannot be used without due thought and consideration as to their impacts, and an ability to prevent unwanted outcomes - there should be a high bar associated with their use where there is a high impact on the consumer. Relying on post-hoc explanations like LIME or SHAP is not sufficient.
Here’s three key characteristics of regression-based models that allow for optimal explainability and transparency:
Any change in the value of the predictor variable is directly reflected in a change in the value of the response variable at ‘a’ constant rate ‘b’. The interpretable prediction yielded by the model can therefore be directly inferred from the relative significance of the parameter/weights of the predictor variable and have high inferential clarity and strength.
When the value of the predictor changes in a given direction, the value of the response variable changes consistently either in the same or opposite direction. The interpretable prediction yielded by the model can therefore be directly inferred. This monotonicity dimension is a highly desirable interpretability condition of predictive models in many heavily regulated sectors, because it incorporates reasonable expectations about the consistent application of sector specific selection constraints into automated decision-making systems.
The number of features (dimensionality) and feature interactions is low enough and the model of the underlying distribution is simple enough to enable a clear understanding of the function of each part of the model in relation to its outcome.
There are tools to enable you to more seamlessly adopt Machine Learning, for example Archetype. You can see how these early adopters have trialed AI and seen uplifts in model performance of around 18%*: Secure Trust, Virgin Money, Hitachi Capital, Nationwide, Newcastle Building Society.
What are the benefits of adopting it?
- Typically we see a 5-20% uplift in predictive power through use of these techniques
- Extremely strong benefits case for adoption, with very strong ROI
- Better customer outcomes!
What are the risks of not adopting it?
- Early adopters are already benefiting from adoption of these techniques, giving them an advantage in an extremely competitive lending market.
For support with adopting Machine Learning and AI-based approaches, get in touch.