Earlier this year, the UK Government wrote to regulators, including the ICO, FCA and the Bank of England (incorporating the PRA) asking them to publish details outlining their strategic approach to Artificial Intelligence (AI).
In its response, the Bank referenced a wide range of regulation to evidence its capability to supervise the use of AI in a rapidly evolving environment. However, SS1/23 was the star of the show when it came to the bank's supervisory approach across at least two of five areas highlighted by the government . This makes sense, given that practically all AI-driven applications are based on mathematical models of one type or another. Hence, AI-related risk and model-related risk are invariably linked.
SS1/23 came into effect on May 17th this year. It covers all models used by firms, and by all - we mean all.
The PRA has clarified SS1/23’s scope as covering a firm’s entire operations, rather than just credit and market risk, which for many firms has been the focus of their model risk management in the past. If HR use models to screen job applicants, or if AI-driven chatbots are employed to interact with customers, or generative AI creates personalised advertising content - then these are all within scope.
Is SS1/23 up to the task of supporting such a broad range of models?
The core strength of SS1/23 is it’s principle-based, technology agnostic approach. This makes it as applicable to the most complex large language models imaginable as it is to the simplest rule-based ones. SS1/23 take a top-down approach, focusing on potential risks and outcomes rather than the specific application areas, technological underpinnings or method of operation.
It might then, be safe to assume that one can rest easy, safe in the knowledge that adherence to SS1/23 supports a comprehensive risk management framework that mitigates the dangers AI-based tools present. However, there is one obvious gap, which receives only a passing mention in the Bank’s response to the government.
SS1/23 only applies to firms with IRB permissions for calculating capital requirements. Of more than 1,300 firms regulated by the PRA, only 23 have such permissions. Admittedly, these firms include the UK’s largest banks and building societies, thus covering most UK banking operations. However, as we’ve seen with Google, Amazon and Tesla, new entrants and market disruptors can start small and grow rapidly. There is no regulatory requirement for a newly established FinTech to apply for IRB permissions and there are now fewer incentives for them to do so given the forthcoming implementation of the Basel 3.1 standard. Consequently, the pool of IRB approved firms to which SS1/23 applies is unlikely to materially increase any time soon.
How is the Bank going to ensure that AI-appropriate model risk management principles are applied to the rest of its flock?
Based on the PRA’s previous comments , a pretty sound bet is for the risk management principles described in SS1/23 to be rolled out across the board in the near future. We expect this will be in a simplified form, applied proportionally based on a firm’s size and complexity but we envisage two key impacts:
- All organisations can expect to increase the level of resources applied to model risk management in the future.
- To successfully embed model risk management principles, firms will need to consider their cultural approach to model risk. Model risk should be front and centre, clearly visible to the board and allocated a similar level of scrutiny and oversight as other areas of risk.
The table below outlines how we see this playing out in terms of actions firms will need to undertake to manage AI-based risks (and model risk in general).
SS1/23 Principle |
Actions Required |
|---|---|
|
Principle 1 – Model identification and model risk classification |
|
|
Principle 2 – Governance |
|
|
Principle 3 – Model development, implementation, and use |
|
|
Principle 4 – Independent model validation |
|
|
Principle 5 – Model risk mitigants |
|
In short, SS1/23’s model risk management principles may not yet apply to most PRA regulated firms. However, it’s only a matter of time before they do. Therefore, forward thinking firms should begin planning now for the inevitable changes that are expected in the MRM regulatory landscape.