The Prudential Regulation Authority’s (PRA) recent consultation paper CP6/22 communicates a material increase in expectations on model risk management (MRM).
The consultation paper promotes model risk management as a risk-discipline in its own right and sets five key principles that must be integrated into each firm’s risk management framework. The June 2022 paper includes a draft supervisory statement that is expected to be implemented in Q1 2023.
In this blog, Jaywing use 20+ years of regulatory risk, model building, validation and governance experience to evaluate the MRM principles and their implications for UK firms. Keep reading to discover what you can do now to prepare for the PRA’s upcoming expectations.
New principles for model risk management (MRM)
The new principles are intended to address the shortcomings the PRA has observed in firms’ MRM by providing a consistent set of expectations for all model types. You can read our summary of the new principles in a recent publication in Global Risk Regulator.
Areas of focus for impacted firms
The topics and themes presented in the CP and SS should be familiar at some level to most firms. However, certain expectations, (in particular the broadening scope of MRM to include more model types and the requirement for external models to undergo the same MRM policies as internal models) will require a significant increase in effort to establish a standalone MRM framework.
MRM is already a considerable challenge for firms given the volume of work required to manage model risk effectively, combined with the specialist resource requirements for Independent Model Validation (IMV). All impacted firms must make decisive changes to embed model risk culture in the working practices of both senior management and technical staff.
What firms should do now
Firms must assess their current MRM framework against the principles in the CP and the draft SS. Firms should look to review their current model risk management framework across policy, governance and their operating model to produce a gap analysis and a plan to remediate issues in the time available prior to the expected policy date in Q1 2024. We have identified some of the key challenges, by principle.
Principle 1 - model identification and risk classification
- Review current model inventory against the PRA’s model and model risk definitions – noting that the scope of each firm’s model risk framework may materially increase as it is applied to a broader population of models. Plans for the inclusion of all external models to the inventory should be devised.
- Review the system used to host and manage the firm’s model inventory - the increase in expectations on record keeping and reporting using a consistent firm-wide approach may mean current inventory approaches are unsuitable. The additional information for limitations, deficiencies, and restrictions to be maintained as a live view over time would benefit from a dynamic, systemised solution.
- Review each model captured in the new inventory against the new standards – there is likely to be a significant amount of work to retrospectively bring existing models to the standard that the new SS requires. Beginning this work prior to the implementation date, starting with the most material models and before models require periodic assessment may prevent significant backlogs of actions being raised.
- Consider the difference between the management of ‘models’ and ‘quantitative methods’ that don’t meet the definition of a model – there is an expectation to manage and demonstrate controls of tools/applications that don’t meet the PRA ‘model’ definition. Setting clear expectations on the different approaches will help create boundaries for the MRM framework and support application owners.
- Review the current inventory against the newly defined tiering criteria - the additional criteria for assessing the importance of a model (e.g. complexity and other qualitative factors) may increase the overall level of model risk present in a firm’s framework. The additional risk may require greater effort and resource to manage.
Principle 2 – governance
- Identify the most appropriate senior manager to be accountable – an accountable executive will need to include model risk under their SMF Statement of Responsibilities.
- Review the current MRM operating model – assessment of resource requirements to manage the new expectation means that a plan to hire or train staff can be produced. Significant senior management training and engagement is likely to be required for all firms.
- Review and refine the firm’s approach to setting and measuring risk appetite – model risk appetite is generally inconsistent between firms, but with model risk management considered as a standalone risk by the PRA, this should now be formalised.
- Review the scope of internal audit – audit oversight may historically have been limited to accounting outputs and (in the case of IRB firms) capital requirement outputs but more significant oversight and review of the overall MRM framework will now need audit input.
Principle 3 – model development, implementation, and use
- Focus on the enhanced requirements for AI and ML models – while the SS is not specific on the management of AI or ML applications, the requirements for assessing complexity and unstructured or interconnected data sources means these applications are likely to be classified as highly material and require the highest level of oversight and control.
Principle 4 – independent model validation
- Review the need for IMV to assess model implementation more thoroughly – most firms’ validation activity is focussed on desktop review of model methodology, data, assumptions, and performance monitoring. The new SS increases the focus on assurance over the physical implementation and systems used to support the model’s use.
- Consider how to meet the assurance obligation for external models that are provided or managed by 3rd parties – 3rd party model providers may be unwilling or unable to provide all relevant model elements required for IMV (e.g., data, code) due to concerns over intellectual property or privacy. This makes full assurance as per the SS requirements unachievable. A suitable approach that meets both parties’ needs must be negotiated and implemented.
Principle 5 – model risk mitigants
- Focus on the process for defining, approving, and removing post model adjustments (PMAs) – the process for defining a PMA should be clear and subject to review and challenge through IMV where appropriate.
- Set the expectations that internal model’s may be restricted where MRM requirements are not met – there should be clear provision in MRM policy for how non-compliant models can be restricted while MRM gaps are present.
With these draft regulations expected to be formalised into a new supervisory statement, the impacts on your business should be considered with plans drawn up and resources arranged now to meet the expected Q1 2024 deadline. Enhancing your model risk policies to adhere to the principles of CP6/22 will be the first step in the journey – embedment of these policies will be imperative with the PRA requiring evidence of full deployment.