Over the past few years, many lenders have been busy preparing for the implementation of IFRS 9. In preparing for the new standards, lenders may have dedicated most of their time to technical and methodological issues.
But as we move into the parallel run phase, we encourage lenders to understand how best to monitor IFRS 9 models to ensure they remain robust over time.
Monitoring the models
A typical lender uses over 50 analytical models to support its underwriting, pricing and credit risk functions, and will have developed monitoring suites to support the maintenance and use of these models.
While the IFRS 9 models may appear quite different from more traditional models, the objectives from any monitoring are broadly similar:
- Is the power of the model maintained?
- Do the model predictions match the actual observed?
- Do actual and predicted align across the different levels of both model variables and non-model variables?
Monitoring is almost a repeat of the original model development, with appropriate samples created in a similar fashion, but with the additional data points from the last few time periods extending the available sample.
The interpretation of these reports informs what changes or adjustments are required to improve the models – for example, whether parameters need re-estimating, variables need to be removed or different variables need to be included, or a complete model restructure is necessary.
When the economic environment starts to move, this is a particularly critical time to ascertain whether the macroeconomic factors included in the models, and their corresponding lags, remain appropriate. Monitoring should be executed on a regular basis – the recommendation is every quarter.
Monitoring the forecasts
Monitoring the ECL forecasts presents some unique difficulties, namely from the required time lapse, and the use of scenarios. The time taken to fully realise the actual losses means that you can only truly evaluate the performance of forecasts made a number of years ago; this appraisal is not always going to be of much interest.
Lenders will typically make several different forecasts based on different macroeconomic scenarios. By design, only one of these predictions can be right, and so the identification of a ‘wrong’ forecasted scenario should not be seen as an issue.
Our recommendation is to select the aspects of the forecast that can be evaluated without bias. All previous scenarios should be monitored, to give an insight into varying outcome lengths.
The importance of this evaluation should not be under-estimated; just because model components appear to perform well on historical development samples does not guarantee that they will work accurately when applied to make a forward-looking prediction.
- Forward-looking PD. Since these models are predicting a clear outcome at each time period, it is possible to compare without bias the predictions with the actual outcomes, even when the elapsed time since the forecast was made is small.
- Forward-looking EAD. As with the PD, the models are predicting a clear outcome and therefore actual and predicted can always be compared regardless of the length of the outcome window.
- Forward-looking LGD. Since the time from default to the loss being realised can vary, this means that a direct comparison and actual and predicted will be biased. One solution is to understand the distributions of the component models of the LGD, for example, to understand how the proportion of cases reaching possession or write-off builds up by time from default. This then allows loss estimates for monitoring purposes to be adjusted depending on the elapsed time since the forecast, and unbiased comparisons to be subsequently made.
With all monitoring, it is important to clarify what actions, if any, will be made if any pre-determined triggers are breached. Data management is also critical. It should always be possible to execute a historic scenario and replicate the original results exactly. The storage of historic models and scenarios is therefore important, as is the structure and populating of datasets, for example, fields should never simply be over-ridden.
In our experience, the essential elements of IFRS 9 monitoring can be created easily. Risk leaders can take these steps quickly, and senior managers should not hesitate to pull critical experts into the exercise for a few weeks – it’s typically a worthwhile investment that pays off, easing the pressure off a much larger body of work.
For guidance on IFRS 9 model monitoring, read our latest blog.
Jaywing are experts in IFRS 9 and regulatory modelling and has developed a toolkit to support lenders through the unique requirements for IFRS 9 monitoring.