Following the 2008 recession, economic modelling has increased in prominence in credit risk management, as regulators expect financial institutions to be able to predict the impact of an economic downturn on portfolio losses, to ensure they could withstand another recession. In this blog, we compare the pros and cons of traditional modelling methods with non-linear alternatives to calculating losses.
Increased scrutiny on economic modelling impacts both stress testing and impairment models – particularly as the IFRS 9 accounting standards implemented in January 2018 require financial institutions to incorporate forward looking information and multiple economic scenarios in the calculation of expected losses.
The approach generally adopted in the industry is to include macro-economic drivers in the prediction of Probability of Default (PD) through linear or log-linear Economic Response Models (ERMs). Linear models assume that the impact of a change in an economic driver on losses will be of the same magnitude, regardless of the starting point or direction of movement. This is not realistic, as a downturn is likely to have a more significant negative impact on losses compared to the positive impact expected in recovery times.
The limitations of this approach led to firms having to introduce significant expert-judgment based overlays on model results, and often to maintain separate ERMs for stress testing and IFRS 9 impairment modelling.
Traditional versus non-linear modelling
The background on our test
Based on these premises, we tested the assumption of asymmetric economic effects in the prediction of probability of default on a recessionary versus benign period, comparing a traditional linear model with non-linear alternatives, namely threshold models and a Markov switching regime model.
All the non-linear modelling approaches adopted entail the estimation of different relationships between the variable of interest - in this case an approximation of PD for UK secured mortgages - and a set of economic drivers in a benign economy vs a downturn, here represented by the 2008 recession.
The analysis, developed on industry-level data for mortgages sourced from the Bank of England, supports the hypothesis that non-linearity of macroeconomic effects is a more reasonable assumption, with the magnitude of default rate changes depending on the underlying state of the economy.
This hypothesis should be explored by financial organisations further, in light of ever-increasing regulatory expectations on firms’ modelling capabilities, as a mean to improve accuracy and intuitiveness of the size and timing of losses forecast under a stress scenario.
Assumption
Linear models assume a linear impact of changes in the economic drivers on losses. For example, a 1% increase in interest rates will predict default rates to increase by the same amount as a 1% decrease would reduce them.
In reality, an asymmetric response of defaults and losses to economic changes is more likely, with a more severe impact expected under worsening conditions. In the example above, an increase of 1% in interest rate should have a more significant negative impact on losses compared to the benefits of a 1% interest rates reduction.
The non-linearity assumption is supported by data over the 2008 recession. The following chart represents the relationship between GDP quarterly changes* and BOE secured write off rates ** in the period 2008-2012. It is evident how the contraction in GDP experienced as a result of the economic crisis in 2008 had a bigger negative impact on secured write-off rates (measured as higher peaks) than the write-offs reduction experienced in the recovery period.
Models explored
The analysis has been developed on the secured write-off rate sourced by the Bank of England for the period March 1994 to December 2018, shifted two quarters to approximate probability of default trends. There are some limitations in using industry-level data - not capturing change in quality, maturation effects etc.- but at this stage the results are to be intended as a proof of concept on the overall dynamic of losses.
To investigate the hypothesis, a simple linear model has been estimated and compared with several non-linear alternatives, all belonging to the category of threshold models, specifically a simple Threshold Model (TM) approach vs a Threshold Autoregressive Model (TAR) and a Markov Switching Regime (MSR) model.
The theory behind threshold models is that the default rate may behave differently, and therefore a different model would apply, depending on the value of a variable exceeding a pre-determined threshold. In simple form:
The Threshold is a trigger that prompts the estimation of an alternative model. In Threshold Models it is set based on the exogenous variable(s), such as GDP in the example above, while in Threshold Autoregressive Models it is set depending on the dependent variable – the (shifted) write-off rate in this example – crossing a certain threshold in the previous time period.
Both approaches are extremely sensitive to the way the threshold is set, which requires a good understanding of the data available and of the interactions between economic drivers. In Markov Switching Regime models the threshold is not pre-defined, but instead the probability of being in a certain state (e.g. benign vs recession) is estimated on the default/loss data analysed.
The comparison between modelling approaches (linear vs TR vs TAR vs MSR) was done assessing both goodness of fit, in particular over the 2008 Recession, and response to a stress scenario.
Goodness of fit
As expected, threshold models present a better fit of traditional linear models, driven by the higher responsiveness during the 2008 recession (TAR not included in the chart as marginally different in this example):
The following table summarise the R-Squared goodness of fit metric measured for the different model options during the 2008 recessionary period, which confirms the reading from the chart of a superior fitting delivered by non-linear model approaches, with TR and TAR models delivering the strongest fit.
|
Model |
R-Squared (over 2008 Recession) |
|
Linear |
85% |
|
TR/TAR |
95% |
|
MSR |
91% |
The main concern around threshold models, specifically in case such as this where they are estimated including only one recessionary period, is that there may be an excessive influence on the results of the dynamics and relationship between macroeconomic variables experienced over the one recession period analysed.
This calibration of the variables’ coefficients on specific sections of the data - in this case particularly the 2008 recession peak - increases the risk of overfitting - i.e. tailoring the model to the available data thereby limiting its capability to retain accuracy in different samples.
In absence of additional recessionary data to use in development or as validation, MSR models represent an interesting alternative. Despite implying a slightly more sophisticated approach and higher implementation effort, MSR models can overcome some of the limitations of the other methodologies, in particular overfitting, as the parameters identifying the benign vs recessionary period are estimated on the loss data and not set based on the dynamics of the macroeconomic variables.
Stress response
In terms of forecasting performance, the improved fit over the 2008 recession of non-linear models leads to higher response to stress, a limitation often seen when applying stress scenarios on IFRS 9 PD models***:
Threshold Models present the strongest response, while the default reaction driven by the MSR model is somewhere in between linear and threshold models.
The results, developed as an exercise on industry-level data, suggest stronger fit and better performance for non-linear models, but as any modelling exercise, their use in an operational environment should be supported by credible validation, expert challenge - particularly when limited validation data is available - and robust governance.
Conclusions
While there is a certain level of expert judgment to be applied, depending on individual portfolio characteristics, our analysis conducted on industry-level data supports the idea of asymmetric economic effects on portfolio losses.
Financial institutions should consider exploring this option in the next generation of Economic Response Models, for different reasons. Firstly, regulators expect firms to continuously refine their forecasting capabilities, and would look favourably at a more robust and scientific approach to estimating the impact of economic scenarios on losses, both in impairment calculation and stress testing.
Additionally, the improved fit provided by non-linear economic models would reduce the requirement to deploy significant economic overlays satisfying the recommendations from the recent PRA review and, as models would be more reactive under stress, facilitate the integration between IFRS 9 and stress testing models.
If you would like to explore the best approach for your loss forecasting, you can get in touch with our risk experts by emailing [email protected] or calling 0333 370 6600.
*source Office for National Statistics, series ‘YBHA’, “Gross Domestic Product at market prices: Current price: Seasonally adjusted £m”
**source Bank of England Interactive database, series ‘LPQVTYI’ (UK secured write-offs) and ‘RPQB7YW’ (UK secured exposure), combined with internal calculations
*** Models forecast tested on Bank of England 2019 CST Stress scenario