“United States banking stability: An explanation through machine learning”

In this paper, an analysis of the prediction of bank stability in the United States from 1990 to 2017 is carried out, using bank solvency, delinquency and an ad hoc bank stability indicator as variables to measure said stability. Different machine learning assembly models have been used in the study, a random forest is developed because it is the most accurate of all those tested. Another novel element of the work is the use of partial dependency graphs (PDP) and individual conditional expectation curves (ICES) to interpret the results that allow observing for specific values how the banking variables vary, when the macro-financial variables vary. It is concluded that the most determining variables to predict bank solvency in the United States are interest rates, specifically the mortgage rate and the 5 and 10-year interest rates of treasury bonds, reducing solvency as these rates increase. For delin- quency, the most important variable is the unemployment rate in the forecast. The financial stability index is made up of the normalized difference between the two fac- tors obtained, one for solvency and the other for delinquency. The index prediction concludes that stability worsens as BBB corporate yield increases. in the interpretation of the results of complex machine learning models. In this work, a new approach to bank stability is proposed, creating models that allow predicting bank variables. With these models, the work also addresses the problem of generating reference values for macro-financial variables that can be applied to create bank stress test scenarios.


INTRODUCTION
As a result of the financial crises, with special emphasis on the 2008 crisis with its genesis in the United States, studies on banking stability have been expanded. Being able to consider said stability as a public good is due to the importance it plays in the proper functioning of the economy and due to the negative externalities that cause its lack.
There are many methods that try to address the problem of measuring said stability, highlighting early risk detection methods and stress tests that measure the reaction of banking variables to supposed scenarios, as well as the creating indices that synthesize banking stability. However, different problems are detected in the measurement of bank stability both in its interpretation and in the macro-financial variables that may have a greater influence on it, as well as in the interpretation of the results of complex machine learning models.
In this work, a new approach to bank stability is proposed, creating models that allow predicting bank variables. With these models, the work also addresses the problem of generating reference values for macro-financial variables that can be applied to create bank stress test scenarios.

LITERATURE REVIEW
In the analysis of the state of the art, three aspects are combined that are useful to understand the development of the work and represent the key benefits. First, the development of scenarios on the stress test bench, second, bank stability and the ways to achieve it, and finally, the application of a methodology does not apply to the field of finance.
It should be noted that financial stability is an objective currently pursued by various central banks, and as Criste and Lupu (2014) emphasize, central banks must guarantee a balance between maintaining price stability, as their main objective, and promoting financial stability, which is a more general objective. Jhan and Kick (2012) point out that the periodic assessment of financial stability and the identification of early warning indicators that signal future risks for the banking system are imperative tasks for central banks and supervisory authorities.
Banking stability is of vital importance for proper economic functioning. The level of resistance of the banking system to the macroeconomic impact is analyzed in a very broad way with the so-called stress test. This requires the development of macroeconomic scenarios to understand the relationship with bank stability. Among the most important bank stability variables, Budnick et al. (2019) highlight the importance of bank capital, the quality of bank assets for the propagation of shocks to the financial sector and the real economy, to perform macroprudential stress tests. In this paper, these variables are evaluated to determine the stability of the US banking system.
However, there are critics with the analysis of stress tests such as Borio et al. (2012), who argue, with current technology, that macro stress tests are not suitable as early warning devices, that is, as tools to identify vulnerabilities in seemingly quiet moments to trigger corrective actions. They argue that most models are estimated using linear logarithmic relationships, when in fact such relationships do not occur. At work, these problems are solved through the methodology used, which considers non-linear relationships.
Another problem that may exist is that analyzed by Bookstaber et al. (2013), who explain how supervisors who depend on a small and fixed set of scenar-ios can make banks learn to anticipate the characteristics of a scenario, thus reducing the effectiveness of tests. That is why he advocates reverse stress tests. Reverse stress tests start with a result and look for scenarios that produce that result (traditional stress tests assign a scenario to a result). This approach is very similar to that applied in the studio. From the scenarios of low solvency or high delinquency values, scenarios can be deduced according to the most important variables. Supporting the approach to the study of historical series, Onder et al. (2015) perform a macro stress test in the Turkish banking sector and highlight how in the most recent and significant crisis situations they can also be taken directly as a scenario.
Among the studies looking for macro-financial channels is the study of Dua and Hema (2018), who use scenario analysis to capture the impact of macroeconomic stress on the stability of Indian banks through the evaluation of financial strength indicators, such as credit quality and capital adequacy, to find macro-financial channels. The results obtained by the previous authors indicate a cointegrating relationship between credit quality and different macroeconomic variables such as the growth rate of the product and the interest rate. On the other hand, Jhan and Kick (2012) identify important macroprudential early warning indicators such as asset price indicators and money market indicators, applying panel regression techniques. The indicator proposed by the previous authors consists of three components: the standardized probability of default, a credit margin and a stock index for the banking sector. One of the study's ambitions is also to create a bank stability index for which different variables are used, including prudential ones. Creel et al. (2015) also construct a statistical index of financial stability with a principal component analysis, based on several aggregated prudential variables, but with the intention of evaluating the depth of the EU financial system. They show that the effect of delinquent bank loans is detrimental to economic performance, as is macroeconomic financial instability.
There are several studies examining banking crisis forecasting, such as Bussiere and Fratzscher (2006), Dredhman and Juselius (2013), Alessi et al. (2015) and Demirgüc-Kunt and Detragiache (1998). In this last work, the probability of crisis is identified using a logit model, highlighting low real economic growth, high interest rates and inflation as the most important variables. According to Boyd et al. (2001), inflation is a variable that can also damage bank stability; a state that lowers real returns caused by high inflation can reduce the availability of credit and attract additional lower quality borrowers to the pool of loan applicants. The combination of the decrease in the availability of funds and the erosion in the quality of the group of borrowers results in an increase in the severity of frictions in the credit market.
On the other hand, Borio and Lowe (2002), through compound indicators, stipulate that the common use of credit with respect to GDP, gross fixed investment and real estate prices are among the best indicators to predict banking crises. For this last variable, Pang and Wang (2013) consider in their study two indicators of the price of housing; the first changes in house prices and the second, deviations of house prices from long-term equilibrium. The results suggest that the existence of an income growth threshold affects the relationship between house prices and bank instability. In the present study, it should be noted that a dual classification is not made between crisis and non-crisis; this provides a broader view of the behavior of banking variables and macroeconomic variables.
Finally, the models applied in the work allow improving the prediction of different banking variables and approaching banking stability from a new perspective. The PDP and ICES curves are used to interpret the results, however, their use has not been found in the field of economics. Zhao and Hastie (2019) and Apley and Zhu (2019) demonstrate how machine learning algorithms often function as predictive black box models. It is possible to extract causal information from these models using partial dependency (PDP) and individual conditional expectation (ICE) plots. They also highlight the usefulness of these curves.

AIMS
The aim of this work is to study the banking stability of the United States by predicting different banking variables with machine learning models 1 The strengths and weaknesses of the regression trees are developed by Loh (2011), and the algorithm is developed by Breiman et al. (1984). and performing the sensitivity analysis of the results using the PDP and ICES curves.

METHODS
The study begins with empirical analysis using a regression tree 1 , which is a type of machine learning method for building prediction models from data. As advantages, they present an easy interpretation and offer robustness to extreme values. Regression trees also allow us to capture linear and nonlinear relationships, indicating when there may be correlation between the variables. As for problems, this methodology has a very high variance, that is, a small change in the data can cause different partitions of the data. This fact will be corrected using different machine learning ensemble methods.
In prediction trees, like all statistical models, the balance between bias and variance must be considered. In prediction tree models with many nodes, there is a tendency to adapt extremely well to the training sample, but at the cost of greater variance. Assembly methods used in this work seek a balance between bias and variance, using a combination of predictive models.
In the bagging model, one of the models used, several trees are adjusted simultaneously, all providing their prediction. The average of the predictions is taken as the final value.
In the boosting model, multiple simple models are sequentially adjusted, each new model learns from the mistakes of the previous one. As a final value, the average of the predictions is taken.
In the bagging model, there is a lot of variance, but little bias; on the contrary, in the boosting models, there is a lot of bias and very little variance. Adjusting many models in both methods can reduce variance in the first and reduce bias in the second.
With the bagging model, repeated sampling is performed. With the different samples of the population, a model is adjusted, and the result is averaged by reducing the variance. For this, the bootstrapping model is used, generating different samples by resampling. With each of these samples, a tree is made, which is not pruned, having a reduced bias but a greater variance. The algorithm stop system is the minimum number of observations that the end nodes must have. Bootstrapping is done using two thirds of the sample, for the remaining third (out-of-bag) the response variable 'xi' can be predicted, using the trees in which that variable has not been used. Performing this process, the predictions are obtained for the rest of the 'n' observations, and by calculating the average, the mean square error can be obtained, which serves as an estimate of the test-error.
The Random Forest method 2 , is a modification of the bagging model by mitigating the correlation between trees. This is achieved by selecting the predictors randomly, avoiding a very influential predictor dominating in the construction of trees.
Boosting is another assembly technique that is based on sequentially adjusting weak learners. Each tree built depends on the previous one, in the first tree all the predictors are used, and then the weights of the predictors are manipulated, so that later, in each iteration, the weight of the elements poorly classified by that predictor increases.
Factor analysis is also used in the work to reduce dimensions; this technique is used to obtain the bank stability index.
To interpret the results, the PDP and ICES curves are used that show the variation of the predictions of a particular machine learning model.
x c sc sc c Breiman (2001) develops the advantages of random forest.
Each subset of S predictors has its own partial dependency function f S , which provides an average value of f when x S is fixed and x C varies within its marginal distribution dP (x C ). Since neither the true f nor dP (x C ) is known, equation (1) is estimated as follows: where {x C1 , ..., x CN } represent the different values of xc observed in the training data. The true value is estimated with f, which is the true value of the result of the algorithm. The integral over x C is also estimated by the average over the N x C values observed in the training sample.
Unlike the PDP curves, the Individual Conditional Expectation curves allow evaluating the variation for each of the observations. These curves also allow differentiating whether the relationship is additive or influenced by other variables. The ICE curves disaggregate the output of the PDP functions. Instead of plotting the average partial effect of the target covariates, in the predicted response, the estimated N conditional expectation curves are plotted. Each reflects the predicted response as a function of the covariable x S , due to the observed x C . Therefore, the ICE curves, as stated by Goldstein et al. (2014), update PDP.
The c-ICE graphs are obtained in the same way as the ICE graphs, with the difference that for each of the curves, the predicted value is subtracted for the observed minimum of the predictor. This allows all curves to have their origin at 0.
One way to identify the interactions between predictors is by representing the partial derivatives of the ICE (d-ICE) curves. In the absence of interaction, all curves are approximately parallel. The derivatives are approximately equal and the graph of derivatives is represented by a line. In the case of interactions, the representation is heterogeneous.

RESULTS
The study is conducted for the period from the second four-month period of 1990 to the fourth four-month period of 2017. For this, Table 1 shows macroeconomic variables. Macroeconomic variables are collected from the Federal Reserve, the bank solvency and delinquency of the FDIC of all banks adhered to it. Ultimately, the newly created bank stability indicator is examined. The solvency measure is used to assess the level of regulatory capital among risk-weighted assets.
This measure is chosen to assess bank solvency because it is the one used at the regulatory level in capital requirements, also because it is an average that adjusts to risk. It is recognized that the denominator is the risk-weighted assets that measure risk, as is well known in the system. To measure delinquency, the percentage of failed loans is chosen because it is a measure that faithfully reflects the final delinquency. The construction of the bank stability index is explained in section 4.3.
Next, the variables used as dependents are shown in Table 1. The variables are mostly used by the US authorities to perform stress tests. At the same time, variables attempting to represent the debt in the US economy, from the consumer and business perspective (e17 and e18), are added. In addition to a measure of risk of repayment of debts, such as the debt payment service on net income (e19), which is not represented in the stress test scenarios in the United States, it is intended to introduce the degree of leverage of the economy. With regard to the rest of the variables, they can be subdivided into economic variables (e1, e2, e3, e4 and e5), evolution and risk variables of the stock market (e13 and e16)., and variables of price growth in different markets (e6, e14, and e15). Finally, there is a block of variables that can be grouped into different interest rates and performance (e7, e8, e9, e10, e11 and e12).  where is the prediction for the observation obtained by averaging the individual tree predictions for which this observation was excluded from the training sample, and is the real value of the response variable.
Given that the average square error with the Random Forest is 0.03692717, which is equivalent to predictions departing from the real values, there are on average 0.1921 units for solvency and 0.02103 or 0.1450 units for the default. Therefore, a Random Forest model is chosen for the analysis, as this is the model that achieves the best results for both predicted variables.

Random forest results
To calculate the importance of the predictors, an increase in the MSE and an increase in the purity of the nodes are used. The increase of the MSE identifies the influence of each predictor in the MSE of the model estimated by the out of bag error. After this, for each variable X in each tree t, the difference between the two measures, MSE OOB (Xj permuted) and MSE OOB, is calculated. For each variable, this difference is added in all trees, averaged and normalized between the standard deviation of the differences. The result of this process is the measure of the importance of each variable. If the predictor that is not included provides information about the model, the MSE will increase.
An increase in the purity of the nodes is calculated by the decrease of the MSE, which is calculated as the average decrease achieved as a group of the trees that form the assembly. Therefore, the higher the value, the greater the contribution of the predictor to the model (see Figure  2a). It follows that the most important variables for the model are e11, e9, e8, e10, e18 and e12. It is determined as a block of interest rates and performance of BBB bonds, together with an economic variable that is the unemployment rate and a debt variable that is the percentage of household debt over GDP, which are the most determining variables to predict solvency, as defined.
Specifically, the mortgage rate, along with the interest rate on the state's 5 and 10-year bonds, appear as the three most important variables to predict bank solvency. Another way to assess the importance is to compare the times that the variable appears as the root of the tree (see Figure 2b), versus the minimum average depth of each variable in each tree. Henceforward, the most important variables for the model are e11, e9, and e8.

Variables for the delinquency
For default, the most important variable is e5, the unemployment rate (see Figures 3a and 3b).

Bank stability index
The bank stability index is constructed from 15 bank variables, which represent ratios of the banking industry (see Appendix A), and these are collected from the FDIC. Principal component factorial analysis is applied to these variables, which can be used to reduce the variables to two factors. The variance explained is more than 80% of the data, and the adjustment represented by KMO of 0,776 ensures a good fit of the model (see Appendix A). Factor loads, after performing a varimax rotation, show how the variables are grouped into the two factors (see Appendix A). The first factor is called delinquency because the variables that mainly bind delinquent situations should indicate how profitability is negatively correlated with this factor, as well as with the loan provision coverage. The second factor, called solvency, in addition to grouping different variables of accounting and regulatory solvency, negatively groups the cost of financing, which affects solvency.
The indicator is formed by the difference of both factors, the greater the value, the greater the stability, and vice versa ( Figure 4a shows the factors and their differences). The indicator is normalized by subtracting each value by the lowest value and dividing by the difference between the highest and lowest values (Figure 4b).
Next, the same techniques as for solvency and delinquency are applied to predict the behavior of the index. Again, the best results are obtained with the Random forest, the lowest MSE is obtained (see Figure 5).  1990Q2  1992Q1  1993Q4  1995Q3  1997Q2  1999Q1  2000Q4  2002Q3  2004Q2  2006Q1  2007Q4  2009Q3  2011Q2  2013Q1 2014Q4 2016Q3 Index Figure 6 shows the most important variables for predicting the Random Forest, in this case, the most important variable is e10, which represents the BBB Corporate Yield.

DISCUSSION
This section interprets the predictions of the model using the graphs shown below. The interpretation of the variables that were previously justified is made because they represent the most important ones for determining the prediction in the Random Forest model. It begins with the ICES (Individual Conditional Expectation) curves.

Solvency and delinquency interpretation
It is clearly observed that with e11 (see Figure 7), which represents the mortgage rate, bank solvency increases. By sections, it is observed that for the values of 5% they remain high, but from 5% they begin to decrease, seriously worsening with values of 8.5%. It is clear that the C_ICE graph centered shows that as the variable e11 falls, the solvency decreases in the model. On the other hand, in the graph of the partial derivative curves (d-ICE plot), it is observed that in the model there is a great interaction for values between 4.7% of e11 and 6%, as well as for values from 8% and above. The standard deviation of the partial derivatives at each point is shown in the lower area of the graph, observing the areas with high heterogeneity.
This leads to the fact that there are two breakpoints in solvency, in this model, in particular, the point of 5%, in which there is a great interaction and a great drop in solvency from this value and the next one from approximately 8% in which there is a great drop in solvency again. Figure 8 for e9, which represents 10-Year Treasury Yield, shows similar behavior to e11, but experiences two inflection points in the model's solvency behavior, first at 3% of the value for which it is experienced a significant drop in solvency according to the model, and second, for 7%, where it is observed that the aggregate bank solvency decreases from these values. However, it is distinguished how it remains stable for the model between these values. It is also observed that around the commented values, there are interactions with other variables that are seen in the d-ICE plot, detecting the increase in the standard deviations for these points of the partial derivatives. In Figure 9, the variable e8 (5-Year Treasury Yield) presents a decreasing relationship with solvency by tranches, two breakpoints are presented, the first over 2.5% and the next over 7.5%. In these two types of interest, there is a greater interaction with the variables represented by the standard deviation of the d-ICE plot. Between 2.5% and 7%, the solvency in the model remains constant in most cases.
PDP curves are used to evaluate the interaction, and as the model predicts for two variables, in this case, there is an interaction of both curves of each two predictive variables for predicting solvency. Figure 10a shows   As the mortgage interest rate increases, the model predicts lower solvency; this is due to an increase in risk-weighted assets. As a result, the solvency ratio decreases. Although increases in the regulatory capital base may occur, an increase in risk-weighted assets is generally greater than regulatory capital. Also, as the yields on the 10-year bonds increase, the mortgage interest rates decrease the solvency in the system. Both are supposed to move the same way, because when the interest rate is higher than the demand for bonds, it is lower and hence the economy is worse.
For treasury bonds, when there is a lot of demand, investors bid at face value or above. In this case, the return is low because investors will get a lower return on their investment. In any case, they are willing to accept a low yield, in exchange for a lower risk, therefore, during the economic contraction phases the yield rates of treasury bonds tend to fall. Thus, bank interest rates will be reduced, providing liquidity when the economy needs it.
When there is a bull market or the economy is in the expansion phase of the economic cycle, there is a precept of less risk, and investors are looking for more profitability than would be granted by treasury bonds. As a result, there is not much demand, and investors are willing to pay less than the nominal value. When that happens, the return is higher, and Treasury bonds are sold at a discount, so there is a higher return on investment. It can be seen how these rates largely determine the state of the economy, and, therefore, the risk appetite of investors.
As for the solvency coefficient, it is observed that a greater risk appetite determines more risky investments, since so many investors decide to make more risky investments, and the solvency coefficient falls.
Regarding the mortgage rate, it is usually high when economic conditions are flattering, the stock market is rising, and unemployment is low, economic growth is taking place.
In the predictions, it is observed how the model for high values of said variables produces low solvency values, on the contrary, for low values of said variables high values of the solvency variable are predicted.
It can be said that in the moments of greater economic growth and less risk perception in the economy, there is the greatest weakness in Figure 10a. Prediction of the forest (e8, e11) Figure 10b. Prediction of the forest (e9, e11) the banking system. Conversely, when there is more weakness in the economy, marked by the variables described above, the solvency is stronger.
For delinquency, the interpretation of the most important variable, which is the unemployment rate, is e5. Figure 11 shows the relationship between delinquency and unemployment. It is observed that for unemployment values above 6%, delinquency experiences a strong increase. Between 6% and 7%, there is the greatest interaction with other variables (Figure 11c).

Interpretation of results for the financial stability index
To interpret the random forest, the partial dependency functions are applied to predict the model and observe how the model behaves when predicting the bank stability index, compared to the most important variables in the prediction of the said index.

CONCLUSION
Among the machine learning models used, the random forest is the most accurate. As a novelty, the ICES and PDP curves can be used to take full advantage of the predictive and interpretive benefits claimed by the regulatory and supervisory authorities, taking precedence over other more traditional methodologies. Specifically, the mortgage rate, together with the interest rate of the 5 and 10-year government bonds, appear as the three most important variables to predict bank solvency.
For delinquency, the most relevant variable in the prediction is the unemployment rate, concluding that for unemployment rate values between 6% and 7%, there is a significant jump in delinquency in the model.
The interpretation of bank stability with greater amplitude is achieved by constructing a new index, through the difference between two factors that represent solvency and delinquency.
In the bank stability index formed, the most important variable in its prediction is e10 (BBB Corporate Yield) for values above 8.5%, the index takes very low values, so there is greater bank instability.