No One Can Forecast Well – What Does That Mean For CECL?

Current Expected Credit Loss (CECL) is the new standard for assessing loss on credit instruments as mandated by the Financial Accounting Standards Board (FASB). Specifically, the current “incurred loss” standard, in which losses on loans and bonds are recognized only when a probable loss has been incurred, is being replaced by the CECL standard in which holders of debt will be required “to measure all expected credit losses for financial instruments held at the reporting date based on historical experience, current conditions and reasonable and supportable forecasts”[1].

The FASB, of course, does not specify the things for which reasonable and supportable forecasts must be made except that those forecasts affect the collectability of the cash flows on the credit instrument. Those institutions that must comply with the new standard can assume that the forecasts are of factors relevant to estimating expected credit loss.

It turns out that no one – NO ONE – can forecast well. Let me provide an example that is directly relevant to CECL.

Forecasts of the U.S. Economy

Every month, Consensus Economics Inc. surveys a panel of prominent United States economic and financial forecasters for their predictions on a range of economic variables including future growth, inflation, budget balances, and interest rates. Twenty-eight individual firms and academic organizations provide both annual and quarterly forecasts of these variables through two full calendar years. Consensus Forecasts summarizes these individual forecasts into median and mean values with associated standard deviations.

For this example, I have selected forecasts of real gross domestic product (GDP), the national unemployment rate and consumer price inflation, starting in June 2017, and I compared the quarterly forecasts to actual values through 1Q2018.

In Figure 1 below, I show the year-over-year change in real GDP for the history of this variable from 1Q2016 to 2Q2107, the average forecast for 2Q2017 to 4Q2018 published by Consensus Economics as of June 2017, and the actual values for this variable from 2Q2017 through 1Q2018. Over the full history of this variable from the middle of 2016 through the first quarter of 2018, there is a linear increase in the rate of real economic growth, meaning that the economy has been expanding at an increasing rate.

The consensus of economic forecasters is somewhat different. The forecasters’ average view predicted that the rate of economic growth in the U.S. economy would range from 2.1% year-over-year to 2.4% but be essentially flat from mid-2017 to the end of 2018. For the period of this forecast in which there are observable values for economic growth, these forecasters were more conservative in their views about economic growth than turned out to be the case.

Figure 1

Importantly, the divergence between the actual rate of economic growth and the consensus forecast appears to widen over time (Figure 1).

With respect to the national unemployment rate, actual values declined steadily from 4.93% at 3Q2016 to 4.1% 1Q2018 (Figure 2), a pattern consistent with the growing rate of expansion in the real economy. However, the forecasts published by Consensus Economics show a different pattern: the average of predicted unemployment rate falls to 4.3% to 4.4% in the second half of 2017 and remains at 4.3% through 2018. Since the consensus forecast on economic growth is more conservative than the actual growth, it is not surprising that these forecasters should expect higher unemployment than actual.

Figure 2

Again, the divergence between forecast and actual unemployment rate appears to widen over time (Figure 2).

Finally, the pattern of forecasts and actual values for consumer inflation is shown in Figure 3. The actual inflation rate fell sharply from 1Q2017 to 2Q2017 and rose gradually to 1Q2018. The average forecast of Consensus Economists shows an initial decline of similar magnitude although over four quarters and the increase in the inflation rate occurs later in this forecast. Overall, though, the consensus forecast of consumer inflation is not materially different from actual inflation although the timing of forecast vs actuals is different.

Figure 3

What Do We Make of These Results?

Professional economists and their organizations don’t appear to be able to forecast the future pattern of important elements of the U.S. economy if these examples are representative. Others have observed and commented on this phenomenon, including Moody’s Analytics and Michael Fadil of Citizens Bank, so my ideas aren’t new or unique. Even these observers weren’t first if we are to believe the iconic explanation for the term “dismal science” as ascribed to economics—that the Scottish writer, historian and essayist Thomas Carlyle first applied it to T. R. Malthus’ prediction that the human population would always outgrow its food supply and people would forever live in misery and poverty[2].

So, you can get wrong in a really big way and still go down in history.

In fact, economists are well known to get it wrong in economic forecasting. For example, Prakash Loungani of the International Monetary Fund studied the performance of consensus forecasts of GDP growth for industrialized and developing countries from 1989 to 1998[3]. Among the questions he asked was: “how well do forecasters predict recessions?” And the answer was: only two of 60 recessions that occurred worldwide during this period were correctly predicted a year in advance. Loungani concluded: “the record of failure to predict recessions is virtually unblemished.”

Yet, CECL requires “reasonable and supportable forecasts” of the factors that lead to expected loss. This requirement becomes quite problematic if the forecast inaccuracy presented above is generalizable.

Here’s an example. Let us suppose, reasonably, that the probability of default for commercial & industrial (C&I) companies increases in economic recessions and decreases in expansions because their operating profits (and thus, debt repayment capacity) increase with a growing economy. And we do observe that historical C&I default and loss rates are negatively correlated with change in real GDP and positively correlated with the unemployment rate.

Let us suppose, further, that the manager of the Allowance for Loan and Lease Losses (ALLL) at Bank XYZ uses these correlations to build a statistical model forecasting C&I loan losses as a function of change in real GDP and the national unemployment rate. This model is heuristically sound and meets all internal model development and validation standards so our manager feels confident in applying this model to determine CECL-specific loss provisions.

What happens if our ALLL manager also uses the consensus economic forecasts presented above? In today’s economic expansion, she will over-forecast defaults and losses relative to those that emerge under actual conditions because the consensus forecast is more conservative than the actual economy. Under CECL, this will mean higher current EL, higher loan loss provisions and lower earnings.

So, errors in forecasting the macro-economy will likely mean larger-than-necessary ALLL. And if the divergence between forecast and actual that are suggested in Figures 1 and 2 is maintained, the provisions may become more overstated.

Unlike bank stress test programs like CCAR and DFAST where there are regulatory benefits for holding more capital than a bank’s models would imply, there’s no benefit in CECL for being conservative in loss forecasting or taking more loss provision than sound and well-governed analytics would imply. The higher-than-necessary loss provisions mean lower earnings per share and, presumably, unhappy shareholders.

Conversely, it possible, even likely, that consensus economic forecasts will understate or even miss economic recessions, if we take Loungani’s work at face value. What happens to our ALLL manager then? She will under-forecast defaults and losses and will have an ALLL that is potentially too low for actual conditions.

As we see, real consequences of inaccurate forecasts begin to emerge. This is model risk if there ever were model risk.

So What Do We Do?

Get better governance into the ALLL process.

CECL will require that institutional investors in credit have credible data and analytics, including forecasting models, to support their estimates of EL. Further, CECL will put a premium on accuracy since, in my simple example, it’s quite easy for a CECL-based all process to over-forecast losses sometimes and to under-forecast them at other times. The place where that accuracy can be assured is in the CECL governance process.

The structure of good governance that should guide the use of such data, analytics and models in CECL is found in SR11-7, “Guidance on Model Risk Management.”[4] For example, one of the critical elements of good model risk management (MRM) is validation of data, tools and processes.

SR11-7 is directly applicable to regulated banks and it is a joint publication of the major U.S. bank regulatory agencies (e.g., the identical document as issued by the Office of the Comptroller of the Currency is OCC11-12). More importantly, the principles of SR11-7 –among which is good governance around processes that rely on data and models and other mathematical analytics—should be adopted by all firms subject to CECL. For the sake of good management and shareholder value.

And the creation of “reasonable and supportable forecasts” is one of those processes that should be subject to the governance disciplines of SR11-7. Specifically:

1. If CECL firms use consensus economic forecasts to guide the creation of their own reasonable and supportable forecasts, the firms will need to determine, explain and document why, when such consensus forecasts are demonstrably inaccurate, they are “reasonable”.That they may be better than anything else is, in my judgement, not a sufficient criterion for “reasonable”.Accuracy strikes me as a better criterion and, in the case of consensus forecasts, is measurable.

2. CECL firms that use consensus forecasts should have, in their ALLL process, a program to track the accuracy of these forecasts as actual economic performance emerges and to “mine” the differences between forecast and actual economic conditions for further information by which to make their EL estimates more accurate.The use of time-series analytics with appropriate time lags will become more and more important with this on-going refinement.

3. There should be robust discussion, challenge and debate within each institution’s governance structure (e.g., ALLL committee) about each quarter’s reasonable and supportable forecasts for CECL.This governance process should be documented to the standard that a qualified, independent third party can understand that governance process and the conclusions reached.

I welcome your thoughts and comments.

[1] Financial Accounting Standards Board, ASU 2016-13 Financial Instruments—Credit Losses (Topic 326), https://www.fasb.org/jsp/FASB/FASBContent_C/CompletedProjectPage&cid=1176168232014.

[2] Not exactly true but it makes for a dramatic tale. Carlyle actually did ascribe “dismal science” to economics but in the context of slavery in the West Indies.

[3] Loungani, P. 2000. How accurate are private sector forecasts? Cross-country evidence from consensus forecasts of output growth. International Monetary Fund Working Paper 00/77.

[4] Board of Governors of the Federal Reserve System. Guidance on Model Risk Management. April, 2011. https://www.federalreserve.gov/supervisionreg/srletters/sr1107.htm.