The Economics of Models

Economic and financial models have come in for a lot of criticism in the context of the global financial crisis, much of it deserved. One of the primary targets is models that financial institutions widely used to (mis)estimate risk, such as Value-at-Risk (VaR) models for measuring risk exposures (which we’ve discussed elsewhere) or the Gaussian copula function for quantifying the risk of a pool of assets.
In September, the Subcommittee on Oversight and Investigations of the House Science and Technology Committee held a hearing on the role of risk models in the financial crisis and how they should be used by financial regulators, if at all. The hearing focused largely on VaR models, which attempt to quantify the amount that a trader (or an entire bank) stands to lose on a given day, with a certain confidence level. (For example, a one-day 1% VaR of $10 million means that on 99% of days you will lose less than $10 million.)
Although the witnesses ranged from Nassim Taleb, who has been arguing for years that VaR models are toxic, to Gregg Berman, who heads a company that develops VaR models for customers, there was surprising agreement on the problems of VaR. As Richard Bookstaber put it, VaR depends on three assumptions, all of which are generally false: not all assets, particularly illiquid ones, are included in the VaR calculation; estimates are based on past data that is unrepresentative of the future; and because financial returns exhibit “fat tails” (extreme outcomes are more likely than you would expect), VaR estimates tell you very little about how bad things can get that last 1% of the time. 
The question, then, is what to do about it. One thing that seems clear is that risk models that are designed to function in normal market conditions should not be relied upon to predict outcomes in times of crisis. On this account, VaR doesn’t kill banks; executives who don’t recognize the limits of VaR kill banks. As Bookstaber put it, “one has to look beyond VaR, to culprits such as sheer stupidity or collective management failure: The risk managers missed the growing inventory [of risky assets], or did not have the courage of their conviction to insist on its reduction, or the senior management was not willing to heed their demands. Whichever the reason, VaR was not central to this crisis.”
Given that everyone is agreeing sophisticated risk models are worthless in crises, it seems particularly remarkable that regulators allowed some banks to use their in-house models in determining their own capital requirements – since one of the purposes of capital requirements is precisely to provide a cushion that protects banks (and their creditors, and taxpayers) in the event of a crisis. The obvious solution is that regulators should rely on cruder constraints, such as an absolute limit on leverage that banks cannot arbitrage around (one of the recommendations of Treasury’s recent white paper on capital requirements, which we discussed here), or periodic stress tests that estimate how bank asset portfolios will perform in a real crisis.
But there is a more interesting question to ask as well: why did VaR become so popular? It’s important to remember that competition among models is shaped by the human beings who create and use them, and those human beings have their own incentives.
David Colander made this point about economic models: the sociology of the economics profession gave preference to elegant mathematical models that could describe the world using the smallest number of parameters. “Common sense does not advance one very far within the economics profession,” he says.
A similar point can be made about VaR models. Sure, maybe all the financial professionals who design and work with VaR know about its shortcomings, both mathematical and practical. But nevertheless, using VaR brought concrete benefits to specific actors in the banking world. If common sense would lead a risk manager to crack down on a trader taking large, risky bets, then the trader is better off if the risk manager uses VaR instead.
Not only that, but imagine the situation of the chief risk manager of a bank in, say, 2004. As Andrew Lo has argued, if he attempted to reduce his bank’s exposure to structured securities such as CDOs, he would be out of a job; VaR gave him a handy tool to rationalize a situation that defied common sense but that made his bosses only too happy. And at the top levels, CEOs and directors who probably did not understand the shortcomings of VaR were biased in its favor because it told them a story they wanted to hear.
In other words, models succeed because they meet the needs of real human beings, and VaR was just what they needed during the boom. And we should assume that a profit-seeking financial sector will continue to invent models that further the objectives of the individuals and institutions that use them. The implication is that regulators need to resist the group think of large financial institutions.  If everyone involved is using the same roadmap of risks, we will all drive off the cliff again together. 
By Simon Johnson and James Kwak
This is a slightly edited version of a post that first appeared on the NYT.com’s Economix, and it is used here with permission.  Anyone wishing to reproduce the entire post should contact the New York Times.