Against Value at Risk

Against Value-at-Risk: Nassim Taleb Replies to Philippe Jorion

Note to the reader: this was written a decade ago. A better explanation is here (easy to read). A more potent argument is presented in this paper and this one. See also my articles with Benoit Mandelbrot.

Trader Risk Management Lore : Major Rules of Thumb

Rule 1 - Do not venture in markets and products you do not understand. You will be a sitting duck.

Rule 2 - The large hit you will take next will not resemble the one you took last. Do not listen to the consensus as to where the risks are (i.e. risks shown by VAR). What will hurt you is what you expect the least.

Rule 3 - Believe half of what you read, none of what you hear. Never study a theory before doing your own prior observation and thinking. Read every piece of theoretical research you can - but stay a trader. An unguarded study of lower quantitative methods will rob you of your insight.

Rule 4 - Beware of the trader who makes a steady income. Those tend to blow up. Traders with very frequent losses might hurt you, but they are not likely to blow you up. Long volatility traders lose money most days of the week.

Rule 5 - The markets will follow the path to hurt the highest number of hedgers. The best hedges are those you are the only one to put on.

Rule 6 - Never let a day go by without studying the changes in the prices of all available trading instruments. You will build an instinctive inference that is more powerful than conventional statistics.

Rule 7 - The greatest inferential mistake: this event never happens in my market. Most of what never happened before in one market has happened in another. The fact that someone never died before does not make him immortal. (Learned name: Hume's problem of induction).

Rule 8 - Never cross a river because it is on average 4 feet deep.

Rule 9 - Read every book by traders to study where they lost money. You will learn nothing relevant from their profits (the markets adjust). You will learn from their losses.

Philippe Jorion is perhaps the most credible member of the pro-VAR camp. I will answer his criticism while expanding on some of the more technical statements I made during the interview (DS, December/January 1997). Indeed, while Philippe Jorion and I agree on many core points, we mainly disagree on the conclusion: mine is to suspend the current version of the VAR as potentially dangerous malpractice while his is to supplement it with other methods.

My refutation of the VAR does not mean that I am against quantitative risk management - having spent all of my adult life as a quantitative trader, I learned the hard way the fails of such methods. I am simply against the application of unseasonned quantitative methods. I think that VAR would be a wonderful measurement if we had models designed for that purpose and knew something about their parameters. The validity of VAR is linked to the problem of probabilistic measurement of future events, particularly those deemed infrequent (more than 2 standard deviations) and those that concern multiple securities. I conjecture that the methods we currently use to measure such tail probabilities are flawed.

The definition I used for the VAR came from the informative book by Philippe Jorion, "It summarizes the expected maximum loss (or worst loss) over a target horizon within a given confidence interval". It is the uniqueness, precision and misplaced concreteness of the measure that bother me. I would rather hear risk managers make statements like "at such price in such security A and at such price in security B, we will be down $150,000". They should present a list of such associated crisis scenarios without unduly attaching probabilities to the array of events, until such time as we can show a better grasp of probability of large deviations for portfolios and better confidence with our measurement of "confidence levels". There is an internal contradiction between measuring risk (i.e. standard deviation) and using a tool with a higher standard error than that of the measure itself.

I find that those professional risk managers whom I heard recommend a "guarded" use of the VAR on grounds that it "generally works" or "it works on average" do not share my definition of risk management. The risk management objective function is survival, not profits and losses ( see rule-of-thumb 8 ). A trader according to the Chicago legend, "made 8 million in eight years and lost 80 million in eight minutes". According to the same standards, he would be, "in general", and "on average" a good risk manager.

Nor am I swayed with the ususal argument that the VAR' s wide-spread use by financial institutions should give it a measure of scientific credibility. Banks have the ingrained habit of plunging headlong into mistakes together where blame-minimizing managers appear to feel comfortable making blunders so long as their competitors are making the same ones. The state of the Japanese and French banking systems, the stories of lending to Latin America, the chronic real estate booms and bust and the S&L debacle provide us with an interesting cycle of communal irrationality. I believe that the VAR is the alibi bankers will give shareholders (and the bailing-out taxpayer) to show documented due diligence and will express that their blow-up came from truly unforeseeable circumstances and events with low probability - not from taking large risks they did not understand. But my sense of social responsibility will force me to menacingly point my finger. I maintain that the due-diligence VAR tool encourages untrained people to take misdirected risk with the shareholder's, and ultimately the taxpayer's, money.

The act of reducing risk to one simple quantitative measure on grounds that "everyone can understand" it clashes with my culture. As rule-of-thumb 1 from "trader lore" recommends: do not venture in businesses and markets you do not understand. I have no sympathy for warned people who lose money in these circumstances.

Praising VAR because it would have prevented the Orange County and P&G debacles is a stretch. Many VAR defenders made a similar mistake. These events arose from issues of extreme leverage -and leverage is a deterministic, not a probabilistic, measurement. If my leverage is ten to one, a 10% move can bankrupt me. A Wall Street clerk would have picked up these excesses using an abacus. VAR defenders make it look like the only solution where there are simpler and more reliable ones. We should not does not allow the acceptance of a solution on casual corroboration without first ascertaining whether more elementary ones are available (like one you can keep on a napkin).

I disagree with the statement that "the degree of precision in daily volatility is much higher than that in daily return". My observations show that the one week volatility of volatility is generally between 5 and 50 times higher than the one week volatility (too high for the normal kurtosis). Nor do I believe that the ARCH-style modeling of heteroskedasticity that appeared to work in research papers, but has so far failed in many dealing rooms, can be relied upon for risk management. The fact that the precision of the risk measure (volatility) is volatile and intractable is sufficient reason to discourage us from such a quantitative venture. I would accept VAR if indeed volatility were easy to forecast with a low standard error.

The Science of Misplaced Concreteness
On the apology of engineering, I would like to stress that the applications of its methods to the social sciences in the name of progress have lead to economic and human disasters. The critics of my position resemble the Marxist defenders of a more "scientific" society who seized the day in the '60s, who portrayed Von Hayek as backward and "unscientific". I hold that, in economics, and the social sciences, engineering has been the science of misplaced and misdirected concreteness. Perhaps old J.M. Keynes had the insight of the problem when he wrote: " To convert a model into a quantitative formula is to destroy its usefulness as an instrument of thought."

If financial engineering means the creation of financial instruments that improve risk allocation, then I am in favor of it. If it means using engineering methods to quantify the immeasurable with great precision, then I am against it.

During the interview I was especially careful to require technology to be "flawless", not "perfect". While perfection is unattainable, flawlessness can be, as it is a methodological consideration and refers to the applicability for the task at hand.

Marshall, Allais and Coase used the term charlatanism to describe the concealment of a poor understanding of economics with mathematical smoke. Using VAR before 1985 was simply the result of a lack of insight into statistical inference. Given the fact that it has been falsified in 1985, 1987, 1989, 1991, 1992, 1994, and 1995, it can be safely pronounced plain charlatanism. The prevalence of between 7 and 30 standard deviations events (using whatever information on parameters was available prior to the event) can convince the jury that the model is wrong. A hypothesis testing between the validity of the model and the rarity of the events would certainly reject the hypothesis of the rare events.

Trading as Clinical Research

Why do I put trader lore high above "scientific" methods ? Some of my friends hold that trading is lab-coat scientific research. I go beyond that and state that traders are clinical researchers, like medical doctors working on real patients --a more truth revealing approach than simulated laboratory experiments. An opinionated econometrician will show you (and will produce) the data that will confirm his side of the story (or his departmental party line ). I hold that active trading is the only close to data-mining free approach to understand financial markets. You only have one life and cannot back-fit your experience. As a result, clinical experiences of the sort are not just the best verifiable accounts of the accuracy of a method: they are the only ones. Whatever the pecuniary motivation, trading is a disciplined truth-seeking proposition. We are trained to look into reality's garbage can, not into the elegant world of models. Unlike professional researchers, traders are never tempted to relax assumptions to make their model more tractable.

Option traders present the additional attribute of making their living trading the statistical properties of the distribution, therefore carefully observing all of its higher order wrinkles. They are rational researchers who deal with the unobstructed Truth for a living and get (but only in the long term) their paycheck from the Truth without the judgment or agency of the more human and fallible scientific committee.

Charlatanism: a Technical Argument

At a more philosophical level, the casual quantitative inference in current use is too incomplete a method. Rule 1 conjectures that there is no "canned" standard way to explore stressful events: they never look alike since humans adjust. It is indeed hard to conciliate standard naive inference (based on past frequencies) and the dialectic of historical events (people adjust). The crash of 1987 caused a sharp rally in the bonds. This became a trap during the mini-crash of 1989 ( I was caught myself ). The problem with the adjustments to VAR by "fattening the tails" as an after-the-fact adaptation to stressful events that happened is dangerously naive. Thus the VAR is like a Maginot line. In other words there is a tautological link between the harm of the events and their unpredictability, since harm comes from surprise. As rule-of-thumb 2 conjectures (see Box), nothing predictable can be truly harmful and nothing truly harmful can be predictable. We may be endowed with enough rationality to heed past events (people rationally remember events that hurt them).

Furthermore, the simplified mean-variance paradigm was designed as a tool to understand the world, not to quantify risk (it failed in both). This explains its survival in financial economics as a pedagogical tool for MBA students. It is therefore too idealized for risk management, which requires higher moment analysis. It also ignores the forays made by market microstructure theory. As a market maker, the fact of having something in your portfolio can be more potent information than all of its past statistical properties: securities do not randomly land in portfolios . A bank's position increase in a Mexican security signifies an increase in the probability of devaluation . The position might originate from the niece of an informed government official trading with a local bank. For having been picked on routinely traders (who survived the sitting duck stage) adjust for these asymmetric information biases better than the "scientific" engineer.

The greatest risk we face is therefore that of the mis-specification of financial price dynamics by the available models. The 2 standard deviations (and higher) VAR is very sensitive to model specification. The sensitivity is compounded with every additional increase in dimension (i.e. in the number of securities included). For portfolios of 75 securities (a small portfolio for a trading room), I have seen frequent 7 and higher standard deviation variations during quiet markets. Thus VAR is not adapted for the brand of diversified leverage we usually take in trading firms. This risk I call the risk of incompleteness issue, or the model risk. A model might show you some risks, but not the risks of using it. Moreover, models are built on a finite set of parameters, while reality affords us infinite sources of risks.

Options may or may not deliver an estimation of the consensus on volatility and correlations. We can compute, in some markets, some transition probabilities and, in some currency pairs with liquid crosses, joint transition probabilities (hence local correlation). We cannot, however, use such pricing kernels as gospel. Option traders do not have perfect foresight, and, as much as I would like them to, cannot be considered prophets. Why should their forecast of the second moment be superior to that of a forward trader's future price ?

I only see one use of covariance matrices: in speculative trading, where the bets are on the first moment of the marginal distributions, and where operators rely on the criticized "trader lore" for higher moments. Such technique, which I call generalized pairs trading, has been carried in the past with large measure of success by "kids with Brooklyn accent". A use of the covariance matrix that is humble enough to limit itself to conditional expectations (not risks of tail events) is acceptable, provided it is handled by someone with the critical and rigorous mind that develops from the observation of, and experimentation with, real-time market events.