I woke up in a London hotel last Friday, sleep deprived and just aching for some real American coffee. I nibbled my black pudding with fried tomatoes and glanced at the Financial Times. Milton Friedman had died. Friedman is best known for his advocacy of monetarism, a mistake that irrevocably associated him with the gyrating interest rates of the early 1980's. We all make mistakes. Another of Friedman’s mistakes was the time he sat on the Ph.D. examination committee of one Harry Markowitz. He argued against granting the degree on technical grounds. Friedman made many valuable contributions in his lifetime. For me, the most interesting was an article he penned in 1953 called the Methodology of Positive Economics. Drawing on 20th century philosophy of science, he distinguished between a positive science and a normative science. The former describes the way the world is. The latter describes how it should be. A positive science is descriptive. A normative science is prescriptive. Philosophers of the Vienna Circle, and later Karl Popper, explained that a positive science can be judged according to one criteria: the accuracy of the predictions it makes. Science progresses through the development of models (also called laws, theories or hypothesis). These make predictions. Newton’s law of gravitation predicts that, if we drop an apple, it will fall. Any number of meteorological models predict that, if air pressure drops, there will be a storm. We reject models that make poor predictions. We accept those that make good predictions—at least until other models come along that make better predictions. Friedman advocated this outlook for economics. A decade later, William Sharpe would grab it as a lifeline. He was trying to get his capital asset pricing model (CAPM) published. Later, it would win him a Nobel Prize, but Dudley Luckett, editor of the Journal of Finance was refusing to publish it. Sharpe’s model assumed that all investors hold identical beliefs about the expected values, standard deviations and correlations of asset returns. Luckett found this so preposterous as to render Sharpe’s model uninteresting. Eventually, it would take a change of editorship at the Journal of Finance to get the paper published. In the mean time, Sharpe redrafted and resubmitted his paper a number of times. In one of those redrafting, no doubt in exasperation, he included the comment
Needless to say, these are highly restrictive and undoubtedly
unrealistic assumptions. However, since the proper test of a theory is
not the realism of its assumptions but the acceptability of its
implications, and since these assumptions imply equilibrium conditions
which form a major part of classical financial doctrine, it is far from
clear that this formulation should be rejected … In my October 26 blog posting, I commented that Basel and other regulatory requirements that require that value-at-risk (VaR) be measured as, say, 10-day 99% VaR are ill-advised because such numbers can’t be calculated in any meaningful way. Now let’s put that assertion on solid theoretical ground. Positive philosophy of science requires that models be accepted or rejected based on the predictions they make. What predictions does a VaR measure make? The VaR numbers a VaR measure spits out are not predictions. Predictions have to be something we can experience. Saying that a portfolio has a 90% probability of losing less that $100,000 over the next day isn’t a prediction. The next day, when the portfolio has realized a loss of $24,000, who is to say if the VaR measure was “right” or “wrong?” We don’t experience probabilities. If the weather forecast is for a 40% chance of rain, we aren’t going to experience that 40% chance. We are either going to experience rain of not rain. What makes a VaR measure meaningful is our backtesting of it. The backtest is the prediction. We run the VaR measure for a period of time and predict that the results will perform in some manner when backtested. If they do, we accept the VaR measure. If they don’t, we reject the VaR measure. If the VaR measure addresses risky events of such low frequency that backtesting is impossible, then there is no prediction being made. Philosophically, the VaR measure is meaningless. Officially, banks do backtest their ten-day 99% VaR measures. What they actually do is backtest a similar one-day 99% VaR measure and argue that, if it passes the backtest, then the ten-day 99% VaR measure extrapolated from it would also pass a backtest. Such arguments are empty because they are making predictions about things that cannot be experienced. We can never experience a ten-day 99% VaR measure passing a reasonable backtest. Some model that states that, if a one-day 99% VaR measure passes a backtest then so will a ten-day 99% VaR measure, is wrong simply because the ten-day 99% VaR cannot be backtested. The very notion that a ten-day 99% VaR measure can be “good” or “bad,” “right” or “wrong,” “accurate” or “inaccurate,” are meaningless simply because the VaR measure makes no predictions about anything that will be experienced. Karl Popper relates a story involving child psychologist Alfred Adler: As
for Adler, I was much impressed by a personal experience. Once, in 1919,
I reported to him a case which to me did not seem particularly Adlerian,
but which he found no difficulty in analyzing in terms of his theory of
inferiority feelings, although he had not even seen the child. Slightly
shocked, I asked him how he could be so sure. ‘Because of my
thousandfold experience’, he replied; whereupon I could not help saying:
‘And with this new case, I suppose, your experience has become
thousand-and-one-fold.’
Dr. Adler's theories made no predictions about experiences that would allow us to accept or reject them. They were just so many words—meaningless words without predictive content. Popper's example illustrates how the positive philosophy of science is not some abstract idea, far removed from day-to-day scientific practice. It is our logical justification for rejecting astrology while accepting astronomy. It is why we ignore creation science and embrace Darwin’s theory of evolution. In the 20th century, it forced a rethink of the methods used in various social sciences. Friedman wrote his piece on positive economics in 1953 as a part of those efforts. The positive philosophy of science tells us that ten-day 99% VaR, like Dr. Adler's theory of inferiority feelings, is just so many words, devoid of predictive content. Both belong in the same dubious camp as astrology and creation science. This is relevant for more than VaR. Today, financial institutions, encouraged by regulators, are constructing ever more elaborate models for economic capital, portfolio credit risk and operational risk. They all sound very scientific. They are all meaningless. Are any regulators listening? Glyn A. Holton
|
See
Glyn Holton's blog posting
