Article Detail

The Failure of Financial Econometrics: Confirmation and Publication Biases

Journal 36: Global Finance and Regulation

Finance academics tend to believe that they are always ahead of practitioners because they are the smart people who do things rigorously, unlike practitioners who indulge in “Mickey Mouse” research, the results of which are printed on glossy paper (in color, of course) to impress clients. Yet most of the work done by mainstream finance academics is plagued by confirmation and publication biases, which are indeed general phenomena that are not confined to financial econometrics. In fact, confirmation bias transcends academic work to the world of politics and any social debate. In this article, however, we are concerned only with these concepts as they pertain to finance and the empirical tool of finance, financial econometrics, which is the enabler of confirmation and publication biases.

Following some definitions and clarifications, these biases are illustrated with a topic of general interest: whether or not the Chinese currency is undervalued. It will be shown, with the help of an empirical exercise, how financial econometrics is conducive to indulgence in these biases.

Some definitions and clarifications
Confirmation bias (also called confirmatory bias or myside bias) is the tendency of people to prefer information that confirms their prior beliefs. This bias is displayed when people collect, interpret, or remember information in a selective manner. Thus, confirmation bias boils down to the tendency of individuals to avoid rejecting a prior belief, whether in searching for evidence, interpreting it, or recalling it from memory. This kind of behavior is observed more conspicuously in conjunction with emotionally charged issues such as gun control and climate change. Confirmation bias creates the misconception that one’s opinions are the product of years of rational and objective analysis, but the truth is that these opinions are the result of years of paying attention to information that confirms prior beliefs while ignoring information that challenges these beliefs. There is no way, for example, that any piece of evidence would convince a hardcore global warming denier of the existence of this phenomenon – irrespective of any evidence, global warming remains to them “the biggest intellectual fraud in history”.

Confirmation bias has been detected through experiments. For example, Jones and Sugden [(2001), 59] obtain, through an experimental design, “strong evidence of positive confirmation bias, in both information acquisition and information use”, revealing that “this bias is found to be robust to experience”. Based on their findings, they suggest that “the bias results from a pattern of reasoning which, although producing sub-optimal decisions, is internally coherent and which is self-reinforcing.”

Related to confirmation bias is publication bias, which is present when the publication of research results depends on their nature and direction. One obvious example is that a paper is likely to be published in a journal if the results support a mainstream idea, such as free trade, the trickle-down effect, and the benefits of privatization and deregulation. One particular type of publication bias is “positive results bias”, which arises when researchers are more likely to submit, or editors accept, positive rather than null (negative or inconclusive) results. This is why it is common to be in a presentation and hear such expressions as “the results are poor” or “the results are disturbing”, just because they do not support the underlying hypothesis. One consequence of publication bias is that researchers obtaining negative results find it tantalizing to try other “avenues” until they converge on positive results. The danger here is that the motivation will shift from going on a quest for the truth, which is what scientific research should be all about, to getting a publication (and consequently promotion). Needless to say, confirmation bias and publication bias are related and may overlap.

Publication bias is not a recently recognized phenomenon, it goes back to the 1950s and perhaps earlier. Sterling (1959) analyzed publication bias with respect to papers in psychology using statistical tests of significance. He referred to evidence indicating that “in fields where statistical tests of significance are commonly used, research which yields nonsignificant results is not published” and concluded that “the possibility thus arises that the literature of such a field consists in substantial part of false conclusions resulting from errors of the first kind in statistical tests of significance.” To support his proposition, Sterling presented the following statistics: out of 362 papers published in four psychology journals in 1955 and 1956, 294 used tests of significance and out of these, 286 papers rejected the null hypothesis and only eight failed to do that.