Receiver operating characteristics curve

Travis Berge and Oscar Jorda of the University of California, Davis have an interesting new paper on statistical criteria for distinguishing economic expansions from recessions.

Berge and Jorda evaluate rules of the form that would declare the economy to be in a recession when some indicator Yt falls below a specified threshold c, for example, saying that the economy is in a recession whenever GDP growth comes in below -0.6%. For any choice of the threshold c, there is some observed fraction of observations for which the economy wasn't in a recession and yet Yt was less than c (the false positive rate), and a fraction of the time when the economy was in a recession and Yt was less than c (the true positive rate). By choosing a lower value for c, there will be fewer false positives and fewer true positives.

The receiver operating characteristics curve plots the false positive rate on the horizontal axis and the true positive rate on the vertical axis, moving along the curve by specifying alternative possible values for c. For example, here's Berge and Jorda's estimate of the ROC for Yt corresponding to the Chicago Fed National Activity Index. The greater the area under the ROC, the more useful that indicator Yt would be for identifying recessions.

ROC curve for Chicago Fed National Activity Index. Source:
Berge and Jorda (2009).

Berge and Jorda evaluate a number of possible indicator series Yt that one might use for this purpose, and find that the Chicago Fed index is one of the best. If you put equal weight on the two kinds of errors you can make with this measure (declaring a false positive versus missing a true positive), Berge and Jorda calculate you'd use an optimal threshold of c = -0.82, that is, declare the economy to be in a recession whenever the Chicago Fed index falls below -0.82. The figure below plots the values for the Chicago Fed index, with shaded regions corresponding to recessions as dated by the NBER. On the basis of this indicator, Berge and Jorda would say that the U.S. recovery began in September, for which the index came in at -0.69, its first reading above -0.82.

Three-month average value for
Chicago Fed National Activity Index with -0.82 threshold. Shaded regions correspond to NBER recession dates.

Another indicator that comes out well on the basis of the area under the ROC is the ISM Manufacturing PMI Composite Index, for which Berge and Jorda propose a threshold of c = 44.7. Note that this is below the Yt = 50 reading at which as many managers are reporting improvement as report deterioration-- things need to be getting significantly worse before it would be characterized as a recession. By this indicator, the recovery began in July.

ISM manufacturing PMI with 44.7 threshold. Shaded regions correspond to NBER recession dates.