The Prognosticators Reply – Not a Model of the Future

Michigan Capitol Confidential reviewed some of the past job projections on the state's economy by the University of Michigan's Research Seminar in Quantitative Economics (RSQE) and how some have been incorrect about job growth in the last eight years.

Two University of Michigan economists responded to the article in e-mails posted below.

U-M Economist George Fulton, director of RSQE:

"We do document our forecast record yearly for our primary forecast release, which occurs each November at our annual Economic Outlook Conference. By the way, we are the only forecasting organization of which I'm aware that does so. The data cited are based on the annual change in calendar-year averages, the most standard method of presenting these data.

In the period between 1982 and 2000, we correctly forecast the direction of job change for every succeeding year.

In the eight years since 2001, (i.e., 2002-2009, inclusive), we have correctly forecast six years of job decline for the succeeding year, and twice have incorrectly forecast modest job gains for the succeeding year. The two years where we forecast modest job gains (0.8 percent for each year) were 2004 and 2005, when the smallest losses of the decade subsequently occurred (0.4 percent and 0.2 percent, respectively). This was a time of cyclical job recovery in the nation, and the Michigan economy did suffer much smaller job losses then, but structural difficulties among the Detroit Three automakers held back growth from moving into positive territory for the state.

Although we have correctly forecast job loss during most of the decade (an unprecedented run of job loss for the state, by the way), the losses have tended to be more severe. This is particularly the case for 2009, when we forecast the largest job loss the state had ever seen, but were still too light on the losses-as was everybody else, including forecasters of the national economy.

Last November, we forecast a job loss of about two percent for 2010. With eight months of data now reported, it appears that we will indeed lose jobs again this year, but at a smaller rate-around one percent.

I hope that this gives you a broad perspective of our forecast record."

U-M Economist Don Grimes, senior research area specialist RSQE:

"Forecasting is a tough business. The difference between most forecasters and RSQE is that RSQE puts out forecasts on a regular basis, thus their accuracy can be tracked, while most forecasters put out their numbers on a more ad hoc basis. RSQE, in fact, every year as part of their November forecast reports on the accuracy of their previous years forecast.  I have never seen any other forecaster acknowledge their errors on a regular basis.

People need forecasts to make decisions. That includes both businesses and governments.  The two ways forecasts are made are "educated" guesses and structural models.  All of the structural models incorporate educated guesses as well.  The problem that structural models have is that they are predicting the future based upon some historical relationship (educated guesses tend to incorporate some sort of informal mathematical relationship as well). For example, the history in the U.S. up until about 2000 suggested that if U.S. vehicle sales increase by 10% than employment in the auto industry in Michigan would go up by 8% (balance reflects productivity gains).  The problem is when this historical relationship breaks down.  Auto sales go up, but they are all by imports/transplants and so Michigan's employment doesn't increase.

The 2000s, seem to have been a period where a lot of the historical relationships are breaking down, at least temporarily, not just in Michigan and not just with respect to the auto industry, but nationally and with a lot of markets. For example, with mortgage rates this low, the historical relationship would suggest that housing sales should be booming, but of course they are not. Other unincorporated (and often immeasurable) features are holding sales back, but would it be appropriate to assume that the market will never boom if interest rates stay this low?  That seems illogical to me.

So if people have to have forecasts then it seems to me that you have to eventually incorporate mathematical historical relationships, which should generate a reasonable set of forecasts in the long-run. Unfortunately the models can generate inaccurate forecasts in the short-run, especially when the historical relationships are breaking down.

RSQE's state of Michigan forecasts over the past decade have tended to be too optimistic, as they acknowledged every year, but that is because all of the historical evidence in the post World War II period would have said that when the national economy was growing the Michigan economy (as measured by jobs) would be growing.  In sum, during this past decade, Michigan's economic performance was decoupled from the national economy.

Ironically those old historical relationships may be returning in 2010, so far employment in Michigan has been behaving more like the 1950-2000 period than the 2000-2009 period with respect to the nation.  You will want to check this, but I think the Nov. 2009 RSQE forecast for employment in Michigan in 2010 looks like it was slightly too pessimistic so far this year.

It is very easy to criticize forecasting errors in hindsight, but critics who haven't systematically laid out their own forecasts and opened themselves up for similar assessments are nothing but a bunch of Monday morning quarterbacks. I have a lot more respect for critics who put their money (or in this case their forecasts) where their mouth is than those who only complain about someone else's forecasts in hindsight, and you should too.