We need economic forecasters even though we can’t trust them

Walter Friedman, director of the Business History Initiative at Harvard Business School, on the pioneers of market prediction. For more, read his book, Fortune Tellers: The Story of America’s First Economic Forecasters.

TRANSCRIPT

JUSTIN FOX: Welcome to the HBR IdeaCast from Harvard Business Review. I’m Justin Fox, and I’m talking today with Walter Friedman a lecturer and director of the business history initiative at Harvard Business School. And author of the new book Fortune Tellers the Story of America’s First Economic Forecasters. Walter, welcome to IdeaCast.

WALTER FRIEDMAN: Thank you so much for having me on.

JUSTIN FOX: So your focus in this book is on the country’s first economic forecasters. When did they arrive on the scene?

WALTER FRIEDMAN: They arrived on the scene around the turn of the century. One of things that was interesting to me about forecasting, was said it was a lot like other things that were happening at this time. Which was a period from late 19th century to the early 20th century, when the American economy was moving from agriculture to industry. And a lot of historians have looked at things like Taylorism, which is to say, production management and personnel management.

And I also looked at the history of sales management. All of these things develop at this time, to make the functioning of business more rational and more efficient. So I became interested in these forecasters who were trying to do something different, which is to say, they were trying to understand the very atmosphere in which business operated.

And to me, that was a different project. And one that called on similar tools, that is to say, they also looked at metrics, but they used them in different way. They used them in a way to make sense of how big changes in the economy happened.

JUSTIN FOX: Did think actually makes sense of it? Because there’s a lot of failure in your book.

WALTER FRIEDMAN: There is a lot of failure in my book. But I think forecasting is a very maligned industry. I think that most people tend to look at the history of forecasting and think of it as the march of folly. Just because they look at whether or not they become more accurate over time.

And the truth is, they really haven’t. No one’s ever been able to produce a reliable method of forecasting and making predictions of where the economy is going. And they won’t be able to. But in the effort to forecast, they’ve done a lot of great things. They’ve introduced leading indicators.

They’ve created econometric models. They founded important institutions, like the National Bureau of Economic research. And they’ve come up with ideas of how to conceptualize the economy.

The first idea, of course, was popularizing the idea of a business cycle. And that was really important to people in this early period from the late 19th and early 20th century. If you could think about an economy that’s moving from one with agricultural cycles and rhythms, the idea that the economy also had its own internal rhythms was very appealing and comforting. And I think that’s one of the main functions of forecasting. Is to make people comfortable about the capitalist economy.

JUSTIN FOX: And that’s one of the things you talk about the book. Is just this very idea that there’s this thing called the economy, was something new. There was this great sense I really like. The idea of an autonomous economy that followed decipherable rules. That’s what these guys were about.

WALTER FRIEDMAN: It was. And a lot of ways, to me, these guys there’s a parallel between everybody talks about the Silicon Valley being this bunch of entrepreneurs who created the computer industry in their garage. Well in some ways, this was a bunch of entrepreneurs. Because it was an entrepreneurial endeavor at the beginning and not an academic one.

A bunch of people, who gathered data, either in their basements, or in some subdivision of their office, and invented the idea of the economy and popularized it to people. And they made it tangible by sending out economic charts. And sending out weekly newsletters. So that people got a sense that the businesses around the United States were connected in some way. That they all went up and down together. And that what somebody did in Cleveland, had an impact on what somebody did a New York. And that they weren’t all just separate endeavors. And so they gave people a greater sense of what it was that everybody was part of.

JUSTIN FOX: I guess we ought to talk about some of these individuals at this point. Roger Babson is the guy who seems to start everything off in your account. Who was he?

WALTER FRIEDMAN: Babson is one of my favorites. Babson was an entrepreneur. He was an MIT graduate. He was fascinated by economic statistics. And after the panic of 1907, he decides that he’s going to create the Babson chart, which will signal when future ups and downs in the economy will come. Now what he does is he gathers all the economic statistics he can. And creates a huge aggregate. And makes a beautiful red and black chart, that shows how areas of boom and bust will equal out over time.

Now he’s also a bit crazy. He’s influenced mostly by his fascination of Sir Isaac Newton and ideas of gravity. In Babsonn’s mind, if he can add together all the things in the economy that are moving up, and all the things that are moving down, they’ll equal out over time. Now it’s a very appealing thought. That the economy moves in this perfect way in which ups always match downs, in which booms always match busts.

And I think that’s appealing to people. And I think that’s the way a lot of people tend to think about the economy these days now. It’s got a very strong psychological appeal. If you see the housing market, for instance, going way, way up your mind tells you that it’s probably going to come down a similar amount and reach some kind of normal level. And that was Babson’s idea. That there was a normal level at which the economy should operate.

JUSTIN FOX: He had this line, trend line, that he drove through his charts.

WALTER FRIEDMAN: Babson did have a trend line that he drew through his charts. And that was the normal growth rate. Which he guessed to be about 3% per year. Which I think it is not a completely crazy amount. That’s what most people would hope for. And Babson gets that going. So it was a nice addition to his chart.

He was extremely popular. He was also a very good marketer. He had people go around to rotary clubs and to churches, to explain how the Babson chart worked. So if you were an investor, a small investor, I remember reading about one dentist in Pennsylvania who wanted to figure out how to invest. And so a Babson representative came around, showed him how the chart worked, and that gave him a plan. It turned out, for the ’20s, it was an OK plan. Not a great plan. But at least it was a plan. And everybody needs some plan for investing.

JUSTIN FOX: What about the more supposedly scientific approaches? Irving Fisher became a rival of Babson, right?

WALTER FRIEDMAN: Fisher is completely different from Babson. I mean they share a lot of similarities, which makes them interesting. They both are interested in food faddism they’re both interested in eugenics, they’ve both become very politically active, and they both have a lot of strange interests like making inventions, and securing patents for all sorts of devices.

But they couldn’t be more different in the ways that they think about the economy. Babson really believes in historical trends. He’s just a trend analysts. So he looks over time to see how trends have changed. Fisher really doesn’t think trends are important at all. Instead he believes in causation. He tries to figure out how the economy actually works. What effect does it have if we innovate?

What effect does it have if we change the way we manage firms? What effect does it have we plow earnings back? What effect does it have if we pour dividends back in to earnings? What effect does it have if we come up with new innovations that change the way people work? Fischer tried to find ways to quantify these changes and the effects they would have. Not surprisingly, he was an early pioneer of econometrics. And one of the founders of the econometrics society.

So on the one hand, you have somebody who doesn’t believe in causation at all, which is Babson, who just looks at trends. And somebody who believes entirely in causation. You have to figure out what’s making things change in the economy. And they both have different legacies. Some people today still like to look at historical trends. And others are much more in the causation camp.

JUSTIN FOX: It seems like academic economics clearly has followed Fisher’s lead.

WALTER FREEMAN: I think it mostly has. Although there’s others. So there are people who don’t really believe that they can figure out how the economy works. So instead they focus on observation. And there are people amongst the big data group for instance, who just want to look empirically and what happened. And try to figure out what will happen next. Without actually creating a big mathematical model to solve it.

JUSTIN FOX: So in the late ’20s, the Babson approach and the Fisher approach, came in to pretty much direct collision. Who won that battle?

WALTER FREEMAN: Who won that battle is hard to say. If you’re an investor, you like Babson much better. Because by luck, he got it right in fall of 1929. He had gotten it wrong. He had predicted a great crash also in ’26, ’27 and to ’28. So if you followed him then you lost what you would have earned, had you kept your money in the market.

But the truth is, that Fisher wins out in the long run. Because nobody really takes Babson seriously anymore. And whereas Fisher, who gets it wrong in 1929, and loses everything including his house, his ideas start to spread. The econometrics society is founded in 1930. Alfred Cowles of the Cowles Commission, who’s one of Fisher’s former students, start spending a lot of money promoting the idea of mathematical approaches to forecasting and to econometrics.

So in the post World War II period, Fisher’s ideas really catch on. Whereas Babson is much more marginal as a figure.

JUSTIN FOX: Although he does have a university named after him. And Fisher doesn’t.

WALTER FREEMAN: That’s true. But Fisher is cited as one of the greatest economists of the 20th century.

JUSTIN FOX: So this move to a more theoretical mathematical approach. Has it resulted in better forecasts?

WALTER FRIEDMAN: I think the big move in forecasting that really has made a difference, is the transition from the idea of a fixed business cycle which really predominated in the 1920s. That is to say a business cycle that you can’t alter in any way. And that is a bit like a meteorological cycle, that it comes and goes, and that institutions and people can’t do anything about it. To one in which the government can actually try to flatten cycles.

That really has changed things. And Fisher did believe in that. And the people who tried to understand causation believed in that. And to me that’s a fundamental change. Once you start believing that’s cycles aren’t something that simply happens to an economy, but are ones that government can make a difference in, that’s a big change.

JUSTIN FOX: Is there a risk that you get overconfident because of that? Because that seems to be one of the things that happened in the ’20s. Is things kept going so well, and the forecasters who predicted a continued boom or right again and again. If you get very confident that government knows what it’s doing, and can manage the economy well, does that increase risk at all?

WALTER FRIEDMAN: I think the real danger, is believing that any forecaster really knows what he or she is doing. There’s a real danger of falling for gurus. Particularly after a crash. When somebody claims they knew it was coming and knew they got it right. I think that one of the real problems that people have with forecasting is that they tend to look at past accuracy as an indication as to whether or not to follow a forecaster.

When in fact, past accuracy is much more often based on luck, as far as I’m concerned. And what you really have to do, is try to figure out what these forecasters are thinking about. What their reasons are. And then you can start to decide whether you want to follow one or another. But if you just sort of blindly look, as to whether they got it right or wrong, then you’re probably going to fall for a guru one of these days.

JUSTIN FOX: And that again fits this wonderful theme that goes through the book, which is that you shouldn’t really ever entirely trust any of these people. Yet at the same time, they’re useful. They’re valuable to the economy. Right?

WALTER FRIEDMAN: I think you need to think about forecasts a bit like a treasure maps. In that you always have to be skeptical of the person who would want to print and sell treasure maps on mass. When it’s much better for you to simply keep a treasure map, if you actually have a valid one, and get the treasure yourself. You have to keep in mind that these people are trying to sell their predictions. And that persuasion is as big a part of this industry as prediction is.

JUSTIN FOX: Walter, thanks again for talking with us.

WALTER FRIEDMAN: Thank you so much for having me on. I really enjoy talking with you.

JUSTIN FOX: That was Walter Friedman. His new book is called Fortune Telling the Story of America’s First Economic Forecast. And this was the HBR IdeaCast. For more, go to hbr.org.

–HBR IdeaCast, http://blogs.hbr.org/2014/02/we-need-economic-forecasters-even-though-we-cant-trust-them/