Irritating and I Don't Agree

Tim Harford, whom I respect greatly for his ability to write lucidly and clearly on economics, has written about Angrist and Pishke's latest paper on econometrics, building on the supposedly ground-breaking ideas and suggestions of Ed Leamer back in the 1980s.  David Roodman takes up the story here.

As a Christian and econometrician I can't stand his last comment about Christians in America (why is believing in a God that creates un-scientific?  Many Christians are able to bridge this gap David struggles greatly with, myself included.  We're discovering the wonder of God's creation using Science, and the two answer totally different questions - the how and why), but let's get past that.

More importantly, the con has long been taken out of econometrics, and it was done before Angrist and Pishke - in fact I'd say they perpetuate the con. I went to a launch for the Angrist-Pishke book, Mostly Harmless Econometrics, at the 2009 RES conference in Surrey.  Pishke said, basically: Back in the day (1970s) nobody trusted anybody else's econometrics, but now it's different (oh really?).  Now if we have heteroskedasticity (or autocorrelation) in our residuals, we just correct for it using standardised residuals.

Excuse me, but how is that not conning the audience?  We introduce some botched correction which makes a whole stack of assumptions, and is only valid in large samples, and somehow it's all ok?  If I have a gaping wound in my knee, and someone says: Here's a plaster (band aid for the Americans), would you be happy?  That's the equivalent.

Autocorrelation causes bias in estimates, heteroskedasticity causes imprecision.  They are both symptoms of a greater malaise - model misspecification.

So how is it thus ok to paper over the cracks and not address the more serious underlying problems?

That, for me, is the con in econometrics, and it is being continually perpetuated, especially in the econometric sentiment of Angrist and Pishke.

Huh? Why Bayesian is better I don't understand

So it seems Ziliak and McCloskey are getting a lot of attention at the moment, as they've published a new book.  Tim Harford blogged about them, and now there's an article in Science News about them.

I suppose I should really be pleased that scepticism about statistical testing is getting a more widespread audience.  There's little doubt a lot of dubious stuff is done, especially in economics by people wedded to their theories.

But there's something very off-putting about the virulence of Ziliak and McCloskey.  They are no better than the academics they criticise for taking some argument/rule, and pushing it fundamentally.  If you have a moment I'll leave you to their responses to critiques from two of the finest minds in doing statistics properly, Aris Spanos and Kevin Hoover (Google link).  They also don't refrain from taking a pop at Clive Granger because he dares not to fully agree with them.  If anyone disagrees with them, it's the other guys that are misunderstood, it's not that Ziliak or McCloskey could ever need to learn anything new - they've already cornered the entire field, and made sense of all the mistakes the rest of us have ever made.

But anyhow, the Science News article puzzles me.  It goes the whole length talking about how statistics are being manipulated and this reduces their credibility, then finally says: The way forward is Bayesian!  Why is that?  Because Bayesians also use priors.  Now, priors are our prior hypotheses about the thing we're investigating.  We warp the statistical results by our prior belief about what something should look like.

Isn't that manipulation?????

DSGE Scepticism

There's a great new blog being run by Christian Zimmerman (that guy is amazing, the amount of stuff he does online, promoting research is just phenomenal) which aims to discuss research, and in particular DSGE modelling research. The idea is that people comment on the research, and some active online blog discussion of recent research is fostered, instead of online discussion amongst academics just of recent events. This is an excellent idea, and one I hope can be promoted also by what I write here, although my web presence is nothing like Christian Zimmerman's.

The first paper discussed is on estimation of DSGE models in the context of VARs. My sceptical take on the use of econometrics generally was borne out of DSGE models, and their forerunners, RBC models. I've left the following comment (awaiting moderation, last I looked):

I will likely always remain a sceptic of DSGE “estimation”, although it is something that needs to be done. I disagree strongly with any suggestion that estimating an economic model is not a valuable exercise, as is being suggested in the comments here. If DSGE models are ever to have meaning for policy, and in particular macro policy, they have to be estimated, full stop (or period). That they will be rejected is a reason to carry on improving these models, not for rejecting empirical validation per se.

My concerns about the estimation is that little attention appears to be paid to the data, and to the problems it may cause when modelling. There is not a single plot of any of the data series being modelled, nor any reference to their stationarity, or lack of it: By that I mean difference stationarity, or absence of structural breaks. It is well documented the adverse affect of such data ailments, hence my concern.

On top of that, there is no reporting of the actual model estimated, and no attempt to check whether the model is an accurate representation of the data: Whether the model is a good fit. This is essential if any trust is to be put in the coefficients pre-identification, let alone afterwards.

I find the recent trend towards assessing econometric models by their impulse responses deeply disturbing. Not least the identification required is untested, but importantly, as Luca Sala and Fabio Canova have shown, impulse responses are very sensitive indeed to changes in identification.

Put simply, there is no justification for departing from traditional methods of reported econometric results, other than that the author of the paper has something to hide.

There is little doubt that estimating a DSGE model will prove a difficult task given non-stationarity and given structural breaks. And the reported paper is a step forward. However, it is still a long way short of what is required if DSGE “estimation” is to establish a decent reputation for itself, a reputation that would allow the speech marks to be removed from the word estimation.

This Blog

This blog is intended as a place where current or recent research is discussed. It is my intention that critical reviews of the econometric aspects of research will be aired here. Too much economic research is based on shoddy econometrics at best, and bizarrely, the comments on that research tend to bypass the econometrics entirely, and focus on the econometric output, or the economic output, of the paper.

Yet the output of any econometric model stands and falls by whether the econometric model fits the data, and is appropriate. Hence almost certainly, these comments are based on a distorted sense of reality, if little is mentioned in the paper about model specification and fit.

Thus it is my intention to be rather dull and boring on here, to ignore sexy findings in papers and instead ask upon what basis were those sexy findings found in the first place.

It should be emphasised I am a practicing econometrician myself, and it is scepticism with the use of econometrics, not econometrics itself, which will draw my critical comments in this blog.