Imagine a group of adventurers set out to explore a new continent, each one choosing a different valley to map. And suppose that, instead of sending back reports of what they found (“a forest, a swamp, mosquitoes”), each one wrote reports calculated to appear as exciting as possible (“a forest, abundant water, rich fauna, couldn’t disconfirm rumors of a city of gold”).
This is how I feel when I’m refereeing economics papers: everyone’s trying to tell an exciting story, and it takes a great deal of work to figure out what actual novel facts they have discovered. It’s frustrating because it’s such an inefficient way to explore the territory: so many people have spent so much time on this, and we have so little to show. What have we discovered about decision-making in the last 50 years after proposing thousands of different models, running tens of thousands of experiments, and regressing millions of variables? Couldn’t we have accumulated more knowledge if we’d organized things differently?
(Although I have to admit that many of my own papers do include some speculation about cities of gold. But I think the most useful thing a referee can do is to suggest changes to the title and abstract, to make it more transparent exactly what the paper has found.)
–
Followup: what have we discovered about decision-making in the last 50 years?
Once, when I taught a graduate class in behavioural economics, a couple of students came from civil engineering & dentistry, hoping to learn something useful that they could use in modelling decision-making - e.g. in modelling how people make decisions about commuting. I was embarrassed in how little I could help them. I was able to think of a lot of useful stuff about decision making that’s ~50 years or older: utility & expected utility, exponential (& hyperbolic) discounting, Engel curves, responses to permanent & temporary income, tractable demand estimation, the offsetting income & substitution effects of wages. All of this is immediately useful in quantifying decision-making. But I can think of few recent examples of quantitatively useful findings. Special mention to kahneman & tversky. If you ask, say, about attitudes to uncertainty, to time, to temptation, to intertemporal complementarities, I would begin my answer with “there are various schools of thought…”.