Correlation Implies Causation

Note

I wrote this around 2020, when working at Facebook, trying to reconcile practical decision-making with how economists talk about identification.

In a sense thes points are all well known. The idea is treated formally in Imbens (2003), Manski’s identification bounds; Oster (2019). I think Mostly Harmless Econometrics discusses the point informally. Still I found this a useful way of explaining the distinction.

Correlational evidence is always informative about causal effects.

The correlation is the sum of the causal-effect and the confounding-effect. To update your estimate of the causal effect you need to filter it through your prior means and uncertainties about each.

We use correlations all the time to help estimate causal effects.

The slide-decks that circulate in businesses and governments are full of scatter plots and conditional means, and that’s a good thing. It would be madness to make decisions without collecting that data.

Statisticians teach taboos and fetishes.

There’s a taboo against consuming raw correlations because they can make you sick. The taboo is there for a reason but the truth is that they are very nutritious when prepared correctly.

There’s a fetish for identification – finding circumstances where we can assume the confounding-effect is exactly zero – but that’s way too conservative. There are lots of cases where the confounding-effect is sufficiently contained that we can learn a lot about causation from correlations, without thinking it’s exactly zero. We do this every day but unfortunately statisticians are typically prudish and so rarely give useful advice on how to do it well.

Why the taboo?

The taboo is because sometimes people forget entirely about confounding: they get a regression coefficient and say “this is our best estimate of the causal effect”. Doing that is positively harmful, and if that’s the only thing you’re doing with the data you’re probably better off not looking at the data at all.

The right way of interpreting correlations goes something like the following: “the coefficient is above what I expected by 40%, and my uncertainty about the causal & confounding effects are roughly similar, and so I’ll revise upward my estimate of the causal effect by 20%.” If you’re relatively more certain about the causal effect, then you update relatively more on the confounding, and vice versa.

This sounds complicated, but we do it intuitively all the time.

Another reason for the fetish for zero-effects: it lets you keep things objective.

As soon as you allow for some non-zero confounding effect then your interpretation relies on more inputs – you have to put in your best-estimates of means and ranges for different variables.

This is hard in science: it typically means we have to allow subjective judgment to enter the inference process, and that can cause pain – through people being either insincere, or self-deceiving, or clumsy in reporting their judgments. So if you don’t trust other people, or don’t trust yourself, maybe you should stick with well-identified regressions.

Put more broadly: insisting on identification is approprirate in a low trust environment.

The taboo is holding back science.

There are a lot of scientific fields where it’s hard to find good identification, and it feels like those fields have been stuck for decades looking for their keys under lamp-posts: economic development, returns-to-education, nutrition. (Obviously this claim deserves a substantial argument to back it up, & I don’t have anything prepared, but this is how I feel).

Some best practices for interpreting causal effects.

• show rich scatter plots to get a full picture of correlations;
• benchmark against other correlations;
• brainstorm differential diagnoses, with a causal diagram;
• be explicit about ranges for both the causal and confounding effect you’re inferring from the correlational data.

Identification comes from assuming some causal effects are zero, but they rarely are.

We often talk about whether some causal effect “is identified” or “is not identified” as if it’s a binary thing. Realistically it’s a continuum. Identification happens when we identify a causal effect with a correlation. The simplest case is univariate regression: \(E[\hat{\beta}]=\frac{cov(y,x)}{var(x)}=\beta+\gamma\). When you assume unconfoundedness, i.e. \(\gamma=0\), then you get identification: \(E[\hat{\beta}]=\beta\). You get identification from trimming the causal tree.

A lot of academic social science uses identification based on an assumption that some effect is probably pretty small relative to the effect of interest, such that we can ignore the bias (e.g. instrumental variables, diff-in-diff, matching, etc), and they use 0 as a stand-in for “small”. But by searching for places where we have clean identification, we’re passing over mountains of good evidence.

Doesn’t it depend on your assumptions about the true model?

People talk about different DAGs as if one of them was true and the other ones not (“does X affect Y, or does Y affect X?”). But we know that everything affects everything, and the idea that some causal arrows have coefficients of zero is just a convenient fiction to help us simplify a complex problem into a simpler one, and achieve identification.