Causal Relationships

The reason why we would care about about the long-run probability of the null is not that we’re interested in ‘establishing causality’ per se. If there is a link between two variables in the actual population, then there is a causal link between them. There has to be some reason why the variables take on the values that they do.

However, researchers tend to distinguish between causal and spurious relationships, or relationships that are ‘due to chance’ (whatever that means) and relationships that are not. There is for example a relationship between pool drownings and films Nicolas Cage has appeared in from 1999-2009. Researchers call relationships like these non-causal. But of course there is something that caused this relationship. What researchers seen to mean when they say there is no causal relationship between two processes is that there is no causal relationship specified by the theory or model the researcher is testing. When they say that an effect isn’t causal (like the case of drowning and Nicolas Cage) what they’re really saying is that they don’t know of any interesting theory that would explain the relationship.

The statement that there is no interesting explanation for an observed relationship is equivalent to the statement that the observed relationship would all-else-equal not obtain in the ‘long-run.’ It makes sense to talk about long-run probabilities when making inferences to infinite populations. When I roll a six-sided die arbitrarily many times so as to determine whether it is fair, I am testing the theory that the long-run probability of rolling each number is 1/6. But it is less clear what it means to say, for example, that rich and poor voters are equally likely to vote Labor in the long-run. After all, there are only so many Australian elections that will occur. The sample of rich and poor is finite. Instead, the ‘long-run probability’ of an event occurring for a finite population should be interpreted as the probability of the event occurring in the ‘superpopulation.’ The superpopulation contains all possible populations, including the actual one, that have belong to the researcher’s reference class. If there is a causal relationship between variables of interest, the variables will all-else-equal correlate in the superpopulation.

It might be the case that certain facts generally obtain that all-else-equal cause one process to influence another in a way specified by an interesting model. By ‘generally obtain’ I mean ‘obtain in most possible populations that are relevantly similar to the actual one,’ or ‘obtain in most populations contained in the superpopulation. Suppose, for example, that a political scientist wanted to test the theory that people generally (in other similar populations) voted for parties that advanced their narrower material interests. Suppose also that said political scientists managed to collect data which suggested that higher-income voters in Australia (or other hypothetical ’Australias’) all-else-equal voted for the Coalition. The data would support the theory that people vote for self-interested reasons all-else-equal.

However, all else is not equal. First of all, there may be other facts which generally obtain that the theory we are testing does not specify (for example, racial differences between high- and low-income earners). The way to solve this problem is to ensure that the model we’re using to generate the relevant estimates is fully specified or that the experiment we’re conducting controls for as many irrelevant factors as praticable.

Second, there may be certain facts about the actual population which bring about the link between the variables of interest but which do not generally obtain in other populations where the variables take on the same values. Perhaps Nicolas Cage happened to appear in more movies in hotter years where people spent more time swimming in pools. The thought is that weather patterns could just as easily have been cooler without Cage’s career taking a nosedive. In other words, I am just as likely to sample a population from the superpopulation in which there is a correlation between Cage appearances and drowning as I am to sample a population where there is not. In the long-run the effects will ‘cancel out’ and the null will be true. Of course, researchers often do not need inferential statistics to determine whether ‘noise’ explains a correlation they have discovered in the actual population. It is for example hard to see what interesting model could explain the link between Cage’s career and drownings. But for cases where it seems like an interesting model might explain a correlation, we need a statistical technique which informs us about whether the correlation is also present in the superpopulation.

The only reason why researchers would bother to test a point null, then, would be because it was equivalent to the hypothesis that there is no relationship in the long-run between variables of interest. When a researcher rejects the point null and concludes that a relationship they observed could not have been ‘due to chance,’ the most charitable reading of their conclusion is that there is at least one interesting model that may explain the relationship. Another way of saying this is that, all-else-equal, the relationship would have been observed in the superpopulation.