Trusting Consensus vs. Evidence

Matt Ridley has published an opinion piece at the Wall Street Journal, and a follow up piece on his blog, in which he tells us that scientific consensus doesn’t matter, only scientific evidence does.

Phil Plait takes him to task for relying on “evidence” from denialist sources.

While Plait is certainly right here, there is a real philosophical issue that deserves our attention. Because Ridley is also right that what really matters is evidence, not the opinions of scientists.

The underlying question here is when is an argument from authority a fallacy, and when is it a legitimate form of reasoning?

In general, if you’re not an expert and you’re ignorant of (or don’t understand) the evidence in favor of some claim, it makes sense to trust the consensus opinion of the experts. If we didn’t do this, we would end up radical skeptics. We trust the judgement of doctors when it comes to questions of medication, we trust engineers about the safety of bridges, and so on.

But it’s also true that when we have experts debating the quality and force of evidence, the question of what’s the current scientific consensus shouldn’t decide the debate. And if we have experts arguing on both sides of some issue — and there is no established consensus — then there is no authority that we as non-experts can appeal to.

But what about Ridley’s suggestion that we trust the experts about the data, but that he reserves for himself the right to judge the interpretations offered to account for the data. Is this reasonable?

It seems to me that it depends on whether you’re actually in a position to reliably make those judgments. This, of course, is a difficult task (cf. Socratic Humility and the Dunning-Kruger effect); we are likely to overestimate our own cognitive abilities and downplay those of others.

To reliably take on the burden that Ridley suggests we (or at least he) should take on, we’d have to be able to:

1. Make sure we have all the relevant data.
2. Be able to sort out good sources of evidence from bad sources (e.g., avoid denialist sources).
3. Be in a position to understand what sort of inferences the data justify. (Note that scientists undergo years of difficult and rigorous training to develop this ability.)

And so on. (Homework assignment: Come up with three more features that are relevant to the question of whether a person should consider herself well-positioned to reject a scientific consensus.)

Note that it seems that Ridley may have fallen down on point 2, since he seems to be trusting denialist sources — which is the point that Plait is hammering on.