The first principle is that you must not fool yourself — and you are the easiest person to fool. So you have to be very careful about that. After you've not fooled yourself, it's easy not to fool other scientists.
Richard Feynman, 1974

Suppose that I have done a seismic inversion and have a new seismic attribute volume that predicts Poisson's ratio (a rock property that can help predict fluid type). According to my well calibration and my forward modelling, low Poisson's ratio means Gas. This is my hypothesis; I need to test it.

So here's a game: I have some new wells, represented by double-sided cards. Which cards do I need to turn over to prove the hypothesis that all the cards with Low PR on one side have Gas on the other? Take a moment to look at the four cards and decide which you will flip:

In the course of evolution, our brains have developed heuristics, rules of thumb, for dealing with problems like this one. Our intuition is made of heuristics: we're wary of the outsider with the thick accent; we balk at a garden hose in the grass that could have been a snake. We are programmed to see faces in the topography of Mars (left). The rules are useful to us in urgent matters of survival, letting us take the least risky course of urgent action. But I think they're limiting and misleading when rational decisions are required.

That's why most people, even educated people, get this problem wrong. As scientists we should be especially wary of this, but the fact is that we all tend to seek information that confirms our hypotheses, rather than trying to disprove them. In the problem above, the cards to flip are the Low PR card (of course, it had better have Gas on the other side), and the Water card, because it had better not say Low PR. Most people select the Gas card, but it is not required because its reverse cannot prove our disprove our hypothesis: we don't care if High PR also means Gas sometimes (or even all the time).

Think of a hypothesis you have about the data you are working on right now. Can you think of a test that might disprove it? Would you get funding for a test like this? 

This post is a version of part of my article The rational geoscientist, The Leading Edge, May 2010. I recently read this post on the OpenScience Project blog, and it got me thinking about this again. The image of Mars was created by NASA and the JPL, and is in the public domain.