Thoughts from thesis
The gold standard of statistical significance is p<0.05, which means that there’s less than a 5% chance of getting the data you did if your conclusion is false. But this means that up to one in twenty papers presents an incorrect conclusion! And if you account for the fact that only interesting stuff gets published, the number of false positives could be a lot higher.
Chew on that, science reporters!
(Update: this article talks about publication bias, claiming that such bias might invalidate up to one in three animal-model studies.)