Raindrops keep fallin' on my head05 Nov 2008
Orac comments on the rainfall-autism paper:
Note that the authors did not correlate autism prevalence directly with raw mean precipitations but instead used a "relative precipitation variable." When I see something like that, I know right away that there was no correlation between raw mean precipitation levels and autism. If there had been, you can rest assured that the authors would not have bothered to go to the trouble to do this little bit of mathematical legerdemain--excuse me, I mean "transformation."
He has much more on this study, which looks like a fishing expedition using data acquired for other purposes. The trouble with fishing expeditions is that you’ll find a significant correlation with one variable out of twenty (for p < 0.05), or one in a hundred (for p < 0.01). This study has some of both, and many nonsignificant results as well (some of those one in twenty…).
I wouldn’t ordinarily post on this, but I wanted to point to Orac’s post, since it is what the blogosphere is the best for: clear, factual critiques that focus on methods. There are a lot of hidden tricks in the methods sections of papers, many of which even all the papers’ authors may not understand. It takes work to plow through them, and when a study gets press attention, it deserves to be vetted thoroughly by skeptics.