Make it work!

RPM points to a really clever editorial by Michele Pagano in Cell, titled "American Idol and NIH Grant Review." I'm going to quote the same part, because it captures the rest:

A typical day at one of the many NIH study sections goes something like this. Of approximately 50-70 investigator-initiated/R01 applications reviewed, about half are triaged and the rest are subjected to lengthy discussion, despite the fact that in most of the cases the initial scores are close. Like the amateur singers on the television talent show American Idol, each grant application is evaluated by three reviewers. And, when opinions are conflicting, the three reviewers may display a peculiar resemblance to the American Idol judges, Paula Abdul (sympathetic), Randy Jackson (neutral), and Simon Cowell (hostile). Due to the specialization of science, the discussion is often limited to the three reviewers, with the other study section panelists rarely participating. Indeed, sometimes, while the three reviewers wrangle over a particular application, others are busy on their laptop computers. It is difficult to determine whether these panelists are reading the application under discussion, preparing for the next discussion, or answering their emails. The necessarily inexpert or distracted panelist often sides more easily with the Cowellesque reviewer, who is trashing the application, especially when there is not enough money to go around. This leads to the perception that "the nasty reviewer always wins." Remember, everyone on the study section votes to determine the final score -- even those who are busy with their emails.

Now, see if Simon always won, the show wouldn't be nearly as entertaining. And there is this problem:

Much of the current 25-page format comprises scientific fluff that includes extensive details, alternative strategies/approaches, and pitfalls that are intended to circumvent the critique of whoever is playing the Simon Cowell role.

I just have one suggestion: could they please make it more like Project Runway? I mean, we all know that the successful grants are the ones that are applying to fund results that they have already obtained. That's why they look good -- they have a really good idea what the outcome is going to be.

So if there is already some production, they should get all the PI's to walk their results down a runway to see which ones "work" and which ones don't. Some researchers will be like Robert, running along with whatever trends, but basically predictable and boring. Others will be like Jeffrey -- usually a little "out there", but occasionally really inspired. And after all, aren't most scientific disagreements basically like Jeffrey vs. Laura?

References:

Pagano M. 2006. American Idol and NIH Grant Review. Cell 126:637-638. DOI link