john hawks weblog

paleoanthropology, genetics and evolution

grants

  • Ecologists against public access to peer reviewed publications

    Fri, 2012-01-06 14:59 -- John Hawks

    This seems incredible, from Jonathan Eisen: "YHGTBFKM: Ecological Society of America letter regarding #OpenAccess is disturbing".

    Wow -- I am really disturbed by the letter the Ecological Society of America (ESA) has written to the White House OSTP in regard to Open Access publishing.

    ...

    So - the justification here for not making ecological articles available is that they are MORE important over time? So the taxpayers pays for research that is valuable and because it is valuable over time we should make it less freely available? Seriously?

    This next week is an important one for proponents of open access publication and data access, as the White House Office for Science and Technology Policy has requested public comments related to both these issues for federally funded research. I will be posting my letter about data access when I complete it this weekend. I encourage everyone to pay attention and submit a letter if possible. It is dismaying to see professional scientific societies take public stands against making their members' research available.

  • "Transformative" research can't come from milquetoast

    Sat, 2011-12-10 15:25 -- John Hawks

    Philip Ball writes in The Guardian about another new initiative from NSF to fund "potentially transformative" research ("Science funding tends to favor mediocrity over grand ideas".

    He begins his essay with this:

    The kind of idle pastime that might amuse physicists is to imagine drafting Einstein's grant applications in 1905. "I propose to investigate the idea that light travels in little bits," one might say. "I will explore the possibility that time slows down as things speed up," goes another. Imagine what comments these would have elicited from reviewers for the German Science Funding Agency, had such a thing existed. Instead, Einstein just did the work anyway while drawing his wages as a technical expert third-class at the Bern patent office. And that is how he invented quantum physics and relativity.

    The moral seems to be that really innovative ideas don't get funded – that the system is set up to exclude them.

    The system is set up to exclude really innovative ideas. But Einstein is a really misleading example.

    For one thing, Einstein didn't need much grant funding for his research. Yes, if somebody had given the poor guy a postdoc, he might have had an easier time being productive in physics. But his theoretical work didn't need expensive lab equipment, RA and postdoc salaries, and institutional overhead to fund secretarial support, building maintenance and research opportunities for undergraduates.

    It is a better question whether we would have wanted Einstein to spend 1905 applying for grants instead of publishing. But even this is terribly misleading. Most scientists who are denied grants are not Einstein. Most ideas that appear to be transformative in the end turn out to be bunk. Someone who compares himself to Einstein is overwhelmingly likely to be a charlatan. There should probably be a "No Einsteins need apply" clause in every federal grant program.

    Setting aside the misleading Einstein comparison, our current grant system still has some severe problems. Is it selecting against "transformative" research, or big breakthroughs? I would put the problem differently. "Transformative" is in the eye of the beholder. Our grant system does what it has been designed for: it picks winners and losers, with a minimum of accountability for the people who set funding priorities.

    We might be perfectly happy if the winners were scientists who all go on to make important breakthroughs. But in reality our system picks winners in a way that often selects against creativity and significance, and selects for established networks of institutions and senior scientists, and above all "grantsmanship". What is the difference between a 35-year-old assistant professor who becomes the manager of 1.5 million dollars of federal money, and his 37-year-old colleague who has been denied twice for the same grant before applying for tenure? In a system where fewer than 20% of grants are funded, the difference may be luck. The "transformative" value of either person's ideas hardly comes into this calculation. Yet this is the system we are currently using to staff the next generation of senior science positions.

    Anyone who has submitted grant applications in multiple years can see this in action. The reviews in one year often completely contradict those of the previous year, even for the same project. An application's chances of being funded are based on the luck of who reviews the application and who is in the room. The way to ensure bad luck is to be an outlier. I have applied for federal grants several times, and have often had strong reviews but have never been funded. A good application can take weeks of effort to prepare, as much as a research paper. Yet only a small fraction of applications are funded. For me, each time has been a costly training for writing the next unsuccessful grant application.

    I don't want there to be a pool of money set aside for "high-risk" or "transformative" work. I want the agencies to set transformative objectives and to fund projects in accordance with them. If my scientific objectives don't match those of the agency for my research area, I want to know that so I don't spend time on fruitless applications. If a grant agency has milquetoast objectives, I want a transparent process by which ordinary scientists can participate in changing those objectives.

    Anthropology has its own unique grantwriting challenges, and we can't easily generalize across fields. Some of the very large grants available from NSF for human evolution research are strongly interdisciplinary, and much of the budget of funded projects is spent outside of anthropology. "Transformative" research in this context may be conceived as the collaboration of scientists from different areas, even when the results themselves may be quite conventional. In my opinion, funding productive field projects is the most effective use of federal money in paleoanthropology. Every new fossil discovery might be the one that transforms our understanding of the important events in human evolution. But those field projects may actually inhibit "transformative" research if they do not make their results available to other scientists in a timely manner, if they do not openly archive and document their activities in the open, and if they do not contribute to the education of future scientists.

    Many have pointed to the problem of transitional funding for early career scientists. Few have noticed that this is a decay curve. As researchers get older, some of them get their first grant, but many others stop trying. Personally, I'm still quite a ways from the age when the average grant recipient receives his first federal grant. But I'm very glad that I've chosen a research area where I can do great research without that federal money.

    Synopsis: 
    Grant agencies should set transformative objectives, not set aside money for transformative research
  • Retractions and grants

    Thu, 2011-04-21 10:46 -- John Hawks

    Pascale Lane reviews a paper about retraction rates in top journals: "Papers 'Not Meant to Be Factual'".

    Rigorous peer review may help uncover fraud or fabrication, but, as the editor of Science wrote, "It is asking too much of peer review to expect it to immunize us against clever fraud."

    I'm just noting this because, on the subject of my previous post about grant applications, the rate of fabulism must be much higher in that system than in the top journals. Nobody will retract your money if your "preliminary results" fail to play out. And the top journals (or "glamour mags" as many call them) have a shockingly high rate of retractions.

  • Funding people

    Thu, 2011-04-21 09:34 -- John Hawks

    From Scientific American's editorial on grants, "Dr. No Money":

    Most scientists finance their laboratories (and often even their own salaries) by applying to government agencies and private foundations for grants. The process has become a major time sink. In 2007 a U.S. government study found that university faculty members spend about 40 percent of their research time navigating the bureaucratic labyrinth, and the situation is no better in Europe. An experimental physicist at Columbia University says he once calculated that some grants he was seeking had a net negative value: they would not even pay for the time that applicants and peer reviewers spent on them.

    The editorial discusses in greater length the problem that the application process encourages derivative work with predictable results. I think this is a problem of peer review for grants that doesn't hold for papers. A fundamentally new idea cannot (and should not) survive review without supporting evidence. A paper can provide the evidence because it is reporting on work that has been done. But a grant application may have no results to show. No results, no money.

    The editorial proposes that funders should reward people instead of projects. I have some sympathy for that idea. But the promising people are not always the ones with the longest publication records.

  • Mailbag: Virtually there

    Sun, 2011-01-09 21:21 -- John Hawks

    Re: Second Life grant reviews

    I have online acquaintances who are second lifers, so I have looked into it on their recommendation.  I find 1) it’s basically creepy, and 2) has a pretty steep learning curve for proficiency in living/negotiating the 2nd life. I can’t imagine being trapped in front of the computer in this for two days on a panel – yuck!

    However, what concerns me most about this trend toward the virtual in NSF is the proposal. Are we now to prepare documents intended to be viewed in the virtual rather than closely read in real time? Will those who digitally, “get it,”  fare better in reviews than the old and fusty? Jeeze, it’s always a crap shoot anyway, and now we have to worry about this?

    Well, I guess if digital savvy will help in this system, maybe I'll have a chance at last!

    Yeah, I don't see how there is even the pretense of accessibility for a diverse pool with "virtual" meetings, at least with the current offerings.

  • Second grant life

    Fri, 2011-01-07 16:46 -- John Hawks

    Would you participate in a "virtual" NSF review panel in Second Life?

    As John Bohannon describes, NSF has been running them for two years, saving $10,000 per panel in the process ("Meeting for Peer Review at a Resort That's Virtually Free"):

    Since March 2009, six grant-review panels have convened on NSF's island in Second Life, known as IISLand. “Realworld panelists are provided with some resources,” says Bainbridge. “So it was felt appropriate to provide them with the cost of a decent set of virtual clothes.” Once the scientists had created avatars, they each received 1000 Linden dollars—which cost NSF $4—to shop in Second Life's virtual stores. (They also received a $240 honorarium of real money per day.)

    Yeah, I've got to think that's going to skew the pool of reviewers. I don't see myself sitting around for two days on a virtual panel. For many good scientists, I have to think this would impede their ability to contribute productively.

    The sheer amount of manpower involved in grant review is amazing:

    Over the past year, more than 19,000 scientists traveled to NSF headquarters in Arlington, Virginia, to take part in traditional review panels. Most of them worked for two solid days, huddled in groups of six to 10, carefully reading, discussing, and scoring dozens of research proposals competing for over $6 billion in grants. It's the scientific equivalent of jury duty. NSF covers expenses, but the small honorarium—typically $500—hardly covers a scientist's time, especially considering the days lost to travel. But how else can NSF evaluate the merits of all those proposals?

    And that's just for the panels who evaluate applications and compare the external reviews. It doesn't count the work of the thousands of external reviewers, who are unpaid and unrewarded.

    I think it would work just as well to find a few objective metrics, retrodict grant success based on those metrics for the past five years, and then use a discriminant function to give out the money. Anything that cuts down on reviews and bureaucracy would net increase productivity. Sure there would be people gaming the system, but there are with the current system!

  • NSF to require data access plan

    Thu, 2010-05-06 12:14 -- John Hawks

    Science Insider reports that the National Science Foundation is going to make a "data management plan" a requirement of every grant application.

    NSF's current policy requires grantees to share their data within a reasonable length of time so long as the cost is modest. "That's nice, but it doesn't have much teeth," said Seidel. Under the new policy, which is expected to be unveiled this fall, a researcher would submit a data management plan as a two-page supplement to any regular grant proposal. That would make it an element of the merit review process.

    NSF wants to avoid a one-size-fits-all approach to the issue, Seidel explained, because each discipline has its own culture about data-sharing. "A scientist might say that my plan is that I don't need one, because I don't save my data," he told the board committee, which has just formed a task force on data policy. "The important thing is that it puts people on notice that they have to think about it, maybe for the first time."

    It sounds to me like it still doesn't "have much teeth." The kind of scientist he describes, who "doesn't need a plan", doesn't need any federal money, either.

    I mean, seriously -- they're going to "put people on notice that they have to think about it"? Give me a break.

  • Mailbag: The Soviet science system

    Wed, 2010-03-03 08:10 -- John Hawks

    Re: "Jobs in American Science":

    Does the U.S. Produce Too Many Scientists?

    Duh! And not only scientists. About an order of magnitude too many bachelors of everything, too.

    instead of being a permanent PI of a small lab, the modal researcher would be a faculty trainer bringing her own funding into a larger lab with multiple workgroups. Senior scientists would succeed by administering larger workgroups; junior scientists could vote with their feet.

    Heh, do you know what you are describing? You are describing a hierarchical structure of governmentally-funded research institute. That's the system I know because I grew up in it - it was a core of the Soviet and German systems. (Bolshevik education reform basically took German system and modified it slightly). No tenures in this system, though - you have to constantly deliver (or do something else to be liked by bureaucrats that give you money). It is, in fact, a very efficient system - contrary to the myth about science in USSR being done under repression in gulags , the success of Soviet physics in the 1950s and 1960s owed primarily to this structure *that was well funded* and attracted enthusiastic best of the best. Has its own problems, of course.

    That's the way I think it should be done: most of the science should be done in research institutes and universities should concentrate on undergrad education, hiring faculty based on teaching abilities to do primarily teaching. (You probably know that in big NIH-drawing fields, even here in UW, teaching qualification is not even among *practical* considerations when hiring/promoting faculty). IMHO, this would results in *enormous savings* on all fronts. But the current system is too entrenched - it will sooner crash and die under its own weight rather than reform.

    Yes, I had in mind the organization of the Max Planck institutes. I hadn't thought of it as a "Soviet" model, which strikes me as rather funny.

    I dislike the idea of giving honchos more power and control, but yet I must say that most of the good research gets funded because some honcho with money uses grant or institutional money "off-label", as it were. Bleeecch, a choice between dictators and bureaucrats!

  • Jobs in American science

    Tue, 2010-03-02 07:30 -- John Hawks

    From Beryl Lieff Benderly in Scientific American's online content: "Does the U.S. Produce Too Many Scientists?"

    This is a long and thoughtful article, which the author describes as a "working draft". It includes many facts counter to the conventional wisdom about U.S. educational outcomes, and a frank discussion of strengths and weaknesses of the scientific funding system.

    Through decisions made haphazardly 60 years ago, “we chose as a country to staff our labs primarily with graduate students and postdocs and a few non-tenured staff people, while other countries have permanent ways of staffing their labs,” often with PhD staff scientists in career positions, says Georgia State University economist Paula Stephan, an authority on the academic labor force. Under some of those other systems, research institutions employ many scientists as long-term, career staff members who have professional-level salaries and clear career paths potentially leading to greater responsibility and leadership.

    A contributing factor:

    Scientists write the grant proposals and do the research, but the grant, which often also provides at least part of the professor’s salary, is technically awarded to the university, which administers it and provides the facilities needed to do the research in return for overhead payments. The limiting factor on young scientists’ abilities to start academic research careers is thus the number of available faculty positions, which over recent decades has fallen farther and farther behind the number of scientists the system is producing.

    Are there independent scientists who might do more with the roughly 50% overhead on each grant now given automatically by contract to university administrations?

    Yes, yes, I know all you graduate students out there start having negative thoughts whenever I post this kind of stuff. To tell you the truth, I think the American system rewards a certain kind of science "entrepreneurship" that on the whole is a good thing. National funding agencies have too much power to dictate what kinds of science get done; we certainly don't need to give them additional power to determine what kinds of job positions will be available.

    But we have to realize that the system selects for a certain kind of scientist -- the kind that works in "minimum publishable units", who is willing to work more hours for a lower salary than people in any other field requiring comparable training.

    A simple suggestion: If we're going to wait to give people federal grants until age 42, can we make the grants contingent on past accomplishment instead of future promises?

    This would reduce a lot of the friction on the demand side of the job market, because institutions would be better able to predict the funding prospects of their candidates. Existing labs could compete to attract young researchers likely to bring dollars back into the lab. That change to the funding structure would alter the job -- instead of being a permanent PI of a small lab, the modal researcher would be a faculty trainer bringing her own funding into a larger lab with multiple workgroups. Senior scientists would succeed by administering larger workgroups; junior scientists could vote with their feet.

  • Driven by curiosity

    Wed, 2010-01-13 07:30 -- John Hawks

    Brit academics ready to emigrate en masse?

    More than a third of academics said they could move abroad if the Government pressed ahead with reforms to cut “curiosity-driven” research.

    Considering that far more than two thirds of grant applications have no evidence of being "curiosity-driven", I wonder if they realize that they're probably net beneficiaries....

Pages

Subscribe to grants

Neandertals

For years, I've worked on their bones. Now I'm working on their genes. Read more about the science studying these ancient people.

Denisova

From a finger bone of an ancient human came the record of a completely unexpected population. My lab is working on the science of the Denisova genome.

Acceleration

The advent of agriculture caused natural selection to speed up greatly in humans. We're uncovering some of the ways that populations have rapidly changed during the last 10,000 years.

Malapa

Just outside Johannesburg, the Malapa site is producing some of the most exciting finds in human evolution. This site is the headquarters of the Malapa Soft Tissue Project.