An ethnographic look at peer review recommends some big changes in training

Gemma Derrick in Nature: “Take peer pressure out of peer review”.

Derrick has done research in the U.K. including direct observation of peer review panels in action. Her Nature essay focuses upon a recent effort to include non-academic voices in grant panels to broaden the representation of the public in funding decisions. In her telling, academics are not shy to hijack the proceedings.

My own and others’ observations show that a peer-review panel is not like some collaborative mural, where everyone contributes a piece to the picture. It is more like a tug of war — with a rope that has many ends. Evaluators form alliances and join various ends of the rope. This sets the panel’s dominant mode for dictating how all proposals are assessed. Those outside this framework are quickly silenced, even if they were recruited for their perspective.

She encourages pre-evaluation training for all participants in peer review. I think this is a helpful suggestion. Discussing implicit bias, having referees discuss the role of public investment in diversifying and strengthening scientific inquiry, and giving explicit credit to effective collaborations.

I would make further suggestions, but the essay is well worth reading and pondering. I believe that public participation in grant review can help to build stronger support for scientific work. I think this process should be more open and responsive to public input, and that the voices of non-specialists should be taken seriously.

Anthropology would be in a much stronger position in the U.S. if we collaborated to develop a Board of Visitors or equivalent group for each of our departments and organizations, and organized our events to broaden the engagement of the public in anthropological research.