Beyond criticism: Creating a collective future for peer review

Over the course of his career, Steven J. Cooke, an esteemed biologist, editor, and Canada Research Professor, has watched the public’s trust in science erode. 

The last decade has been difficult for scientists. Medicine has faced increased skepticism in the wake of a global pandemic, climate science has been hotly politicized even as record-breaking wildfires burn, and science communicators wage an endless battle against viral videos packed with misinformation. In scientific publishing, generative AI models are causing new problems, with publishers rapidly adapting to combat an influx of computer-generated content from paper mills.

Against this backdrop, Cooke has been investigating peer review, a cornerstone of the scientific process by which scientists critique each other’s work—a system that many argue is in crisis. Major publishers have been rocked by peer review-related scandals, with cases of bias, fabrication, and ineffectiveness eagerly discussed and widely debated on social media. But Cooke recognizes that peer review is facing a deeper challenge—the erosion of trust in institutions, experts, and the scientific process itself. 

“Sometimes you see a single tweet that sparks an idea and leads to a paper. This was more the result of witnessing a confluence of events over years,” Cooke reflects. 

After years of collaboration, Cooke and sixteen co-authors have produced a new paper, “A harm reduction approach to improving peer review by acknowledging its imperfections,” which reimagines peer review through twelve pre- and post-publication reforms.

At a time when misinformation and “truthiness” spread faster than fact and when trust in science is at an all-time low, Cooke et al. offer a refreshing vision for the future of science. With his deep involvement in academic publishing and a clear passion for improving the system, Cooke offers his insights on the challenges, successes, and the path forward for peer review. Cooke’s reflections dive into the heart of peer review, a system often criticized, but equally indispensable.  

When analyzing the current state of peer review, Cooke didn’t want to add to the pile of one-sided criticism. “That’s a problem-oriented perspective. We need a solutions-oriented perspective. We spend a lot of time talking about the problems and not a lot of time thinking about rational solutions. We needed to take a big step back and have an actual discussion.”  

Cooke looked outside of the natural sciences for a broader perspective, referencing the literature on crises of trust and bringing together authors from various disciplines ranging from chemistry to English literature. Indeed, in the very first line of the abstract, the paper declares that it is a “candid perspective written by scholars from diverse disciplinary backgrounds”.

What makes Cooke’s perspective particularly compelling is his focus on the need for more science on peer review. Instead of making ad hoc changes to how journals operate, he advocates for research-driven reforms. “We need to turn the peer review process itself into an experiment,” he says, calling for evidence-based decisions on whether models like double-blind peer review truly achieve their intended goals. 

In his view, changes to the peer review system should be deliberate, transparent, and constantly revisited based on empirical data. “We might get it wrong,” Cooke admits. “But that’s part of the process. We’re going to learn, reflect, and adapt, and we’re going to share so that others can learn from our experiences and bring it to their decision-making around how they’re going to operate a given journal or publisher.” Change isn’t a sign that the science is broken—it’s a sign that the science is working. 

For Cooke and his co-authors, peer review is about much more than just producing polished papers—it’s about collaborating as a community to build better scientists. He suggests that thoughtful feedback from referees can shape future studies, creating a ripple effect that advances the field as a whole. And while peer review is a volunteer-driven process, Cooke makes a point of highlighting its often underappreciated labour.  

“If a reviewer wanted to tank a paper, they could do it in one page or less,” he notes. “But someone who takes the time to write a 10-page review is invested in making the research better.”

We tend to forget that science—and peer review—are human endeavours, and therefore subject to our human foibles of ambition, greed, or the simple lack of judgment that might arise from a lack of sleep or a bad afternoon. These human failings are impossible to erase from the system; as Cooke et al. write, “No amount of self-policing, transparency, or reform to peer review will eliminate all bad actors, unscrupulous publishers, perverse incentives that reward cutting corners, intentional deception, or bias.” As a result, some wonder why, in the era of ChatGPT, we shouldn’t just scrap peer review altogether.  

For Cooke, this is entirely the wrong approach. Yes, people are the source of our frustrations with peer review. But they’re also at the heart of what makes peer review powerful. “In this case, you can’t separate the wheat from the chaff,” he says. For Cooke, peer-reviewed journals are more than just vehicles for publishing research—they are communities. 

Peer review is, at its core, a community endeavour, where colleagues across career stages voluntarily take time to read, consider, and respond to each other’s work. Those being reviewed get feedback on how to improve their research, while those reviewing hone their critical thinking skills. As Cooke notes, his work as a reviewer has made him a better colleague and researcher. “As a reviewer, you need to put your critical thinking hat on,” he explains.  

Cooke envisions journals as spaces where early-career scholars and novice authors can grow, where training opportunities, webinars, and blog posts enhance the academic experience. Many of the recommendations in Cooke et al.’s article focus on the need for more training opportunities for reviewers and editors. While some of the problems in peer review come down to intentional “bad actors,” many arise simply from the near-total “absence of mandatory training in peer review and editing.”  

“I’ve always viewed my role as a reviewer and editor as having a training element to it,” says Cooke. “I certainly shouldn’t be deciding which topics are worthy of publishing and which ones aren’t. I should make sure that, for studies that are somewhat imperfect—which is pretty much all of them—that there’s honesty and transparency… I can’t necessarily make them go back and redo that study, but their interactions with me in our editorial process may help them do better science in the future—not just to be better at sharing it, but to actually help them think about how they’re going to do their next study, to improve upon the experimental design, and so on.” 

FACETS, where Cooke is a subject editor, is a prime example of this shift. The journal has officially broadened its scope to embrace research in the social sciences, though Cooke notes that the journal had already been publishing work in this area. “Sometimes it’s just about planting a flag to make it official,” he says. To support this newly broadened scope, FACETS and the Royal Society of Canada (RSC) are building a new reviewer board of RSC members, illustrating how journals can evolve to address peer review challenges by gathering community. 

“Journals can and should be a community of practice,” he muses, acknowledging that this sense of community doesn’t happen by accident. It requires purposeful effort from editors, reviewers, and publishers alike.  

The Canadian Journal of Fisheries and Aquatic Sciences, where Cooke has also been a longtime contributor, has recently launched an early-career reviewer program in collaboration with the Society of Canadian Aquatic Sciences (SCAS), which aims to give junior researchers hands-on experience with the mentorship they need to become the ethical, influential reviewers of the future. 

In an academic landscape that can sometimes feel cutthroat, Cooke’s vision is refreshingly collaborative and forward-thinking. Peer review has its problems, but it’s a system worth saving. Through peer review, Cooke believes, we can build not only better science but also better communities.

Read Cooke et al.’s article and discover his 12 recommendations for peer review at FACETS or on ResearchGate. Join the conversation about peer review with us on X (Twitter), LinkedIn, or Facebook. Tell us how you envision the future of peer review.