Beyond the academic: The value of non-academic peer reviewers

In this series, we’re sharing practices of community-engaged research (CER): researchers and community partners working together to advance community goals and science.

Peer review, the process of having your research scrutinized by experts in the same field, is widely regarded as a cornerstone of scientific publication. Peer reviewers are, in a way, gatekeepers of science. They help filter out unreliable or poor-quality science and improve the quality of works that are eventually published.

For the most part, these expert gatekeepers are researchers themselves. When it comes to research involving or having implications for communities, should those experts include non-university-trained researchers with practice, professional, or lived experience, or traditional or local knowledge?

Scientific knowledge is not the only knowledge

Modern-day science “is founded on Western European ideals of knowledge production, which have become very Imperial,” says Dr. Max Liboiron, Associate Professor of Geography at Memorial University. Among other issues, Liboiron explains that this imperialistic outlook has perpetuated the notion that the scientific process is the only way to acquire data and produce knowledge. On the other hand, local and traditional knowledge are considered less robust. “This idea that the scientific method is the only way to get to the ‘truth’  just doesn’t hold up,” says Melanie Slavitch, Director of Integrity and Engagement at Canadian Science Publishing.

Community-engaged research methods, such as community-based research – where researchers and community partners work together to advance community goals and science, are slowly becoming more prominent. Liboiron’s work, for example, is the result “of a full partnership with community groups…they design the project, they determine the research priorities. I’m there because I know how ‘Science’ works, but they’re there to tell us what’s needed.”

The level and nature of community engagement vary depending on the community’s needs, desires, and expertise. Some may wish to be involved at every stage of the process, others only in parts, such as data collection or analysis. Within the Canadian Science Publishing family of journals, at least, communities and community members can be acknowledged as authors on an individual, group, or a more-than-human basis.

“We’re welcoming multiple ways of knowing, but then we’re putting these manuscripts through a peer review process that is very rigid and academic,” says Dr. Lisa Loseto, Editor-in-Chief for the Arctic Science journal and Research Scientist at Fisheries and Oceans Canada. “If [the community] is good enough to help you design and collect, why would they not be good enough to review what’s coming out of it?”

An additive benefit

Non-academic peer reviewers are not intended to replace university-trained and academic reviewers but rather complement their reviews with additional expertise. “We still expect to get rigorous scientific review on the papers, but there are cases where a paper might also benefit from a non-academic perspective,” says Slavitch. As with many academics and university-trained researchers, non-academic peer reviewers aren’t expected to critique all aspects of a paper. “When you invite a statistician to review a work, you do not expect the statistician to comment on absolutely everything. You expect them to comment on the statistics,” Slavitch explains.

One example of where Canadian science could arguably have benefitted from community perspectives throughout the scientific process is the cod collapse in in Newfoundland and Labrador. “To try and modernize the fishery, the Department of Fisheries and Oceans used ecological population science to figure out the quotas,” Liboiron explains. However, some local fishers were concerned with the decisions being made based on the models. “[They] were saying there was something wrong with the fish. They’re smaller. They’re younger. There are fewer of them. The scientists were saying ‘the models say we’re good.’” The eventual cod collapse and subsequent fishery closure, leading to the largest layoff in Canadian history, is now infamous.

Change Islands is a community located off the northeast coast of the Canadian province of Newfoundland and Labrador.

It wasn’t that the science and models were wrong per se, but that data from inshore waters, where local fishermen were active, was not included. The picture was incomplete. “With peer review, you say here are my variables, here is my model, and here is my math, and someone says oops, you forgot a bit of context, which means you didn’t measure something right, or oops, you did your categories wrong. Now your results are skewed,” says Liboiron. Local knowledge, Liboiron notes, can do a good job “at categorization, or figuring out which variables are most important, or the difference between potential causation and correlation.”

Making space for non-academic peer reviewers

During a workshop held at the focused on Indigenous participation in publications and editorial processes, it became clear that just as communities want different levels of involvement in conceiving and doing research, their desire to participate in editorial processes, like peer review, varies. “Some communities said this is your [the scientist’s] world. We don’t need to be part of it. Other Indigenous groups were saying, we’ve never really had the chance to talk about this,” says Loseto. For those non-academics willing to peer review, “we need different tools and support to facilitate the process,” Loseto says.

From a practical perspective, publication and review systems were built with academic and research institutions in mind. Non-academic reviewers cannot fill in oft-required information such as academic affiliation or ORCID. Particularly when a reviewer is from a remote community, they may neither have an email address nor internet. “We have to be flexible, to be willing to go that extra bit further to make room for participants who historically have been disenfranchised or completely uninvolved,” explains Slavitch.

Another important consideration is the well-being of the reviewer. Scientific inquiry has caused harm to Indigenous communities in Canada. Care is needed to prevent further harm. “Giving non-Indigenous authored papers about Indigenous topics to Indigenous reviewers can often be brutal for the Indigenous reviewers, especially if things aren’t done right. We end up reading unintentionally racist things or unintentionally colonial things, and we end up with this pretty horrific labour to try and deal with that,” Liboiron, who is Red River Métis/Michif raised in Lac la Biche, Treaty 6 territory, explains. Liboiron would like to see editors doing a “pre-review to make sure that it hits certain notes properly. Is there a proper position statement? Is there a record of consent? Just basic stuff that [editors] can say yes or no to [before deciding whether or not to send it off for peer review].”

Loseto acknowledges that some authors will feel uncomfortable having a non-academic peer reviewer assess their work. “It’s a vulnerable thing to submit an article and have people who are experts criticize it and basically tell you it needs to be fixed,” Loseto explains. “Now I’m going to make you extra vulnerable by exposing your work to another type of knowledge.”

Engaging more non-academic peer reviewers is a process that will take time. “It won’t be easy, but we’re starting to have the conversations and provide the tools to welcome [non-academic] peer-reviewers into the process,” says Loseto. In the end, it’s not just communities that benefit. It’s knowledge generation itself.

Photo of Change Islands Newfoundland by BeThenPhotos on Flickr.