Writing replicable and reproducible methods I: The basics

On those days when my brain needs a reset, I sometimes like to browse the archives of journals. Take the Canadian Journal of Fisheries of Aquatic Sciences (CJFAS), in publication since 1901. When I happen upon a paper with photos of an experimental setup that ought to be in a museum, I feel something akin to nostalgia, but for a time and place I didn’t experience.  

In a pair of 1960s papers in CJFAS, John Raymond (J.R.) Brett described the design and use of a water tunnel to measure oxygen consumption of swimming salmon. Figure 1 in each paper stands out in its own way. One is a photograph, offering a glimpse into the past. The other is a drawing of the custom-built tunnel, which would have been helpful to anyone planning to replicate this setup with a different species of fish.  

Including figures that show how you did the research is one of several ways to prepare methods that can be replicated or reproduced.

Brett 1965 https://doi.org/10.1139/f65-128  Brett 1964 https://doi.org/10.1139/f64-103 

Keep reading for tips on how to prepare methods for replication and reproduction.

1. Write early and update throughout the study

Doing science isn’t always straightforward. It’s prudent to start writing your methods early and update them throughout the research process so that unexpected changes and their justification are recorded (not forgotten!) and reported.

You’ll be writing your methods early if you plan to submit your research plans for peer review, also referred to as preregistration, whether for an original study (e.g., data collection methods) or synthesizing existing studies (e.g., literature search methods). This a priori approach “decreases the risk of unintentional project drift or scope expansion, enables early opportunities to identify problems, and allows for early feedback,” says Dr. Trina Rytwinski of the Canadian Centre for Evidence-Informed Conservation.

2. Report randomization and sources of materials and variation  

Whether you’re using a limited-edition version of a drone or a familiar brand of pipette tips, provide the company name and other pertinent information when listing materials (including software) so that researchers can secure the same or similar products. 

Being meticulous about steps that are expected sources of variability is crucial for robust methods. This includes details such as a designated time of day to sample a particular strain of research animal, or a defined duration between sample collection in the wild and sample processing in the lab. 

Rytwinski stresses the need to report whether randomization of experimental treatments was used and, if so, what type (e.g., random, block). Randomization is a procedural step, so including this detail supports replication and affects which statistical tests are appropriate to use.  

3. Publish a Methods paper to focus a reader’s attention    

To provide readers with as much detail as possible, consider publishing in journals that offer manuscript types specifically for reporting new or modified methods and protocols. PhD student Saurabh Tiwari at the University of Calgary was motivated to write a Methods paper in Biochemistry and Cell Biology after struggling to replicate published RNA isolation methods that only cited a manufacturer’s proprietary protocol.

4. Be transparent about what did not work  

Well-written methods can help mitigate limitations like funding, time, and available personnel, especially if they include conditions that did not work. Tiwari notes how authors typically do not explain which experiments failed or why.  A section that transparently reports failed protocols or protocol adjustments “could spare other[s] from optimizing them again,” says Tiwari.  

5. Disclose all use of AI-assisted tools   

Generative AI technology is being used in scientific research and reporting. To allow study replication, describe the use of AI-assisted tools in the methods with the same level of detail as for physical equipment or computational software. Include the name and model/version of the AI-assisted tools and describe how they were used to design or conduct research or data analyses. Consider including input prompts and AI outputs as supplementary files. 

As AI tools and their applications evolve, so too will publisher policies; Canadian Science Publishing’s current AI Policy can be found here.

6. Use figures or videos to enhance reader understanding  

Visual documentation can help readers better interpret written methods. A statistical approach can be visualized with decision trees, or an experimental technique through video. Maps presenting locational data are often essential for replication or to meet inclusion criteria for review papers. 

7. Use discipline-specific reporting guidelines 

Whether you’re writing up a clinical trial, wildlife tracking study, chemical synthesis, or drone auxiliary sensor design, be sure to reference up-to-date reporting guidelines. The EQUATOR Network’s library of guidelines and ARRIVE checklist are good places to start. You’ll also want to meet the reporting requirements of the journal you’re publishing with.

8. Include details about community-engagement processes 

When doing research that involves community partners, it is a priority to report the methods in a way that does not exploit the groups involved. Collaborative writing can help ensure that socio-cultural aspects of the engagement process are accurately presented in the methods. Refer to community-specific guidelines when possible (e.g., CONSIDER checklist for reporting health research involving Indigenous Peoples). 

Replicating and reproducing work is meant to strengthen the scientific record and trust in that record. It can also be an enormous undertaking.   

But there is value in teaching researchers how to report methods in sufficient detail to make replication and reproducibility possible. Early-career researchers, in particular, can gain “robust, transferable skills that improve transparency and usability of their work, making it easier for other researchers to understand, evaluate, and build upon. These practices can also facilitate collaboration, reduce avoidable errors, and contribute to the long-term impact of their research,” says Dr. Marija Purgar, President of the Society for Open, Reliable, and Transparent Ecology and Evolutionary Biology (SORTEE).

Thank you to all community partners, subject-experts, authors, and CSP staff whose knowledge and feedback helped build this blog post, especially Dr. Trina Rytwinski and staff at the Canadian Centre for Evidence-Informed Conservation, Dr. Marija Purgar, Saurabh Tiwari, Dr. Jacob Thundathil, Dr. Chris Rooper, Dr. Becky Furlong, Hilary Belleville, Bruce Patten, and Rebecca Michaels-Walker.