Reproducible workflows

Most science is bad

Thanks to bad incentives

“Today I wouldn’t get an academic job. It’s as simple as that. I don’t think I would be regarded as productive enough” Peter Higgs (The Guardian, 6 Dec 2013)

“I’ve been on a number of search committees. I don’t remember anybody looking at anybody’s papers. Number and IF [impact factor] of pubs are what counts.” Terry McGlynn (realscientists) (21 October 2015, 4:12 p.m. Tweet.)

So many examples

• Brain-wide association studies, median sample size: 25, sample size needed to be robust: >1000 (Marek et al. 2022)
• That time a tower of over 400 false studies was conducted around 5-HTTLPR and depression, good summary here (Border et al. 2019)
• There are lots of great selfish reasons to purse reproducibly too! (Markowetz 2015)
• And of course, the somewhat recent seminal amyloid paper that is suspected to be dodgy (Lesné et al. 2006)

Moving beyond idealogy

• I propose we work towards building a practical workflow for embedding reproducibility throughout all projects:
  • Built around a GitHub organisation (Either at the centre or group level)
  • Employ self-documented templates for consistent folder structure
  • Agree on standardised file naming conventions (descriptive, human and machine readable, sane default ordering and search)

An example - you have a new hypothesis

1. Copy the template to a new repository
2. Begin planning: Use Project boards for building timelines and assigning tasks/deadlines to specific users (students/post-docs)
3. First aim for a registered report
4. Use Issues to track ideas, tasks and troubleshooting (protocols)
5. Use literate programming approach (Quarto, Jupyter Notebooks, Org-mode, etc) for writing all documentation (notes, papers, slides, etc)

Incorporating funding

• If funding isn’t already in place, one could use the registered report and repo to showcase budgets, timelines, lay summaries on your own terms (no need to adapt to asinine funder-specific structures)
• Begin experiments as normal, being sure to continue using the GitHub repo to update progress (tasks completed, delays, troubleshooting, etc.)
• Keep project contributions of any size logged on the repo (acknowledgement for technicians/RAs and others)

Write up and publish

• Write up paper and publish as normal (ideally avoiding for-profit journals, and sticking to diamond open access)
• Share raw data either on GitHub repo, or provide links to location on other repositories (and ideally pre-processed data)
• Provide container with computational environment used for analysis (Docker/Singularity)