Git, GitHub and Reproducible Research

These materials make use of proprietary products and a simplified workflow in order to emphasise the main concepts and to save on installation and configuration time. Some references will be given at the end to direct you to free and open source alternative and more sustainable workflows. The core ideas we will see are largely the same regardless of the software you use, so understanding this material should make it much easier to migrate to more sophisticated methods in the future, should the need arise.

A version control system (VCS) is a system to help manage the writing and maintenance of things such as software, documents, and websites. These systems were developed to manage large software projects but are useful at many levels. For example, version control can help you avoid the following situation: my-analysis.R, my-analysis-final.R, my-analysis-final-again.R, and so on. Try coming back to that 3 months¹ later when a reviewer asks you to re-run something with a slight modification! Similar to the way in which an interactive debugger allows you to step through the evaluation of your program, version control allows you to step through the process in which the code was written. You can think of it as an extreme version of undo/redo.

In addition to helping you organise your files, a version control system (VCS) and its associated tooling can help the scientific community by making your research reproducible and open. It is much easier to download the code from GitHub than trying to email the authors to get a copy. For your computational research to be considered reproducible, it needs to be described in such a way that others can replicate your results. For it to be open, the materials (code, data and sufficient documentation to use it) need to be available for others. Simply dumping all of your code into something like GitHub is not sufficient for your research to be considered reproducible.

Created in 2005 by Linus Torvalds² to help with the development of the Linux kernel, git has become a fundamental tool for software development. In the 2021 Stackoverflow Developer Survey over \(90\%\) of respondents used git; it is nearly synonymous with version control³. If you intend to collaborate in the writing of substantial amounts of code, taking the time to learn how to use git is a good idea. This can be made easier by learning to use a GUI interface too.

Working with git will be much easier once you are familiar with some of the terminology (there is a lot, but only some of it is really necessary.) Unless you are familiar with git already, you should at least skim these before continuing.

Zenodo is an open access archive operated by CERN which allows researchers to archive research materials with a DOI⁵ which makes them easier to find and cite. This is a more permanent form of storage than GitHub. It is easy to archive a particular commit of a repository which is good practice if you want to refer to a particular version of some code in a paper. GitHub has tooling for this build in.

Start by creating the relevant files on your own machine to make sure they work. Make a new directory called git-usage to hold our files, will will refine this as we go along. The data and the code that generated this figure are included below. This data should be saved in a called stackoverflow-git-data.csv.

year,percentage
2015,69.3
2017,69.2
2018,87.2
2020,82.8
2021,93.43

We then need a script to carry out the analysis. Save the following code in a file called make-plot.R

library(ggplot2)

sods_data <- read.csv("stackoverflow-git-data.csv")

g <- ggplot(
  data = sods_data,
  mapping = aes(x = year, y = percentage)) +
  geom_point() +
  geom_smooth(method = "lm") +
  geom_text(
    aes(x = 2020, y = 82.8, label = "only GitHub"),
    nudge_x = 0.2,
    nudge_y = -4) +
  labs(
    x = "Year",
    y = "Percentage who used git",
    title = "Git usage has increased",
    subtitle = "Data from Stackoverflow Developer Survey")

ggsave(filename = "git-usage.png",
       plot = g,
       height = 7.4,
       width = 10.5,
       units = "cm")

sink(file = "regression-summary.txt")
summary(lm(percentage ~ year, data = sods_data))
sink()

Once we have run the make-plot.R script, the directory should contain four files and have a structure like the following.

git-usage
+-- git-usage.png
+-- make-plot.R
+-- regression-summary.txt
+-- stackoverflow-git-data.csv

In the next section we will go through cleaning this up so it is easier for people (including yourself in the future) to make sense of this.

As a first step we will use directories to impose a sensible structure to our files. Organising files in this way is useful as it makes it far easier for someone to understand what each file is needed for. Follow the following steps (starting from within git-usage,) to organise your code more appropriately:

1. Make a directory called src and move make-plot.R there.
2. Make a directory called data and move stackoverflow-git-data.csv there.
3. Make a directory called out which we will write results to.
4. Fix the call to read.csv in the script make-plot.R so it can find the CSV, which now lives in the data directory.
5. Fix the calls to ggsave and sink so that they write their output to the out directory.

Once you have done this, the R script should look like the following.

sods_data <- read.csv("data/stackoverflow-git-data.csv")

...

ggsave(filename = "out/git-usage.png",
       plot = g,
       height = 7.4,
       width = 10.5,
       units = "cm")

sink(file = "out/regression-summary.txt")
summary(lm(percentage ~ year, data = sods_data))
sink()

Once you have run the code (with git-usage as your working directory), the directory structure should look like the following. Note how the output files now appear in the out directory. If you are running the script from an R REPL (for example through RStudio), remember you can use setwd to specify the working directory.

git-usage
+-- data
|   +-- stackoverflow-git-data.csv
+-- out
|   +-- git-usage.png
|   +-- regression-summary.txt
+-- src
    +-- make-plot.R

Now that our code is in a reasonable state, we can upload it to GitHub. If you do not already have a GitHub account, please follow the instructions above, which describe how to make one. Once you have a GitHub account, you can follow the following steps to upload these files:

1. Visit https://github.com/ and create a new repository by clicking New (or Create repository), you will need to pick a name for the repository (I called mine git-usage.) There will be a couple of options but the default settings provided by GitHub are fine. Click Create repository.

2. We now need to commit our files and push them to the remote repository. However, since we are doing this through GitHub, it is all combined into a single step. Click Add file (or creating a new file) to start the process of adding the src/make-plot.R file.

   1. Ensure the name of the file is git-usage/src/make-plot.R (be careful that you have the path with the / correct.)
   2. Copy-and-paste the code in make-plot.R into the text box provided.
   3. Click Commit new file button. Congratulations you have made your first commit!🎊

   If you are struggling to make a new directory in GitHub, see the next section.

3. Repeat this process with data/stackoverflow-git-data.csv and the output TXT file. In the case of the PNG image, git-usage.png, you will need to use Upload file instead of Create new file. To find these, click the Add file button in the Code tab.

You cannot add an empty directory to a GitHub directory. If you want a new directory to be added, you need to commit a file to it. One convention for this is to make an empty file in the desired directory (often this will be a file called .gitkeep). To make this file use Create new file and just leave the text box empty.

A license specifies what people can do with your code. If you aren't sure what license suits your needs, you might find https://choosealicense.com/ has some helpful information. Most of the time, I will opt for the MIT license.

There are two ways you might add a license. The manual method is to copy and paste the license text into a file called LICENSE to your repository, filling in [year] and [fullname] as appropriate. Alternatively, you can Add file and Create new file and specify that the file will be called "LICENSE" and it will offer you some templates to choose from. It will auto-fill the details of your name and the year.

When you encounter a repository online it can be difficult to understand what its purpose is and how to use it. "README" is the name given to a file that contains this sort of information. Typically these will be written in markdown (similar to RMarkdown). Add a file called README.md to your repository with text similar to the following.

This repository contains an analysis of git usage through time.

To run this analysis use the following command:

```
Rscript src/make-plot.R
```

The input data is in `data` and the results are in `out`.

Software gets updated, and sometimes these updates cause things to break. Where possible, it is very good practise to include details of the versions of software you have used. When working with R the sessionInfo command makes this simple. Try adding the following to the end of the make-plot.R script.

sink(file = "out/package-versions.txt")
sessionInfo()
sink()

The next time that you run this script, it will write a description of the version of R you used and the versions of all the loaded packages to the file out/package-versions.txt. Try running the script again to make sure this additional file was generated and contains something similar to the following.

R version 4.1.2 (2021-11-01)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04.3 LTS

Matrix products: default
BLAS:   /usr/local/lib/R/lib/libRblas.so
LAPACK: /usr/local/lib/R/lib/libRlapack.so

locale:
 [1] LC_CTYPE=en_GB.UTF-8       LC_NUMERIC=C
 [3] LC_TIME=en_GB.UTF-8        LC_COLLATE=en_GB.UTF-8
 [5] LC_MONETARY=en_GB.UTF-8    LC_MESSAGES=en_GB.UTF-8
 [7] LC_PAPER=en_GB.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] ggplot2_3.3.5

loaded via a namespace (and not attached):
 [1] magrittr_2.0.1   splines_4.1.2    tidyselect_1.1.1 munsell_0.5.0
 [5] colorspace_2.0-2 lattice_0.20-45  R6_2.5.1         rlang_0.4.12
 [9] fansi_0.5.0      dplyr_1.0.7      tools_4.1.2      grid_4.1.2
[13] gtable_0.3.0     nlme_3.1-153     mgcv_1.8-38      utf8_1.2.2
[17] withr_2.4.3      ellipsis_0.3.2   digest_0.6.29    tibble_3.1.6
[21] lifecycle_1.0.1  crayon_1.4.2     Matrix_1.3-4     farver_2.1.0
[25] purrr_0.3.4      vctrs_0.3.8      glue_1.6.0       labeling_0.4.2
[29] compiler_4.1.2   pillar_1.6.4     generics_0.1.1   scales_1.1.1
[33] pkgconfig_2.0.3

Once you are happy that this has worked, we need to commit these changes. First by editing the script, and second, add the package-versions.txt file.

Suppose that after doing all of this one of your collaborators wants to adjust the figure. We will now go through the steps involved with doing this using branches.

If you want to download the files from GitHub and do not want any of the associated git functionality, you can download a ZIP file that contains the contents of a repository. Figure 8 shows the menu for downloading a ZIP file containing the contents of a repository. If you want to use any of git's features though you should clone the repository instead.

We have used the R programming language in this tutorial, but how we use GitHub is language agnostic (for the most part). Most of the time, organising your data and source code in this way is a good idea. Different programming languages record their package versions in different ways. Recall in this section that we generated a file containing the package versions. For the Python language, there is the pip function. Running pip freeze at the command line will print out the package versions. You can pipe this information to a text file (which is conventionally called requirements.txt) with the following command:

pip freeze > requirements.txt

If you want to use multiple Python packages, it's a good idea to invest time into learning how to set up a virtual environment. A virtual environment is an isolated group of packages used for a project. If you want understand how to do this, see the relevant python documentation.

The Zenodo FAQs contain information about how to archive a GitHub repository if you want a more permanent form of storage. Ideally, one would archive the commit used to generate the contents of a manuscript so it has a DOI and reference both the archive and the live version of the code on GitHub in the manuscript.

• Pro Git by Scott Chacon and Ben Straub is a free book that is the ultimate guide but is a bit technical at times.
• Atlassian/Bitbucket has excellent tutorials.
• Learn Git Branching is a game revolving around explaining git.
• GitHub Learning Lab has some introductory material on the use of git and GitHub.
• Stackoverflow questions will often have answers to your questions.
• Inside the Hidden Git Folder - Computerphile gives a bit of a behind the scenes tour of how git works.

There are lots of features in GitHub that haven't been covered but may be worth looking into:

• the issue tracker,
• the wiki,
• VSCode integration,
• GitHub Pages⁶,
• and GitHub Actions.

Explain (in 50–150 words) how the git, GitHub, and Zenodo complement each other and their respective roles. Describe the value of one of the additional features of GitHub not covered in this tutorial (in 50–100 words).

Explain (in 100–150 words) the function of version control in reproducible research. Give an example (in 50–100 words) where version control alone does not suffice to make a piece of work reproducible.

This question will test your ability to organise the artefacts of a computational project. Download the scripts and data files using the links below and run them. Organise the files you have downloaded and the results of running them in an appropriately structured GitHub repository. Give a brief overview of the decisions you made along the way (in 100–200 words). Once you are happy with this, download a ZIP file for this repository and include it as part of your submission.

• Data description
• Melbourne rainfall data
• Oxford rainfall data
• First R script
• Second R script

If you cannot download these files directly, they should also be available here.

This repository contains an attempt at visualising two datasets. Unfortunately, a bug was introduced somewhere during that attempt. The attempt consisted of the following steps:

1. Plotting the data in iris.csv using make-fig-a.R.
2. Plotting the data in mtcars-renamed.csv using make-fig-b.R
3. Realising that there is duplicated code and refactoring it: reshape-data-function.R.
4. Tweaking the figures to make them clearer.
5. And finally, realising that something is wrong with fig-a.png!

Referring to the commit history of the repository, answer the following questions (in 150–250 words total):

1. What bug was introduced, and how would you fix it?
2. How did you find the bug? How would you do this if the bug was subtle and there were hundreds of files and thousands of commits?
3. Could you have prevented this bug by doing something differently?

Read the editorial Ten Simple Rules for Reproducible Computational Research and (in 150–250 words total) give a brief explanation of how git and GitHub would or would not be relevant to each rule.