Installing Packages

Custom software can be installed into your home directory on H4H. This is useful to augment existing modules available via the module system or even to compile your own custom software libraries or languages. Note, however, that you only have 50 GB of disk space in your personal directory and thus should use existing system modules when they are available!

Build Node

While the Login Nodes do have access to the internet, you are NOT allowed to install packages from them. Instead, the cluster administrators provide a special internet connected build partition where you can install software libraries or compile programs.

Try it yourself

1. Allocate an interactive job on the build parition (Note: can only use 1 CPU and max 12 GB of RAM!)

   Solution

   salloc -p build -c 1 --mem=10G -t 0:30:0
   

2. Load an R module and install the ggplot2 package

   Solution

   module load R
   R  # you need to do this interactively if you haven't setup a local package directory and selected a default CRAN mirror
   # The following should be run in R
   install.packages("ggplot2")  # requires user input
   

3. Check that the installed package is available

   Solution

   R -e "require('ggplot2') || stop('package not installed!')"
   

Installing and Using Miniconda

Since H4H users do not have sudo privileges, it is not possible to install system dependencies directly via the CentOS package manager yum. If a system dependency for one of the packages in your analysis is unavailable or the incorrect version, the miniconda utility can be used to install it locally in your $HOME directory. The tool can also be used to install a range of other software including programming languages (such as R or Julia) as well as Python and R packages.

Use of virtual environments in general enhances the reproducibility of your analysis and is encouraged. But you should be mindful that all dependencies installed via conda will be in your $HOME directory and thus count against your 50 GB drive quota.

Try it yourself

1. Allocate an interactive job on the build parition

   Solution

   salloc -p build -c 1 --mem=10G -t 1:0:0
   

2. Visit https://docs.conda.io/en/latest/miniconda.html and copy the link for the latest Miniconda installer script on Linux then download the script using wget or curl

   Solution

   wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
   

3. Run the installer script and configure your conda installation (Note: make sure you say yes to running conda init or the installation won't be activated)

   Solution

   bash Miniconda3-latest-Linux-x86_64.sh
   

4. Source your ~/.bashrc file to update your shell session to use conda

   Solution

   source ~/.bashrc  # You should now see (base) prepended to your shell prompt
   

5. Create and activate a new conda environment called test_env to install packages into

   Solution

   conda create -n test_env
   conda activate test_env
   

6. Install R into test_env using the conda-forge channel (Hint: use the r-base package)

   Solution

   # this will take about 10 minutes
   conda install -c conda-forge r-base
   

7. Print the path to R

   Solution

   which R  # path should be to ~/miniconda3 directory
   

8. Delete the test_env conda environment to clean up your $HOME directory

   Solution

   conda deactivate
   conda env remove -n test_env
   

References

1. UHN Bioinformatics and HPC Core Intro
2. Miniconda
3. PyTorch