Farm Software Installation

How to find and install software on Farm (and Hive, which uses the same system). Official docs: software, modules, conda.

Install and build things in an interactive job (srun -A jrigrp -p high -t 2:00:00 -c 4 --mem=16G --pty bash -l), not on the login node.

1. Check for a module first

A lot of software is already installed as modules.

module avail                  # everything (long)
module avail samtools         # search by name
module load samtools          # load the default version
module load R/4.4.2           # or a specific version
module list                   # what's loaded
module unload samtools

(Avoid module purge: it also unloads the slurm module, which breaks squeue/sbatch until you log in again.)

Many bioinformatics tools are packaged as conda/<tool> modules (e.g. module load conda/nextflow). Try module avail conda/ followed by the tool name.

Load modules inside your batch scripts. Jobs start with a clean environment.

If something widely useful is missing, request it with the software installation request form. Packages available in Spack are the easiest for them to add.

2. Conda / mamba for your own environments

Use the central conda install. Don't install your own Miniconda/Anaconda in $HOME (HPC@UCD doesn't support that, and it fills your home quota). If you have old conda init lines in ~/.bashrc, remove them.

module load conda
mamba create -p /quobyte/jrigrp/USERNAME/envs/myenv -c conda-forge -c bioconda python=3.12 pysam
conda activate /quobyte/jrigrp/USERNAME/envs/myenv

• Put environments on /quobyte/jrigrp with -p (prefix). They're big, and /home is only 20 GB.
• Use mamba for installs; it's much faster than conda.
• Save an environment.yml (conda env export > environment.yml) with each project so the environment can be rebuilt.

In a batch script:

module load conda
conda activate /quobyte/jrigrp/USERNAME/envs/myenv

If you use set -u in scripts, wrap the activate line in set +u / set -u. Activate scripts reference unset variables.

3. R packages

module load R/4.4.2
R
> install.packages("data.table")   # goes to your personal library

R installs into ~/R/... by default. Set R_LIBS_USER to a directory on /quobyte/jrigrp/USERNAME/ in ~/.Renviron if your home fills up. RStudio is available through OnDemand. Use Hive's if you need lab data, since Farm's OnDemand can't see /quobyte.

4. Compiling from source

You can't install into system directories, so install under a prefix you own:

module load gcc
./configure --prefix=$HOME/local      # or a path on /quobyte
make -j $SLURM_CPUS_PER_TASK
make install

Then add it to your PATH in ~/.bashrc:

export PATH=$HOME/local/bin:$PATH

The same idea applies to tools cloned from GitHub: git clone, follow the build instructions, and put the binaries in your ~/local/bin. Java .jar files need no install: module load openjdk and run java -Xmx8g -jar tool.jar. Set -Xmx below your job's --mem.

5. Containers

Apptainer (Singularity) is available for tools that ship as Docker/Singularity images:

module load apptainer
apptainer exec docker://quay.io/biocontainers/samtools:1.20--h50ea8bc_0 samtools --version