Modules
The HPC is used by many people with diverse software requirements. You might require Python 3.7 for your script, and your colleague requires Python 2.7 for their script. How do we install both on the HPC?
Software Modules are used on the HPC to solve this issue of installing conflicting software. A software module is a partially installed version of the software. When you need want to use this software, you need to activate so it is available to you. Activation “completes” the install in a temporary manner. When no longer required, the module can be removed.
...
| Code Block |
|---|
module list module purge module list |
Module conflicts can occur when loading modules.
You cannot two different versions of the same package:
| Code Block |
|---|
# Load R version 3.6.2
module load r/3.6.2-foss-2019b
R --version
# Load R version 4.0.3
module load r/4.0.3-foss-2020b
{Error} |
Tool chain conflicts.
In the name r/4.0.3-foss-2020b, the foss-2020b part is the toolchain. This means “Free and Open Source Software” at version 2020b.
...
| Code Block |
|---|
module purge module load samtools/1.9-foss-2017a module load bowtie2/2.3.4.3-foss-2017a module list |
Installing Software
Conda:
Conda is a package manager and can be used to install packages to your home folder.
...
You use environments to separate package versions.
The conda modules can be displayed with:
| Code Block |
|---|
module avail anaconda |
Or a conda module with Mamba is available:
| Code Block | ||
|---|---|---|
|
...
module avail mamba |
| Code Block |
|---|
-------------------------------------------------------------------------------------------- /pkg/suse12/modules/all --------------------------------------------------------------------------------------------
mamba/4.14.0-0 |
When you use a conda module, it is a good idea to run the 'conda init' command to update your shell files so conda functions correctly.
Alternatively, you can install miniconda to your home folder by following the instructions. Be sure to choose 'yes' the update shell question.
Once you have run conda init, or installed Minconda, you should logout and log back in again to activate the shell changes.
See here for details:
HPC Software - Conda - QUT MediaHub
Using Singularity
Singularity runs software containers. A container is a packaged collection of software. The advantage of using a container is it is portable, and runs without installation.
By Hand
It is possible to download application source code, compile, and install to your home folder.
This is an advanced topic!
Start by module loading the compiler and tool chain you need to build the software…
...