Content Comparison

...

qsub -I -S /bin/bash -l walltime=10:00:00 -l select=1:ncpus=1:mem=4gb

We will load java next.

module load java

• Line 1: The module load command is necessary to ensure java is available

You should be in your home directory, if unsure you can run the following command:

cd ~

Installing Nextflow for the first time

Important: Please note if you have already installed Nextflow before on the HPC, then go down to the Updating Nextflow section.

To install Nextflow for the first time, copy and paste the following block of code into your terminal (i.e., PuTTy that is already connected to the terminal) and hit 'enter':

module load java
curl -s https://get.nextflow.io | bash
mv nextflow $HOME/bin

• Line 1: The module load command is necessary to ensure java is availableLine 2: This command downloads and assembles the parts of nextflow - this step might take some time.

• Line 32: When finished, the nextflow binary will be in the current folder so it should be moved to your “bin” folder” so it can be found later.

Updating Nextflow

If you have installed Nextflow before on the HPC then you will have to run:

nextflow self-update
nextflow run hello

To verify that Nextflow is installed properly, you can run the following command:

...

mkdir $HOME/nftemp && cd $HOME/nftemp
nextflow run hello

• Line 1: Make a temporary folder for Nextflow to create files when it runs.

• Line 2: Verify Nextflow is working.

You should see something like this:

...

cd $HOME 
rm -rf nftemp

Troubleshooting:

If you have installed Nextflow before on the HPC then you will have to run:

nextflow self-update
nextflow run hello

Nextflow’s base configuration

...

[[ -d $HOME/.nextflow ]] || mkdir -p $HOME/.nextflow

cat <<EOF > $HOME/.nextflow/config
singularity {
    cacheDir = '$HOME/.nextflow/NXF_SINGULARITY_CACHEDIR'
    autoMounts = true
}
conda {
    cacheDir = '$HOME/.nextflow/NXF_CONDA_CACHEDIR'
}
process {
  executor = 'pbspro'
  scratch = false
  cleanup = false
}
includeConfig '/work/datasets/reference/nextflow/qutgenome.config'
EOF

• Line 1: Check if a .nextflow/config file already exists in your home directory. Create it if it does not exist

• Line 2-15: Using the cat command, paste text in the newly created .nextflow/config file which specifies the cache location for your singularity and conda.

• What are the parameters you are setting?

• Line 3-6 set the directory where remote Singularity images are stored and direct Nextflow to automatically mount host paths in the executed container.

• Line 7-9 set the directory where Conda environments are stored.

• Line 10-14 sets default directives for processes in your pipeline. Note that the executor is set to pbspro on line 11.

• Line 15 provides the local path to genome files required for pipelines such as nf-core/rnaseq

More in depth information on Nextflow configuration is described here: https://www.nextflow.io/docs/latest/config.html.

...