...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
Pre-requisites
Nextflow installed → If not see: 1. Getting started with Nextflow Ask for help if you need assistance.
Overview:
Create working directory folders for hands-on training exercises
Copy data and scripts for today’s session
(Optional - those that did not attend Session 3 ) run a test to verify Nextflow is properly installed and to test assess the execution of the nf-core/rnaseq pipeline with example data provided by developers.
...
Before we start using the HPC, let’s start an interactive session:
| Code Block |
|---|
qsub -I -S /bin/bash -l walltime=10:00:00 -l select=1:ncpus=12:mem=4gb |
where:
‘walltime’ is amount of time requested to run the interactive session
‘cpus’ number of CPUs to be used by the interactive session
‘mem’ amount of memory assigned to the interactive session
Create working directories
Let’s create the working directories for today’s session:
| Code Block |
|---|
mkdir -p $HOME/workshop/2024-2/session4_RNAseq mkdir -p $HOME/workshop/2024-2/session4_RNAseq/scripts mkdir -p $HOME/workshop/2024-2/session4_RNAseq/scripts/datalocal mkdir -p $HOME/workshop/2024-2/session4_RNAseq/runsdata mkdir -p $HOME/workshop/2024-2/session4_RNAseq/runsdata/run1_QCmouse mkdir -p $HOME/workshop/2024-2/session4_RNAseq/runs/run2_RNAseq mkdir/data/human mkdir -p $HOME/workshop/2024-2/session4_RNAseq/data/mydata mkdir -p $HOME/workshop/2024-2/session4_RNAseq/runs/run3_RNAseq_T2T |
The folder structure will look like this:
| Code Block |
|---|
mkdir -p $HOME/workshop/2024-2/session4_RNAseq/ ├── data ├── runs │ ├── runs/run1_QC │ mkdir ├── -p $HOME/workshop/2024-2/session4_RNAseq/runs/run2_RNAseq │ mkdir └── -p $HOME/workshop/2024-2/session4_RNAseq/runs/run3_RNAseq_T2T └── scripts |
The above code creates the following folder structure:
...
Get a copy of the scripts
...
| Code Block |
|---|
cp /work/training/2024/rnaseq/scripts/* $HOME/workshop/2024-2/session4_RNAseq/scripts |
...
ls -l $HOME/workshop/2024-2/session4_RNAseq/scripts |
...
Copy public data to your $HOME
| Code Block |
|---|
cp /work/training/2024/rnaseq/scripts/datalocal/* $HOME/workshop/2024-2/session4_RNAseq/scripts/local ls -l $HOME/workshop/2024-2/session4_RNAseq/datascripts/local |
Line 1: Copies all files from /work/datasets/workshop/scripts/ folder as noted by an asterisk to the newly created folder $HOME/workshop/2024-2/session4_RNAseq/datascripts/
TBA
...
| title | Solution |
|---|
...
Line 2: List the files copied to the ‘script’ folder
Line 3: Copies all files from /work/datasets/workshop/scripts/local as noted by an asterisk to the newly created folder $HOME/workshop/2024-2/session4_RNAseq/scripts/
...
Create a folder for running the nf-RNA-seq pipeline
Let’s create an “runs” folder in the ~/workshop/2024/rnaseq folder to run the nf-core/rnaseq pipeline. For example:
...
local
Line 4: List the files copied to the ‘script’ folder
Copy public data to working folder
| Code Block |
|---|
cp /work/training/2024/rnaseq/data/mouse/* $HOME/workshop/2024/rnaseq/runs mkdir-2/session4_RNAseq/data/mouse ls -l $HOME/workshop/2024-2/rnaseqsession4_RNAseq/runsdata/run1_testmouse mkdircp $HOME/work/workshoptraining/2024/rnaseq/runs/run2_QC mkdir data/human/* $HOME/workshop/2024-2/rnaseqsession4_RNAseq/runs/run3_RNAseq cddata/human ls -l $HOME/workshop/2024/rnaseq/runs |
...
Lines 1-4: create sub-folders for each exercise
...
Line 5: change the directory to the folder “run1_test”
...
-2/session4_RNAseq/data/human |
Line 1: Copies mouse RNAseq folder to created ~/data/mouse folder
Line 2: List the files copied to the ‘data’ folder
Line 3: Copies human RNAseq folder to created ~/data/human folder
Line 4: List the files copied to the ‘data’ folder
(Optional ): Running a test with nf-core sample data
First, let’s assess the execution of the nf-core/rnaseq pipeline by running a test using sample data.
Copy the launch_nf-core_RNAseq_test.pbs to the working directory
...
Create a folder for running the pipeline test:
| Code Block |
|---|
mkdir -p $HOME/workshop/2024-2/session4_RNAseq/runs/run0_test cd $HOME/workshop/2024-2/session4_RNAseq/runs/run1_test run0_test |
Line 1: Create folder using the make directory “mkdir” command. We use the flap -p (parental) to create parental folders if not yet created.
Line 2: Move to the newly created folder.
Copy the launch_nf-core_RNAseq_test.pbs from the ‘scripts’ folder to the current folder
| Code Block |
|---|
cp $HOME/workshop/2024-2/session4_RNAseq/scripts/launch_nf-core_RNAseq_test.pbs . |
...
| Code Block |
|---|
cat launch_nf-core_RNAseq_test.pbs |
...
#!/bin/bash -l
#PBS -N nfrnaseq_test
#PBS -l select=1:ncpus=2:mem=4gb
#PBS -l walltime=24:00:00
#work on current directory
cd $PBS_O_WORKDIR
#load java and set up memory settings to run nextflow
module load java
export NXF_OPTS='-Xms1g -Xmx4g'
nextflow run nf-core/rnaseq -r 3.14.0 -profile test,singularity --outdir results
...
nextflow command: nextflow run
pipeline name: nf-core/rnaseq
pipeline version (revision): -r 3.14.0
container type and sample data: -profile test,singularity
output directory: --outdir results
Submitting the job
Submit the test job to the HPC cluster as follows:
...