Aims
Run the nextflow nf-core/rnaseq pipeline in the HPC cluster. Exercises include:
Running a test to verify the execution of the pipeline
Running QC check to determine read trimming parameters
Running the full nf-core/rnaseq pipeline.
Work in the HPC |
|---|
Before we start using the HPC, let’s start an interactive session:
...
| Code Block |
|---|
mkdir -p $HOME/workshop/scripts cp /work/training/rnaseq/scripts/* $HOME/workshop/scripts/ ls -l $HOME/workshop/scripts/ |
Line 1: The -p indicates create 'parental directories as required. Thus the line 1 command creates both /workshop/ and the subfolder /workshop/scripts/
Line 2: Copies all files from /work/datasets/workshop/scripts/ as noted by an asterisk to the newly created folder $HOME/workshop/scripts/
Copy public data to your $HOME
| Code Block |
|---|
mkdir -p $HOME/workshop/data cp /work/training/rnaseq/data/* $HOME/workshop/data/ # list the content of the $HOME/workshop/data/ |
Line 1: The first command creates the folder /scripts/
Line 2: Copies all files from /work/datasets/workshop/scripts/ folder as noted by an asterisk to newly created $HOME/workshop/scripts/ folder
Line 3: a quick challenge - see the previous section for hints
Create a folder for running the nf-RNA-seq pipeline
...
| Code Block |
|---|
mkdir -p $HOME/workshop/RNAseq mkdir $HOME/workshop/RNAseq/run1_test mkdir $HOME/workshop/RNAseq/run2_QC mkdir $HOME/workshop/RNAseq/run3_RNAseq cd $HOME/workshop/ |
Lines 1-4: create sub-folders for each exercise
Line 5: change the directory to the folder “run1_test”
Line 6: print the current working directory
Exercise 1: Running a test with nf-core sample data
First, let’s assess the execution of the nf-core/rnaseq pipeline by running a test using sample data.
...
#!/bin/bash -l #PBS -N nfrnaseq_test #PBS -l select=1:ncpus=2:mem=4gb #PBS -l walltime=24:00:00 #work on current directory cd $PBS_O_WORKDIR #load java and set up memory settings to run nextflow module load java export NXF_OPTS='-Xms1g -Xmx4g' nextflow run nf-core/rnaseq -r 3.12.0 -profile test,singularity --outdir results |
|---|
nextflow command: nextflow run
pipeline name: nf-core/rnaseq
pipeline version: -r 3.12.0
container type and sample data: -profile test,singularity
output directory: --outdir results
Submitting the job
Submit the test job to the HPC cluster as follows:
...
Monitoring the Run
| Code Block |
|---|
qjobs |
Exercise 2: Run RNA-seq QC check
The pipeline requires preparing at least 2 files:
Metadata file (samplesheet.csv) thatspecifies the name of the samples, location of FASTQ files ('Read 1' and ‘Read 2’), and strandedness (forward, reverse, or auto. Note: auto is used when the strandedness of the data is unknown)
PBS Pro script (launch_nf-core_RNAseq_QC.pbs) with instructions to run the pipeline
Create the metadata file (samplesheet.csv):
Change to the data folder directory:
...
Copy the bash script to the working folder
| Code Block |
|---|
cp $HOME/workshop/work/training/rnaseq/scripts/create_RNAseq_samplesheetsamplesheet_nf-core_RNAseq.sh $HOME/workshop/data/ |
Note: you could replace ‘$HOME/workshop/data’ with “.” A dot indicates ‘current directory’ and will copy the file to the directory where you are currently located
View the content of the script:
| Code Block |
|---|
cat create_samplesheet_nf-core_RNAseq.sh |
Example for Single-End data (only ‘Read 1’ is available):
#!/bin/bash -l #User defined variables ########################################################## DIR='$HOME/workshop/data/' INDEX='samplesheet.csv' ########################################################## #load python module module load python/3.10.8-gcccore-12.2.0 #fetch the script to create the sample metadata table wget -L https://raw.githubusercontent.com/nf-core/rnaseq/master/bin/fastq_dir_to_samplesheet.py chmod +x fastq_dir_to_samplesheet.py #generate initial sample metadata file ./fastq_dir_to_samplesheet.py $DIR $INDEX \ --strandedness auto \ --read1_extension .fastq.gz |
|---|
...
Copy the PBS Pro script for QC (launch_nf-core_RNAseq_QC.pbs)
Copy and paste the code below to the terminal:
| Code Block |
|---|
cp $HOME/workshop/data/samplesheet.csv $HOME/workshop/RNAseq/run2_QC cp $HOME/workshop/scripts/launch_nf-core_RNAseq_QC.pbs $HOME/workshop/RNAseq/run2_QC cd $HOME/workshop/RNAseq/run2_QC |
Line 1: Copy the samplesheet.csv file to the working directory
Line 2: move to the working directory
Line 3: copy the launch_nf-core_RNAseq_QC.pbs submission script to the working directory
View the content of the launch_nf-core_RNAseq_QC.pbs script:
...
#!/bin/bash -l #PBS -N nfrnaseq_QC #PBS -l select=1:ncpus=2:mem=4gb #PBS -l walltime=24:00:00 #work on current directory cd $PBS_O_WORKDIR #load java and set up memory settings to run nextflow module load java export NXF_OPTS='-Xms1g -Xmx4g' nextflow run nf-core/rnaseq \ -profile singularity \ -r 3.12.0 \ --input samplesheet.csv \ --outdir results \ --genome GRCm38-local \ --skip_trimming \ --skip_alignment \ --skip_pseudo_alignment |
|---|
We recommend running the nextflow nf-core/rnaseq pipeline once and then assessing the fastqc results folder to assess if sequence biases are present in the 5'-end and 3'-end ends of the sequences.
Version 3.12.0 allows running the pipeline to do quality assessment only, without any alignment, read counting, or trimming.
Submitting the job
Once you have created the folder for the run, the samplesheet.csv file, and launch.pbs, you are ready to submit the job to the HPC scheduler:
...
Once the pipeline has finished running - Assess the QC report:
NOTE: To proceed, you need to be on QUT’s WiFi network or signed via VPN.
To browse the working folder in the HPC type in the file finder:
...
Assess QC reports (FastQC and MultiQC) to define how many nucleotides should be trimmed from the 5'-end and/or 3-end regions of the FASTQ reads (see Case 3 below).
Exercise 3: Run RNA-seq pipeline
Copy and paste the code below to the terminal:
| Code Block |
|---|
cp $HOME/workshop/data/samplesheet.csv $HOME/workshop/RNAseq/run3_RNAseq cd $HOME/workshop/RNAseq/run3_RNAseq pwd |
Line 1: Copy the samplesheet.csv file to the working directory
Line 2: move to the working directory
Line 3: print working directory → verify the folder location
Copy the PBS Pro script to run the nf-core/rnaseq pipeline:
...
| Code Block |
|---|
#gene level expression salmon.merged.gene_counts_length_scaled.rds salmon.merged.gene_counts_length_scaled.tsv salmon.merged.gene_counts.rds salmon.merged.gene_counts_scaled.rds salmon.merged.gene_counts_scaled.tsv salmon.merged.gene_counts.tsv <--- This file will be used for differential expression analysis using DESeq2 salmon.merged.gene_tpm.tsv #transcript level expression salmon.merged.transcript_counts.rds salmon.merged.transcript_counts.tsv salmon.merged.transcript_tpm.tsv |
Tip: Read the help information for Nextflow pipelines
Information on how to run a nextflow pipeline and additional available parameters can be provided on the pipeline website (i.e., https://nf-co.re/rnaseq/3.12.0/docs/usage/ ). You can also run the following command to get help information:
...