Task 3: Run nf-core/rnaseq pipeline

Owned by Roberto Barrero Gumiel
Last updated: Sept 18, 2023 by Vicki Thomson
4 min read

Aim

Demonstrate how to run the nextflow nf-core/rnaseq pipeline in the HPC cluster. Initially, running a test and then processing real-life data.

Create a folder for running the nf-RNA-seq pipeline

Create a new folder under your ‘myname’ folder called nextflow, and also create a subfolder called ‘run1’ for the first test of the pipeline:

mkdir myname/nextflow mkdir myname/nextflow/run1

Go to the subfolder to run the RNAseq test

cd myname/nextflow/run1

Case 1: Running a test

Download and run the workflow using minimal data provided by nf-core/rnaseq. We recommend using singularity as the profile for QUT’s HPC. Another profile option can be ‘conda.’ Note: the profile option ‘docker’ is unavailable on the HPC.

nextflow run nf-core/rnaseq -profile test,singularity --outdir results -r 3.12.0

To execute the above command in the HPC cluster, prepare a PBS Pro submission script as follows:

#!/bin/bash -l #PBS -N nfrnaseq_test #PBS -l select=1:ncpus=2:mem=4gb #PBS -l walltime=24:00:00 #work on current directory (folder) cd $PBS_O_WORKDIR #load java and set up memory settings to run nextflow module load java NXF_OPTS='-Xms1g -Xmx4g' nextflow run nf-core/rnaseq -profile test,singularity --outdir results -r 3.12.0
Submitting the job

Once you have created the folder for the run, the samplesheet.csv file, nextflow.config, and launch.pbs, you are ready to submit.

Submit the run with this command (On Lyra)

qsub launch.pbs
Monitoring the Run

You can use the command

qstat -u $USER

Alternatively, use the command

qjobs

to check on the jobs you are running. Nextflow will launch additional jobs during the run.

You can also check the .nextflow.log file for details on what is going on.

Case 2: Run RNA-seq QC check

Preparing a ‘samplesheet.csv’ file

Prepare a sample sheet file that specifies the input files to be used. To do this, we use an nf-core script to generate the ‘samplesheet.csv’ file as follows (setting strandedness to auto allows the pipeline to determine the strandedness of your RNA-seq data automatically):

Load module Python 3.10

module load python/3.10.8-gcccore-12.2.0

Download the script for creating the ‘samplesheet.csv’ metadata file.

wget -L https://raw.githubusercontent.com/nf-core/rnaseq/master/bin/fastq_dir_to_samplesheet.py chmod +x fastq_dir_to_samplesheet.py

If you do not know already, determine the path to where the FASTQ files were downloaded/are located.

cd /myname/data #then run the following command pwd

Using a text editor (i.e., VIM), edit the following code with the appropriate path to the files:

#generate the samplesheet.csv file ./fastq_dir_to_samplesheet.py /path/to/directory/containing/fastq_files/ samplesheet.csv \ --strandedness auto \ --read1_extension _R1.fastq.gz \ --read2_extension _R2.fastq.gz

Example of 'samplesheet.csv' required for nf-core/rnaseq pipeline version 3.12.0:

sample,fastq_1,fastq_2,strandedness control_1,/path/to/directory/containing/fastq_files/control-1_R1.fastq.gz,/path/to/directory/containing/fastq_files/control-1_R2.fastq.gz,auto control_2,/path/to/directory/containing/fastq_files/control-2_R1.fastq.gz,/path/to/directory/containing/fastq_files/control-2_R2.fastq.gz,auto control_3,/path/to/directory/containing/fastq_files/control-3_R1.fastq.gz,/path/to/directory/containing/fastq_files/control-3_R2.fastq.gz,auto infected_1,/path/to/directory/containing/fastq_files/infected-1_R1.fastq.gz,/path/to/directory/containing/fastq_files/infected-1_R2.fastq.gz,auto infected_2,/path/to/directory/containing/fastq_files/infected-1_R1.fastq.gz,/path/to/directory/containing/fastq_files/infected-2_R2.fastq.gz,auto infected_3,/path/to/directory/containing/fastq_files/infected-1_R1.fastq.gz,/path/to/directory/containing/fastq_files/infected-3_R2.fastq.gz,auto

Prepare the following ‘launch_QC.pbs’ script:

#!/bin/bash -l #PBS -N nfrnaseq_QC #PBS -l select=1:ncpus=2:mem=4gb #PBS -l walltime=24:00:00 #work on current directory (folder) cd $PBS_O_WORKDIR #load java and set up memory settings to run nextflow module load java NXF_OPTS='-Xms1g -Xmx4g' nextflow run nf-core/rnaseq \ -profile singularity \ -r 3.12.0 \ --input samplesheet.csv \ --outdir results \ --genome GRCm38-local \ --skip_trimming \ --skip_alignment \ --skip_pseudo_alignment

We recommend running the nextflow nf-core/rnaseq pipeline once and then assessing the fastqc results folder to assess if sequence biases are present in the 5'-end and 3'-end ends of the sequences. Version 3.12.0 allows running the pipeline to do quality assessment only, without any alignment, read counting, or trimming. To execute that option, add the following flags to your nextflow run nf-core/rnaseq command: --skip_trimming, --skip_alignment, and --skip_pseudo_alignment.

Submitting the job

Once you have created the folder for the run, the samplesheet.csv file, and launch.pbs, you are ready to submit.

Submit the run with this command

qsub launch.pbs

Monitoring the Run

You can use the command

qstat -u $USER

Alternatively, use the command

qjobs

to check on the jobs, you are running. Nextflow will launch additional jobs during the run.

You can also check the .nextflow.log file for details on what is going on.

Assessing QC report

Evaluate the nucleotide distributions in the 5'-end and 3'-end of the sequenced reads (Read1 and Read 2). Define how many nucleotides should be trimmed from each region of the sequences. This will inform the parameter setting for Case 3 below.

Case 3: Run RNA-seq pipeline

Adjusting the Trim Galore options

When the initial trimming is done, verify if any more clipping needs to be done and run the nf-core/rnaseq pipeline that will perform all the steps. For example:

#!/bin/bash -l #PBS -N nfRNAseq #PBS -l select=1:ncpus=2:mem=4gb #PBS -l walltime=48:00:00 cd $PBS_O_WORKDIR module load java NXF_OPTS='-Xms1g -Xmx4g' nextflow run nf-core/rnaseq --input samplesheet.csv \ --outdir results \ -r 3.12.0 \ --genome GRCm38-local \ -profile singularity \ --aligner star_salmon \ --extra_trimgalore_args "--quality 30 --clip_r1 10 --clip_r2 10 --three_prime_clip_r1 1 --three_prime_clip_r2 1 "

Submitting the job

Once you have created the folder for the run, the samplesheet.csv file, and ‘launch.pbs', you are ready to submit.

Submit the run with this command

qsub launch.pbs

Monitoring the Run

You can use the command

qstat -u $USER

Alternatively, use the command

qjobs

to check on the jobs you are running. Nextflow will launch additional jobs during the run.

You can also check the .nextflow.log file for details on what is going on.

Outputs

The pipeline will produce two folders, one called “work,” where all the processing is done, and another called “results,” where we can find the pipeline's outputs. The content of the results folder is as follows:

results/ ├── fastqc ├── multiqc │   └── star_salmon ├── pipeline_info ├── star_salmon │   ├── bigwig │   ├── CD49fmNGFRm_rep1 │   ├── CD49fmNGFRm_rep2 │   ├── CD49fmNGFRm_rep3 │   ├── CD49fpNGFRp_rep1 │   ├── CD49fpNGFRp_rep2 │   ├── CD49fpNGFRp_rep3 │   ├── deseq2_qc │   ├── dupradar │   ├── featurecounts │   ├── log │   ├── MTEC_rep1 │   ├── MTEC_rep2 │   ├── MTEC_rep3 │   ├── picard_metrics │   ├── qualimap │   ├── rseqc │   ├── samtools_stats │   └── stringtie └── trimgalore └── fastqc

The quantification of the gene and transcript expressions can be found in the ‘star_salmon’ directory.

cd results/star_salmon

The following feature count tables are generated:

#gene level expression salmon.merged.gene_counts_length_scaled.rds salmon.merged.gene_counts_length_scaled.tsv salmon.merged.gene_counts.rds salmon.merged.gene_counts_scaled.rds salmon.merged.gene_counts_scaled.tsv salmon.merged.gene_counts.tsv <--- This file will be used for differential expression analysis using DESeq2 salmon.merged.gene_tpm.tsv #transcript level expression salmon.merged.transcript_counts.rds salmon.merged.transcript_counts.tsv salmon.merged.transcript_tpm.tsv