nf-core/bactmap: A mapping-based pipeline for creating a phylogeny from bacterial whole genome sequences

Prepared by the eResearch Office, QUT.

This page provides a guide to QUT users on how to install and run the nextflow nf-core/bactmap workflow on the HPC.

Pre-requisites

• Basic unix command line knowledge (example: Learning Resources: the Linux Command Line ; https://swcarpentry.github.io/shell-novice/ )

• Familiarity with one unix text editors (example Vi/Vim or Nano):

  • VIM ( https://bioinformatics.uconn.edu/vim-guide/ ; Editors (Vim))

  • Nano (https://engineering.purdue.edu/ECN/Support/KB/Docs/BasictutorialforNanou ; https://www.howtogeek.com/howto/42980/the-beginners-guide-to-nano-the-linux-command-line-text-editor/ )

• Have an HPC account on QUT’s lyra. Apply for a new HPC account here.

Install Nextflow

The nf-core/bactmap workflow requires Nextflow to be installed in your account on the HPC. Find details on how to install and test Nextflow here prepare a nextflow.config file and run a PBS pro submission script for Nextflow pipelines.

Additional information available here: https://nf-co.re/usage/installation

Additional details on the workflow can be found at:

Overview: https://nf-co.re/bactmap/1.0.0

Usage: https://nf-co.re/bactmap/1.0.0/usage

Interactive session on the HPC

SRA TOOLKIT

Use singularity container to fetch public data on the HPC:

1. One file at a time:

2. use a list:

3.a compress the fastq files

3.b alternatively run a loop to compress all fastq files in the folder:

Getting Started

Download and run the workflow using a minimal data provided by nf-core/bactmap. We recommend using singularity as the profile for QUT’s HPC. Note: the profile option ‘docker’ is not available on the HPC.

Note: at this time, the test profile will fail to run

Running the test - create a 'launch.pbs' script:

submit the job:

check the job:

Running the pipeline using custom data

Example of a typical command to run a Bactmap analysis:

Note, if the running was interrupted or did not complete a particular step or you want to modify a parameter for a particular step, instead of re-running all process again nextflow enables to “-resume” the workflow.

Preparing a ‘samplesheet.csv’ file

Prepare an samplesheet.csv file containing the information of the samples to be processed. See below examples of index.csv files.

Example samplesheet.csv:

When specifying the path to the data files, it is more portable to use absolute paths rather than relative paths.

check if ascii characters were added in your samplesheet.csv file:

Creating the samplesheet.csv file using Excel can add ascii characters, run the following command to remove them:

Preparing to run on the HPC

To run this on the HPC a PBS submission script needs to be created using a text editor. For example, create a file called launch.pbs using a text editor of choice (i.e., vi or nano) and then copy and paste the code below:

We recommend running the nextflow nf-core/bactmap pipeline once and then assess the results folder to assess if. Then, we can use the PBS script below to ...

Note: The options to the bactmap pipeline can be placed in a nextflow.config file instead.

Submitting the job

Once you have created the folder for the run, the samplesheet.csv file, nextflow.config (optional) and launch.pbs you are ready to submit.

Submit the run with this command (On Lyra)

Monitoring the Run

You can use the command

Alternatively use the following command:

To check on the jobs you are running. Nextflow will launch additional jobs during the run.

You can also check the .nextflow.log file for details on what is going on.

Finally, if you have configured the connection to the NFTower you can logon and check your run.