This instructional material was originally developed by Maely Gauthier in 2024 as part of the QUT eResearch infrastructure. It is free to distribute but we require that you acknowledge eResearch for any outputs (e.g. training, presentation slides, publications) that might result from using this training material.

Some sections of this course were adapted from the Carpentry course: https://carpentries-incubator.github.io/workflows-nextflow/.

Aims

• Learn what is Nextflow

• Install and configure Nextflow

• Find pipelines on repositories (e.g. nf-core and epi2me)

• Run pipelines using either the command line or a PBS script

• IN YOUR OWN TIME (Additional material and exercises):

  • Understand input and parameter specifications

  • Understand the concept of caching and the resume function

  • Understand how Nextflow pipelines output results

What will be covered during the workshop

• 1. Getting started with Nextflow

  • What is Nextflow?

  • Installing Nextflow

  • Nextflow’s base configuration

• 2. Nextflow pipeline repositories

  • nf-core

    • What is nf-core?

    • What are nf-core pipelines?

    • Searching for available nf-core pipelines

    • nf-core support

  • epi2me workflows

• 3. Running pipelines

  • Fetching pipeline code

  • Software requirements for pipelines

  • Install and test that the pipeline installed successfully

    • From the command line

    • Launching Nextflow using a PBS script

IN YOUR OWN TIME (Additional material and exercises):

• 4. Input specifications

  • Samplesheet input

    • Examples of samplesheets

    • Exercise 1

    • Exercise 2

  • Input folder

• 5. Parameters

  • Finding list of parameters available

    • Exercise 1

  • Specifying parameters on the command line

• 6. Nextflow caching

  • Resume option

  • Structure of work folder

    • Task execution directory

    • Specifying another work directory

    • Clean the work directory

• 7. Nextflow pipeline outputs

  • Results folder

  • Nextflow log, metrics and reports

• 8. Where to from now?

Prerequisites

You will require a basic knowledge of Linux/Unix commands to be able to participate effectively in this workshop. For this workshop we assume participants have either attended the first 2 workshops (Introduction to the Unix Shell for HPC Users and Introduction to the HPC), reviewed the materials provided in these workshops (if unable to attend) and are comfortable with it, or are already using the HPC.

Initial requirements

To be able to run these exercises, you’ll need:

1. A HPC account (Instructions for getting a HPC account can be found here).

2. PuTTy installed on your local computer ((Instructions for installing and using putty can be found here)

3. Access your HPC home directory from your PC (Instructions transferring data to and from the HPC can be found here).