This instructional material was originally developed by Maely Gauthier in 2024 as part of the QUT eResearch infrastructure. It is free to distribute but we just require that you acknowledge eResearch for any outputs (e.g. training, presentation slides, publications) that might result from using this training material.

Some sections of this course were adapted from the Carpentry course: https://carpentries-incubator.github.io/workflows-nextflow/.

Aims

What will be covered during the workshop:

• Prerequisites

• 1. Getting started with Nextflow

  • What is Nextflow?

  • Installing Nextflow

  • Nextflow’s base configuration

• 2. Nextflow pipeline repositories

  • nf-core

    • What is nf-core?

    • What are nf-core pipelines?

    • Searching for available nf-core pipelines

    • nf-core support

  • epi2me workflows

• 3. Running pipelines

  • Fetching pipeline code

  • Software requirements for pipelines

  • Install and test that the pipeline installed successfully

    • From the command line

    • Launching Nextflow using a PBS script

• 4. Input specifications

  • Samplesheet input

    • Examples of samplesheets

    • Exercise 1

    • Exercise 2

  • Input folder

• 5. Parameters

  • Finding list of parameters available

    • Exercise 1

  • Specifying parameters on the command line

• 6. Nextflow caching

  • Resume option

  • Structure of work folder

    • Task execution directory

    • Specifying another work directory

    • Clean the work directory

• 7. Nextflow pipeline outputs and PBS outputs

  • Results folder

  • Nextflow log, metrics and reports

  • PBS output

• 8. Where to from now?

Prerequisites

You will require a basic knowledge of Linux/Unix commands to be able to participate effectively in this workshop. If you don’t, please attend the following training [Introduction to HPC].

7. Nextflow pipeline outputs and PBS outputs

Results folder

The results are output in the folder name specified in the .nexftlow.config file under the outdir parameter. It is generally set to be results.

// nextflow.config
params {
  outdir = 'results'
}

Nextflow log, metrics and reports

By default, Nextflow will create a log file in the working directory called .nextflow.log. This file is hidden but you can see it using the command:

ls -a

If you rerun the pipeline in the same folder, the previous .nextflow.log will be renamed .nextflow.log.1 and a new .nextflow.log will be generated.

You can change the default location by specifying a different location

nextflow -log ~/code/nextflow.log run  

PBS output

8. Where to from now?

Nextflow offers free Fundamentals Training: https://training.nextflow.io/basic_training/

Provide links to carpentry course: https://carpentries-incubator.github.io/workflows-nextflow/instructor/01-getting-started-with-nextflow.html