6.2 Setting up working folders

Owned by Roberto Barrero Gumiel
Last updated: Oct 28, 2024
2 min read

Pre-requisites

Overview:

  • Create working directory folders for hands-on training exercises

  • Copy data and scripts for today’s session

  • (Optional ) run a test to verify Nextflow is properly installed and to assess the execution of the nf-core/rnaseq pipeline with example data provided by developers.

Prepare Working Directory Space for Session 4

Prepare Working Directory Space for Session 4

Before we start using the HPC, let’s start an interactive session:

qsub -I -S /bin/bash -l walltime=10:00:00 -l select=1:ncpus=2:mem=4gb

  • where:

    • ‘walltime’ is amount of time requested to run the interactive session

    • ‘cpus’ number of CPUs to be used by the interactive session

    • ‘mem’ amount of memory assigned to the interactive session

Create working directories

Let’s create the working directories for today’s session:

mkdir -p $HOME/workshop/2024-2/session6_smallRNAseq mkdir -p $HOME/workshop/2024-2/session6_smallRNAseq/scripts mkdir -p $HOME/workshop/2024-2/session6_smallRNAseq/scripts/local mkdir -p $HOME/workshop/2024-2/session6_smallRNAseq/data mkdir -p $HOME/workshop/2024-2/session6_smallRNAseq/data/human_disease mkdir -p $HOME/workshop/2024-2/session6_smallRNAseq/data/human_drosha mkdir -p $HOME/workshop/2024-2/session6_smallRNAseq/data/mydata mkdir -p $HOME/workshop/2024-2/session6_smallRNAseq/runs mkdir -p $HOME/workshop/2024-2/session6_smallRNAseq/runs/run1_human_miRBase mkdir -p $HOME/workshop/2024-2/session6_smallRNAseq/runs/run2_human_MirGeneDB mkdir -p $HOME/workshop/2024-2/session6_smallRNAseq/runs/run3_drosha_miRBase

The above code creates the following folder structure:

image-20241024-041834.png

Get a copy of the scripts

Now let’s copy the scripts necessary for today’s session:

cp /work/training/2024/smallRNAseq/scripts/* $HOME/workshop/2024-2/session6_smallRNAseq/scripts ls -l $HOME/workshop/2024-2/session6_smallRNAseq/scripts cp /work/training/2024/smallRNAseq/scripts/local/* $HOME/workshop/2024-2/session6_smallRNAseq/scripts/local ls -l $HOME/workshop/2024-2/session6_smallRNAseq/scripts/local

  • Line 1: Copies all files from /work/datasets/workshop/scripts/ as noted by an asterisk to the newly created folder $HOME/workshop/2024-2/session4_RNAseq/scripts/

  • Line 2: List the files copied to the ‘script’ folder

  • Line 3: Copies all files from /work/datasets/workshop/scripts/local as noted by an asterisk to the newly created folder $HOME/workshop/2024-2/session4_RNAseq/scripts/local

  • Line 4: List the files copied to the ‘script’ folder

Copy public data to working folder

  • Line 1: Copies mouse RNAseq folder to created ~/data/mouse folder

  • Line 2: List the files copied to the ‘data’ folder

  • Line 3: Copies human RNAseq folder to created ~/data/human folder

  • Line 4: List the files copied to the ‘data’ folder

(Optional ): Running a test with nf-core sample data

First, let’s assess the execution of the nf-core/rnaseq pipeline by running a test using sample data.

Create a folder for running the pipeline test:

  • Line 1: Create folder using the make directory “mkdir” command. We use the flap -p (parental) to create parental folders if not yet created.

  • Line 2: Move to the newly created folder.

Copy the launch_nf-core_RNAseq_test.pbs from the ‘scripts’ folder to the current folder

View the content of the script as follows:

image-20241024-061806.png

  • nextflow command: nextflow run

  • pipeline name: nf-core/smrnaseq

  • pipeline version (revision): -r 2.3.1

  • container type and sample data: -profile test,singularity

  • output directory: --outdir results

Submitting the job

Submit the test job to the HPC cluster as follows:

Monitoring the Run