Introduction to using the HPC

Pre-requisites

Have an HPC account on QUT’s lyra. Apply for a new HPC account here.
Introduction to the HPC: https://qutvirtual4.qut.edu.au/group/research-students/conducting-research/specialty-research-facilities/advanced-research-computing-storage/supercomputing/getting-started-with-hpc
Installed conda3 or miniconda3 ( Installing on Linux — conda 24.11.4.dev30 documentation ) → Recommend installing miniconda3 in your HPC’s /home directory.
(optional) Basic unix command line knowledge (example: Learning Resources: the Linux Command Line ; The Unix Shell: Summary and Setup )
(optional) Familiarity with one unix text editors (example Vi/Vim or Nano):
- VIM ( VIM Guide | Computational Biology Core ; Editors (Vim))
- Nano (Basic tutorial for Nano users ; https://www.howtogeek.com/howto/42980/the-beginners-guide-to-nano-the-linux-command-line-text-editor/ )

Installing Putty and connecting to the HPC

Install Putty:

Installing PuTTY - QUT Media Hub

Connect to the HPC:

Connecting to the HPC with PuTTY - QUT MediaHub

Log into the HPC

ssh userID@lyra.qut.edu.au

Brief basic Unix commands

Once you log into the HPC, you will land in your personal home space (i.e. /home/myStudentID/). This space is only accessible to you. To work in collaboration with others we use workspaces (i.e. /work/myProjectName/).

To go to a shared directory for your project named “kenna_team” type the following command and hit enter:

cd /work/speight_team/

Display list of files in a directory

ls -lh

Print working directory

Create a folder

Enter new folder

Move back to the previous folder

Make a backup copy of the file

Move a copy of a file to a newly created folder - note it is recommended to make a copy of important files prior to modifying or executing commands on them.

View the content of a file (note hashtags # at the start of a line is used to provide information of the code underneath it)

Go back to my personal space. Type 'cd' and hit enter. This will move you to /home/mystudentID/

Installing CONDA

Installing on Linux — conda 24.11.4.dev30 documentation

Follow the links on the page above and find the version of ‘Miniconda3’ script to download for your system. For the HPC, we select a Linux installer and right-click to copy the link to the file. The use the ‘wget’ command on the HPC to download the file:

Step1: Download miniconda installer

Step2: Verify the integrity of the downloaded file:

The above should print the following hash:

Step3: Install ‘miniconda3’ as follows:

Follow the prompts and accept all the suggested options.

Step4: Close and reopen the terminal to enable conda to be available.

Step5: Check that conda is installed

Installing BLAST

Once miniconda3 has been installed, you will need to log out and login again to enable the conda command.

To install BLAST or other bioinformatics tools go to https://anaconda.org and search for the tool of interest. You can also use the command

to search for tools in the bioconda channel.

If the tool is available click on the tool link, which will open a new window showing the command line needed to install the tool. For example for BLAST the suggested command is:

run the above command to install blast. Conda will evaluate if the tool or necessary dependencies are available and will automatically install all necessary items to run in this case blast.

Note: Follow a similar process as above to install other tools.

Interactive Job

Sample Data

Demo sample data to compare the similarity of DNA sequences generated by an RNA-seq approach against a reference Miscanthus sinensis mosaic virus (MsiMV) can be found at:

We want to compare the similarity (from 0 to 100%) of the sequences (also called ‘reads’) inside the query_sample.fa file against the reference MsiMV_genome.fasta sequence. Note: RNA/DNA (and protein) sequences can be stored in a ‘fasta format’. This is a header noted by “>” symbol followed by a sequence identifier on the first row. From the second row onwards the DNA/RNA(protein) sequence is presented.

Running blast on the HPC

We use a PBS Pro submission script to submit jobs to the HPC cluster. Create a file called ‘launch_blastN.pbs’ and fill it with this content, substituting email@host for your email address, and the files used as input to blastn:

Submit the job:

Checking the progression of the submitted job:

How to interpret the result? check this tutorial.

ER-User Guides