Downloading data from BaseSpace

Aim:

Download sequencing data generated for your project from Illumina's BaseSpace to your HPC account/

Pre-requisites

• If you do not yet have an account at QUT’s HPC, apply for a new account here.

Additionally, we recommend users be familiar with the following topics:

• Installed conda3 or miniconda3 ( Installing on Linux — conda 24.9.3.dev45 documentation )

• Basic unix command line knowledge (example: Learning Resources: the Linux Command Line ; The Unix Shell: Summary and Setup )

• Familiarity with one unix text editors (example Vi/Vim or Nano):

  • VIM ( VIM Guide | Computational Biology Core ; Editors (Vim))

  • Nano (Basic tutorial for Nano users ; https://www.howtogeek.com/howto/42980/the-beginners-guide-to-nano-the-linux-command-line-text-editor/ )

Download the data from BaseSpace

• Create a BaseSpace account

• Ask for access to Project (having a link to the project should be enough) from the project owner.

Install BaseScapce Sequence Hub

source: https://developer.basespace.illumina.com/docs/content/documentation/cli/cli-overview

Log into QUT’s HPC using your account credentials (see above)

The run the following command:

then change the file permissions to make the downloaded binary executable:

Authenticate the connection to BaseSpace server:

Find the project ID for the data you are interested to download. For example, you can derive this from the url when logged into BaseSpace

NOTE: If you have not yet created a folder to store raw data, you can run for example the following command:

then you can create a subfolder for your project, for example:

Example: now you are ready to download the Fastq.gz files

Download files using a PBS Pro script (i.e., called launch_fetch_BaseSpaceData.pbs):

Submit the job to the PBS Pro scheduler (queue):

Monitor progress:

It can take ~15-20 min to download ~170GB of data.