25S1W1 - 3. Download public metagenomics data from ENA

Today’s we will learn to download FASTQ files from a published paper:

Manuscript: Heikema et al. (2020). https://www.mdpi.com/2073-4425/11/9/1105

STEP 1 : Find where the data is available for download in the above manuscript

• Click on the link above and search for “accession”, “Data availability”, “BioProject ID” or “GEO accession code”

• If, only a GEO accession code is available, go to the GEO database and look for BioProject ID - Note, ENA (Step2) requires this identifier to download the data.

Which BioProject ID host the data used in the above manuscript?

STEP 2: Search for data for the identified BioProject ID at the European Nucleotide Archive (ENA) database

• Go to https://www.ebi.ac.uk/ena/browser/home and search for the BioProject ID using the search option on the top right corner and click on ‘view’:

Open

STEP 3: Download sample report - metadata information

Open

118 samples in this study - metagenomics data generated using Illumina and Nanopore sequencing technologies:

STEP 4: Select FASTQ files (tick boxes next to the file names) and click on “Get download script”. This will download a bash script (e.g., )

Open

• STEP 5: Open the downloaded ena file using TextEdit (NotePad or similar app). The downloaded script looks like this:

Now using the TextEdit or NotePad app, we will add the following lines to the top of the script - copy and paste the following to the above script:

You should have this:

STEP 6: Save the file and now let’s transfer it to the HPC. See below:

NOTE: To proceed, you need to be on QUT’s WiFi network or signed via VPN.

Windows PC: open file finder and type the address below to connect to your home directory in the HPC, and then browse to the /workshop/2024-2/session4_RNAseq/data/mydata folder

Mac: open file finder and press “command” + “k” to open prompt, then type the below command, and then browse to the /workshop/2024-2/session4_RNAseq/data/mydata folder

• Drag and drop the script into the /workshop/2025/S1W1/metagenomics/data/mydata folder

STEP 7: We will ensure the copied file from our laptop / desktop does not have unwanted characters. Let’s move to the data folder:

To simplify next steps let’s modify the name of the downloaded file:

STEP 8: Removing hidden characters from the downloaded file. We use the dos2unix conversion:

• Note: If you create a file using Microsoft Excel, it is likely that it will add ASCII characters, use dos2unix to remove such characters.

If you had issues with the steps above an end-file-download is available. Copy it as follows:

STEP 9: Submit the download job to the cluster

Now we are ready to submit to the HPC cluster the script to download FASTQ files:

Monitor progress of job:

• Note: Downloading the above datasets will take about ~50 minutes.

Alternatively, use the qstat command:

Tip: Do the following to know the number oj jobs on the cluster:

where: we use the qstat command to list all jobs, then we pipe the list to the wc- l command to count the number of lines (equivalent to number of jobs -1)

Other ways to download genomics data

Find in the link below alternative approaches to download data from SRA, BaseSpace or use the nf-core/fetchngs pipeline:

Data Download

Next page

25S1W1 - 4. Run nfcore/ampliseq test