DAD2 denoising of 55 paied end samples

Dear Colleagues
I have 55 samples made up of 8,989,708 paired end sequences. I made paired-end-demux.qza artifact using PairedEndFastqManifestPhred33 command. I am using latest i7 machine with 64 GB RAM to run below command
qiime dada2 denoise-paired
--i-demultiplexed-seqs paired-end-demux.qza
--p-trim-left-f 0
--p-trim-left-r 0
--p-trunc-len-f 277
--p-trunc-len-r 216
--o-table table.qza
--o-representative-sequences rep-seqs.qza
--o-denoising-stats denoising-stats.qza

My paired-end-demux.qza file size is 3.9 GB.
It is more than 2 days since I put command but there is no output, neither there is any error message.
Please suggest if I should wait further or there is something wrong with command or artifact. Below is my interactive quality plot.

Hi @drmusk,

That’s quite a lot of data! I’d be patient a bit longer.

Best,
Justine

2 Likes

Hi @drmusk,

Please try --p-n-threads option to increase the number of threads.
In my case (latest core i7 with 32 GM RAM) can process 12,000,000 paired end sequences in one night.
Another option is --p-n-reads-learn, which decrease the number of reads for training the error model.
See detail here (https://docs.qiime2.org/2019.1/plugins/available/dada2/denoise-paired/).
I hope that you can reduce the time for processing DADA2.

Regards

3 Likes

Hi Suzuki and jwdebelius
Thanks for sharing this valuable information though I couldn't try. That step completed over the weekend. But I am worried now as table.qzv and rep-seq.qzv contain only a small fraction of the actual sequences. I am attaching here my demux and table files for review. demux.qzv tells that I have 9million sequences but table.qzv tells that I have 1800 features. I used this below command for dada2 denoise

qiime dada2 denoise-paired
–i-demultiplexed-seqs paired-end-demux.qza
–p-trim-left-f 0
–p-trim-left-r 0
–p-trunc-len-f 220
–p-trunc-len-r 220
–o-table table.qza
–o-representative-sequences rep-seqs.qza
–o-denoising-stats denoising-stats.qza

demux.qzv (291.9 KB)

table.qzv (318.2 KB)

denoising-stats.qzv (1.2 MB)

Hi @drmusk,

If you look at your stats, you can see you failed merging. Im not sure if I’d try longer reads, or just try with your forward reads.

Best,
Justine

2 Likes

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.