denoise-paired issue for multithread and time

Hi,
I am doing denoise-paired using following command
qiime dada2 denoise-paired --i-demultiplexed-seqs demux-paired-end.qza --p-trim-left-f 10 --p-trim-left-r 10 --p-trunc-len-f 240 --p-trunc-len-r 240 --o-table table.qza --o-representative-sequences rep-seqs.qza --o-denoising-stats denoising-stats.qza --p-n-threads 0 --verbose
and its showing error, please see the attached image. I have 32 core, 512 GB memory but
when i am using less thread like 10, its taking more than 30 hrs and its still running (see another image).


Please can you suggest me what should be my best move from here?
I am bit in time crunch situation,
Thanks,
Khem

Hello @Khem. The more threads you create, the more memory you end up using. I suspect that when using all 32 cores you are running out of memory and that is manifesting as the error you are seeing. Depending on the size of your data it isn’t unusual for you to run out of memory or for a run on ten cores to be taking a very long time.

Additionally, what version of QIIME 2 are you running, and what kind of environment is this? If it is a virtual machine, is the core count and memory amount quoted the amount for the system as a whole or the amount available to the virtual machine?

Thank you,
Anthony Simard

2 Likes

Thanks for your response,

It was my one weird sample, had ~7 million reads and massive big. Once i excluded that sample, all good. finished in a few hours.

I also upgraded my qiime2 to latest version,
Thanks,
Khem

2 Likes

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.