Speeding up dada2 denoise-paired

User Support
5

Hi @emma1 ,

Welcome to the :qiime2: Forum

You have a lot of samples! Do you know if they come from the same sequencing run? DADA2 works assuming that, so if your FASTQ files come from different runs you should run DADA2 once per run and then merge the feature tables if you wish (source).

That being said, if you want to speed up the process here you have some suggestions / ideas (ordered by what I would try first):

  • Have you tried multithreading? Check out the --p-n-threads option (docs here).
  • --p-pooling-method independent is the default option, and should be faster (although people in the forum suggest that pseudo is also a sensible option, e.g. see here).
  • Is there a possibility that DADA2 hangs due to insufficient RAM and your cluster is not killing the job? That would explain why the time limit is reached. Cluster output and error files would be useful to investigate on this.
  • Most clusters have a short and long queue for jobs; and some clusters also have something like a "no limits" queue that you can use under special conditions if you contact the cluster maintainers.

Best,

Sergio

2 Likes