Dear all,
I’m wondering if anyone realizes that the speed of version 2019.1 is much slower than 2018.11. Currently, I’m performing DADA2 script to filter and merge my reads using version 2019.1 and it’s still not yet done after a week of running time, yet the similar raw reads which processed by version 2018.11 were done by 2 days.
FYI, I allocated 5 CPU and 11671MB for this qiime2 package.
Regards,
Benedict
Hmm, nothing has changed with the dada2 package…
Not to state the obvious, but did you remember to specify --p-n-threads?
I’m following the commands in the tutorial and there’s no --p-n-threads specified.
Meaning the multithreaded processing takes shorter time right? Previously, I didn’t specify --p-n-threads in version 2018.11 yet still the denoise step can be completed soon.
Btw, may I request your suggestion on the number of thread for my case?
Sample number: 30
Total reads per sample before merging: 200k - 300k
Estimated read length per sample: 300 bp
Hi @Benedict,
Yes, since the task is being divided across multiple threads.
As @thermokarst mentioned there hasn't been anything changed with regards to the q2-dada2 plugin so the difference you are experiencing is almost certainly coming from your environment. For example you mentioned that your samples were 'similar' suggesting that these were not the exact samples you used previously. Even a single large/bad sample can have quite a dramatic effect on run-time, so just because they were similar in number of samples, it doesn't necessarily mean they are going to take the same amount of time. There is quite a bit of other factors at work here.
This won't really make any difference if you don't include the --p-n-threads which directs the script to split the task.
This is totally dependent on how many threads are available to your system. If you have 5 threads available then you might as well use all 5. This will use almost all of your CPU so don't expect your computer to be running a bunch of other tasks smoothly. In fact I would probably refrain from doing much else while this is running. If you have allocated 5 cores to a virtual machine that runs your qiime2 you want to make that your main system also has some cores since it has to support the Virtual Machine within it. When using VM with 8 cores I generally dedicate 6 to that machine and keep 2 of my main system but then I'm not doing much else. tl;dr, try 5 and hope for the best.