parallelize qiime rescript evaluate-fit-classifier on hpc

[Nicholas_Bokulich]

Hi @brittair ,

@brittair

Or perhaps I am doomed to pay the price of 35 hours

Unfortunately, yes. Because as mentioned on that other topic:

@Nicholas_Bokulich

This is a pipeline with different steps in a series, some paralellizable, others not. Unfortunately, one of the most time-consuming steps — fitting the classifier — is not parallelizable. So when you check top this step is most likely still running.

But the good news is that you only need to fit the classifier once (hopefully), and then you can re-use it in your environment many times, no need to repeat this step.

You can also just fit the classifier with q2-feature-classifier if you do not want to perform the evaluation steps. This will save you some hours of additional time. That's the step that is parallelized by the --p-n-jobs option, though, so you can leverage your hpc to speed up that step.

Sorry I can't give a more satisfying answer!