Training classifier error: [1] 44206 killed

[Muhammad_Arslan]

I am trying to train the classifer but getting this error

(qiime2-2020.2) ➜ SILVA_132_QIIME_release
qiime feature-classifier fit-classifier-naive-bayes --i-reference-reads ref_seqs.qza --i-reference-taxonomy silva132_99_ref_taxonomy.qza --o-classifier classifier.qza --verbose

/Users/macair/opt/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2_feature_classifier/classifier.py:102: UserWarning: The TaxonomicClassifier artifact that results from this method was trained using scikit-learn version 0.22.1. It cannot be used with other versions of scikit-learn. (While the classifier may complete successfully, the results will be unreliable.)
warnings.warn(warning, UserWarning)

[1] 44206 killed qiime feature-classifier fit-classifier-naive-bayes --i-reference-reads

There is no output, there is no clear error. It just say that the command is killed! Any suggestions please?

• Version of QIIME 2 running, and how it is installed (conda)

[thermokarst]

Hi @Muhammad_Arslan - the error is coming from your operating system, it is letting you know that for some reason the command was killed. It looks like you might be running this command on a macos machine - it is probable that you simply don't have enough memory to train a classifier on a large database like SILVA. Can you confirm the available memory on this machine for us?

:qiime2:

[Muhammad_Arslan]

Thanks Matthew for the comment. This is correct, I am using macOS (big sur). I have around 66 gb of free space (see attachment). Do you think 66 gb is not sufficient??

[Muhammad_Arslan]

now i made 90 gb free but still getting same error

[thermokarst]

@Muhammad_Arslan - those numbers are your disk storage space, which unfortunately is not the same thing as memory (RAM). The memory should be listed on the "Overview" tab, like this:

[Muhammad_Arslan]

I am struggling with classifier training but it's not working for some reason. I am using macOS (big sur). There is no error, simply the job is killed

[1] 54127 killed qiime feature-classifier fit-classifier-naive-bayes --i-reference-reads

after running the command

qiime feature-classifier fit-classifier-naive-bayes --i-reference-reads ref_seqs.qza --i-reference-taxonomy silva_132_99_ref_taxonomy.qza --o-classifier trained_classifier.qza

I tried to use pre-trained classifiers but sklearn version I have is 0.24.2 and the newest trained classifer I can find os on 0.24.1. I tried installing the old version but it doesn't help either. I am in a dire need to finish some analysis and looking forward to any quick help (please!).

when I add --versobe, I get the following message additionally

/Users/macair/opt/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2_feature_classifier/classifier.py:102: UserWarning: The TaxonomicClassifier artifact that results from this method was trained using scikit-learn version 0.24.2. It cannot be used with other versions of scikit-learn. (While the classifier may complete successfully, the results will be unreliable.)

Thanks

• Version of QIIME 2: qiime2-2020.2
• I have 80 gb of free hard disk space

[Muhammad_Arslan]

@thermokarst
I just noticed that you already posted a response to my query earlier. Thanks
Here is the screenshot of my system! So 8 GB is not enough for this purpose? Previously, it worked fine in 2020 for the same purpose when I was using even older macOS.

[thermokarst]

Hi @Muhammad_Arslan, sorry for the slow reply.

@Muhammad_Arslan

So 8 GB is not enough for this purpose?

No, not for SILVA - we usually use ~64 GB when training SILVA classifiers.

@Muhammad_Arslan

Previously, it worked fine in 2020 for the same purpose when I was using even older macOS.

Perhaps you were training a different database? Greengenes doesn't need nearly as much RAM/Memory as SILVA, for example.

:qiime2: