Out of memory when running q2-feature-classifier

Hello, I am receiving a similar error but for a different reason that I cannot interpret. I am using Greengenes 99_otus following the train your classifier tutorial. This is what I’ve entered and the error I got with --verbose

(qiime2-2020.8) qiime2@qiime2core2020-8:/media/sf_Desktop/Candice/BabyPoohStudy/breastmilk-Microbiome$ qiime feature-classifier fit-classifier-naive-bayes --i-reference-reads ref-seqs_210.qza --i-reference-taxonomy ref-taxoomy.qza --o-classifier classifier.qza --verbose
/home/qiime2/miniconda/envs/qiime2-2020.8/lib/python3.6/site-packages/q2_feature_classifier/classifier.py:102: UserWarning: The TaxonomicClassifier artifact that results from this method was trained using scikit-learn version 0.23.1. It cannot be used with other versions of scikit-learn. (While the classifier may complete successfully, the results will be unreliable.)
warnings.warn(warning, UserWarning)
Traceback (most recent call last):
File “/home/qiime2/miniconda/envs/qiime2-2020.8/lib/python3.6/site-packages/q2cli/commands.py”, line 329, in call
results = action(**arguments)
File “”, line 2, in fit_classifier_naive_bayes
File “/home/qiime2/miniconda/envs/qiime2-2020.8/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 245, in bound_callable
output_types, provenance)
File “/home/qiime2/miniconda/envs/qiime2-2020.8/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 390, in callable_executor
output_views = self._callable(**view_args)
File “/home/qiime2/miniconda/envs/qiime2-2020.8/lib/python3.6/site-packages/q2_feature_classifier/classifier.py”, line 331, in generic_fitter
pipeline)
File “/home/qiime2/miniconda/envs/qiime2-2020.8/lib/python3.6/site-packages/q2_feature_classifier/_skl.py”, line 32, in fit_pipeline
pipeline.fit(X, y)
File “/home/qiime2/miniconda/envs/qiime2-2020.8/lib/python3.6/site-packages/sklearn/pipeline.py”, line 335, in fit
self.final_estimator.fit(Xt, y, **fit_params_last_step)
File “/home/qiime2/miniconda/envs/qiime2-2020.8/lib/python3.6/site-packages/q2_feature_classifier/custom.py”, line 41, in fit
classes=classes)
File “/home/qiime2/miniconda/envs/qiime2-2020.8/lib/python3.6/site-packages/sklearn/naive_bayes.py”, line 565, in partial_fit
Y = label_binarize(y, classes=self.classes
)
File “/home/qiime2/miniconda/envs/qiime2-2020.8/lib/python3.6/site-packages/sklearn/utils/validation.py”, line 73, in inner_f
return f(**kwargs)
File “/home/qiime2/miniconda/envs/qiime2-2020.8/lib/python3.6/site-packages/sklearn/preprocessing/_label.py”, line 683, in label_binarize
Y = Y.toarray()
File “/home/qiime2/miniconda/envs/qiime2-2020.8/lib/python3.6/site-packages/scipy/sparse/compressed.py”, line 1025, in toarray
out = self._process_toarray_args(order, out)
File “/home/qiime2/miniconda/envs/qiime2-2020.8/lib/python3.6/site-packages/scipy/sparse/base.py”, line 1185, in _process_toarray_args
return np.zeros(self.shape, dtype=self.dtype, order=order)
MemoryError: Unable to allocate 820. MiB for an array with shape (20000, 5377) and data type int64

Plugin error from feature-classifier:

Unable to allocate 820. MiB for an array with shape (20000, 5377) and data type int64

See above for debug info.

Hi @cquin,

Sounds like you have very little memory available, if your machine cannot allocate <1GB RAM! Maybe this is an issue with allocating memory (e.g., if you are using a VirtualBox, follow the VB instructions to allocate more memory) or maybe it is an issue with running too many other processes in the background...

Good luck!

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.