Error_Classifier_SILVA132

Hello!
I am getting the following error when I run this command:
qiime feature-classifier classify-sklearn --i-classifier Classifier/silva-132-99-nb-classifier.qza --i-reads KS-data/taxonomy/rep-seqs-dada2.qza --o-classification KS-data/taxonomy/rep_seqs_silva132-taxonomy.qza

Error:
Traceback (most recent call last):
File “/home/bsen/miniconda3/envs/qiime2-2018.4/lib/python3.5/site-packages/q2cli/commands.py”, line 274, in call
results = action(**arguments)
File “”, line 2, in classify_sklearn
File “/home/bsen/miniconda3/envs/qiime2-2018.4/lib/python3.5/site-packages/qiime2/sdk/action.py”, line 225, in bound_callable
spec.view_type, recorder)
File “/home/bsen/miniconda3/envs/qiime2-2018.4/lib/python3.5/site-packages/qiime2/sdk/result.py”, line 265, in _view
result = transformation(self._archiver.data_dir)
File “/home/bsen/miniconda3/envs/qiime2-2018.4/lib/python3.5/site-packages/qiime2/core/transform.py”, line 70, in transformation
new_view = transformer(view)
File “/home/bsen/miniconda3/envs/qiime2-2018.4/lib/python3.5/site-packages/q2_feature_classifier/_taxonomic_classifier.py”, line 72, in _1
pipeline = joblib.load(os.path.join(dirname, ‘sklearn_pipeline.pkl’))
File “/home/bsen/miniconda3/envs/qiime2-2018.4/lib/python3.5/site-packages/sklearn/externals/joblib/numpy_pickle.py”, line 578, in load
obj = _unpickle(fobj, filename, mmap_mode)
File “/home/bsen/miniconda3/envs/qiime2-2018.4/lib/python3.5/site-packages/sklearn/externals/joblib/numpy_pickle.py”, line 508, in _unpickle
obj = unpickler.load()
File “/home/bsen/miniconda3/envs/qiime2-2018.4/lib/python3.5/pickle.py”, line 1043, in load
dispatchkey[0]
File “/home/bsen/miniconda3/envs/qiime2-2018.4/lib/python3.5/site-packages/sklearn/externals/joblib/numpy_pickle.py”, line 341, in load_build
self.stack.append(array_wrapper.read(self))
File “/home/bsen/miniconda3/envs/qiime2-2018.4/lib/python3.5/site-packages/sklearn/externals/joblib/numpy_pickle.py”, line 184, in read
array = self.read_array(unpickler)
File “/home/bsen/miniconda3/envs/qiime2-2018.4/lib/python3.5/site-packages/sklearn/externals/joblib/numpy_pickle.py”, line 130, in read_array
array = unpickler.np.empty(count, dtype=self.dtype)
MemoryError

Am I going wrong somewhere or there is a problem with the “silva-132-99-nb-classifier.qza” downloaded from qiime2.org. My “rep-seqs-dada2.qza” works fine with UNITE classifier shared by one of the qiime user. Please help. Thanks!

BS

Good afternoon,

Thanks for posting your full command and error message. I think I found the problem:

Your computer ran out of memory. This is not totally unexpected as Silva is a big database and feature-classifier does use a good deal of memory.

How much memory / RAM does your computer have? Can you close down background programs (like web browsers) to free up memory or switch to a computer that has more RAM?

Colin

3 Likes

Hi @colinbrislawn, many thanks for figuring out the problem. Here is my laptop configuration:
34%20AM .

I run qiime2 on this same Windows laptop with a Ubuntu platform (dual boot). Do you think dual boot system can be an issue if it is killing out some memory?

BS

1 Like

No — 6 GB of installed memory is just not going to be enough for training SILVA. Do you have an institutional cluster you can use?

I may have to explore and find out if an institutional cluster is available or not in our institution. Any other alternate solution to run the silva132 classifier with our data? Thanks!

1 Like

This memory error has been discussed extensively on the forum and some other solutions exist. However, 6 GB is really not going to be nearly enough for using a SILVA-trained classifier even with these additional solutions. You should get access to a cluster or more powerful machine, or try using the smaller greengenes database instead.

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.