Plugin in error when assigning taxonomy

mcunningham48 · August 29, 2019, 3:09pm

Hi,

I have been experiencing some problems when trying to assign taxonomy using qiime2 version 2019.4: using the following command
qiime feature-classifier classify-sklearn --i-classifier Silva-515-806-trained-classifier.qza --i-reads rep-seqs1.qza --o-classification taxonomy.qza

I am using the pre-trained classifier from Silva for 515F/806R
It is saying there is a plugin error.

The error message is posted below:
Error;
Plugin error from feature-classifier:

Debug info has been saved to /tmp/qiime2-q2cli-err-5my26ohh.log

Traceback (most recent call last):
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/python3.6/site-packages/q2cli/commands.py”, line 311, in call
results = action(**arguments)
File “</opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/python3.6/site-packages/decorator.py:decorator-gen-353>”, line 2, in classify_sklearn
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 231, in bound_callable
output_types, provenance)
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 365, in callable_executor
output_views = self._callable(**view_args)
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/python3.6/site-packages/q2_feature_classifier/classifier.py”, line 214, in classify_sklearn
reads, classifier, read_orientation=read_orientation)
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/python3.6/site-packages/q2_feature_classifier/classifier.py”, line 169, in _autodetect_orientation

How can this issue be resolved?

Many Thanks,

Mark

Nicholas_Bokulich · August 28, 2019, 3:24pm

Welcome to the forum, @mcunningham48!

It does not look like you posted the complete error message. Could you please edit your post to add the missing lines?

mcunningham48 · August 29, 2019, 11:09am

Hi,

Apologies, Please find the full error below with the current plugin problem:

Debug info save to /tmp/qiime2-q2cli-err-eehfmy35.log

Traceback (most recent call last):
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/pytho
n3.6/site-packages/q2cli/commands.py”, line 311, in call
results = action(**arguments)
File “</opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/pyth
on3.6/site-packages/decorator.py:decorator-gen-353>”, line 2, in classify_sklear
n
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/pytho
n3.6/site-packages/qiime2/sdk/action.py”, line 231, in bound_callable
output_types, provenance)
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/pytho
n3.6/site-packages/qiime2/sdk/action.py”, line 365, in callable_executor
output_views = self._callable(**view_args)
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/pytho
n3.6/site-packages/q2_feature_classifier/classifier.py”, line 214, in classify_s
klearn
reads, classifier, read_orientation=read_orientation)
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/pytho
n3.6/site-packages/q2_feature_classifier/classifier.py”, line 169, in _autodetec
t_orientation
result = list(zip(*predict(first_n_reads, classifier, confidence=0.)))
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/pytho
n3.6/site-packages/q2_feature_classifier/_skl.py”, line 45, in predict
for chunk in chunks(reads, chunk_size)) for m in c)
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/pytho
n3.6/site-packages/sklearn/externals/joblib/parallel.py”, line 917, in call
if self.dispatch_one_batch(iterator):
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/pytho
n3.6/site-packages/sklearn/externals/joblib/parallel.py”, line 759, in dispatch
one_batch
self._dispatch(tasks)
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/pytho
n3.6/site-packages/sklearn/externals/joblib/parallel.py”, line 716, in _dispatch
job = self._backend.apply_async(batch, callback=cb)
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/pytho
n3.6/site-packages/sklearn/externals/joblib/_parallel_backends.py”, line 182, in
apply_async
result = ImmediateResult(func)
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/pytho
n3.6/site-packages/sklearn/externals/joblib/_parallel_backends.py”, line 549, in
init
self.results = batch()
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/pytho
n3.6/site-packages/sklearn/externals/joblib/parallel.py”, line 225, in call
for func, args, kwargs in self.items]
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/pytho
n3.6/site-packages/sklearn/externals/joblib/parallel.py”, line 225, in <listcomp

for func, args, kwargs in self.items]

File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/pytho
n3.6/site-packages/q2_feature_classifier/_skl.py”, line 52, in _predict_chunk
return _predict_chunk_with_conf(pipeline, separator, confidence, chunk)
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/pytho
n3.6/site-packages/q2_feature_classifier/_skl.py”, line 66, in _predict_chunk_wi
th_conf
prob_pos = pipeline.predict_proba(X)
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/pytho
n3.6/site-packages/sklearn/utils/metaestimators.py”, line 118, in
out = lambda *args, **kwargs: self.fn(obj, *args, **kwargs)
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/pytho
n3.6/site-packages/sklearn/pipeline.py”, line 382, in predict_proba
return self.steps[-1][-1].predict_proba(Xt)
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/pytho
n3.6/site-packages/sklearn/naive_bayes.py”, line 104, in predict_proba
return np.exp(self.predict_log_proba(X))
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/pytho
n3.6/site-packages/sklearn/naive_bayes.py”, line 84, in predict_log_proba
jll = self._joint_log_likelihood(X)
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/pytho
n3.6/site-packages/sklearn/naive_bayes.py”, line 731, in joint_log_likelihood
return (safe_sparse_dot(X, self.feature_log_prob.T) +
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/pytho
n3.6/site-packages/sklearn/utils/extmath.py”, line 168, in safe_sparse_dot
ret = a * b
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/pytho
n3.6/site-packages/scipy/sparse/base.py”, line 473, in mul
return self._mul_multivector(other)
File “/opt/gridware/depots/54e7fb3c/el7/pkg/apps/qiime2/2019.4.0/bin/lib/pytho
n3.6/site-packages/scipy/sparse/compressed.py”, line 482, in _mul_multivector
other.ravel(), result.ravel())
MemoryError

The command for assigning taxonomy was ran using qiime 2 2019.4:
qiime feature-classifier classify-sklearn --i-classifier Silva-515-806-trained-classifier.qza --i-reads rep-seqs1.qza --o-classification taxonomy.qza

I have ran this command several times and getting same errors.

I would be thankful for any suggestions.

Many Thanks,

Mark

Nicholas_Bokulich · August 29, 2019, 3:09pm

Thanks Mark!

See the last line of the error message:

You are running out of memory! This is a common problem and you can check the forum archive for more info on this error and how to troubleshoot. Use the search tool:
https://forum.qiime2.org/search?q=feature%20classifier%20memory%20error

hint: try a different classifier (e.g., greengenes) or use the reads-per-batch parameter. SILVA often takes 32 GB+ RAM to run — but can be reigned in a bit (at the expense of longer runtime) by reads-per-batch

Good luck!