Problem with Filter Table command

[PabloCL (Pablo Cannestraci Lemaitre)]

Hello,

I am trying to make a filtered table which only includes the phylum Cyanobacteria, so I run the following Filter Table command with the following parameters:

qiime taxa filter-table
--i-table table.qza
--i-taxonomy taxonomy.qza
--p-include p__Cyanobacteria
--o-filtered-table table-phylum.qza

And I get this error message, which the terminal can't show completely.

It's not a plugin error as I retried using a similar command in the tutorials and my input files seem to be in order.

Help, please?

[thermokarst]

Hello @PabloCL! Sorry to hear things aren't going well.

@PabloCL

which the terminal can’t show completely.

Do you see that little orange rectangle in the bottom right corner of the window, just above the letters "DERECHA"? That is the terminal's scroll bar, you can click and drag that up and down to see the history in your terminal session. Also, the full error log is saved in the file /tmp/qiime2-q2cli-err-dtsxitio.log (this file will be different for each command log, it is a rotating temporary file).

On to the specific error, without seeing the entire thing, I might be barking up the wrong tree , but it looks like an issue with missing feature IDs from your feature table in your taxonomy file. I can't be certain without seeing the beginning of the error message, though.

If the ID suggestion doesn't set you on the right path (for example, maybe you mixed up files and used the wrong table or taxonomy?), please attach the error log, or a screenshot of the beginning of the error message, that way we can get to the bottom of this! Thanks!

[PabloCL (Pablo Cannestraci Lemaitre)]

Hello, @thermokarst.

The error message in this case is so big it won’t show up completely even if I go all the way up in the terminal window. It’s all just a big string of what seems to be Feature IDs.

Anyway, I checked the error logs and now know what seems to be the problem. I used the taxonomy artifact generated in the MP tutorial since when I used the following command:

qiime feature-classifier classify-sklearn
–i-classifier gg-13-8-99-515-806-nb-classifier.qza
–i-reads rep-seqs.qza
–o-classification taxonomy.qza

…on my own rep_seqs.qza artifact, it results in the following error.

Traceback (most recent call last):
  File "/home/qiime2/miniconda/envs/qiime2-2017.12/lib/python3.5/site-packages/q2cli/commands.py", line 224, in __call__
    results = action(**arguments)
  File "<decorator-gen-272>", line 2, in classify_sklearn
  File "/home/qiime2/miniconda/envs/qiime2-2017.12/lib/python3.5/site-packages/qiime2/sdk/action.py", line 228, in bound_callable
    output_types, provenance)
  File "/home/qiime2/miniconda/envs/qiime2-2017.12/lib/python3.5/site-packages/qiime2/sdk/action.py", line 363, in _callable_executor_
    output_views = self._callable(**view_args)
  File "/home/qiime2/miniconda/envs/qiime2-2017.12/lib/python3.5/site-packages/q2_feature_classifier/classifier.py", line 214, in classify_sklearn
    confidence=confidence)
  File "/home/qiime2/miniconda/envs/qiime2-2017.12/lib/python3.5/site-packages/q2_feature_classifier/_skl.py", line 45, in predict
    for chunk in _chunks(reads, chunk_size)) for m in c)
  File "/home/qiime2/miniconda/envs/qiime2-2017.12/lib/python3.5/site-packages/sklearn/externals/joblib/parallel.py", line 779, in __call__
    while self.dispatch_one_batch(iterator):
  File "/home/qiime2/miniconda/envs/qiime2-2017.12/lib/python3.5/site-packages/sklearn/externals/joblib/parallel.py", line 625, in dispatch_one_batch
    self._dispatch(tasks)
  File "/home/qiime2/miniconda/envs/qiime2-2017.12/lib/python3.5/site-packages/sklearn/externals/joblib/parallel.py", line 588, in _dispatch
    job = self._backend.apply_async(batch, callback=cb)
  File "/home/qiime2/miniconda/envs/qiime2-2017.12/lib/python3.5/site-packages/sklearn/externals/joblib/_parallel_backends.py", line 111, in apply_async
    result = ImmediateResult(func)
  File "/home/qiime2/miniconda/envs/qiime2-2017.12/lib/python3.5/site-packages/sklearn/externals/joblib/_parallel_backends.py", line 332, in __init__
    self.results = batch()
  File "/home/qiime2/miniconda/envs/qiime2-2017.12/lib/python3.5/site-packages/sklearn/externals/joblib/parallel.py", line 131, in __call__
    return [func(*args, **kwargs) for func, args, kwargs in self.items]
  File "/home/qiime2/miniconda/envs/qiime2-2017.12/lib/python3.5/site-packages/sklearn/externals/joblib/parallel.py", line 131, in <listcomp>
    return [func(*args, **kwargs) for func, args, kwargs in self.items]
  File "/home/qiime2/miniconda/envs/qiime2-2017.12/lib/python3.5/site-packages/q2_feature_classifier/_skl.py", line 52, in _predict_chunk
    return _predict_chunk_with_conf(pipeline, separator, confidence, chunk)
  File "/home/qiime2/miniconda/envs/qiime2-2017.12/lib/python3.5/site-packages/q2_feature_classifier/_skl.py", line 66, in _predict_chunk_with_conf
    prob_pos = pipeline.predict_proba(X)
  File "/home/qiime2/miniconda/envs/qiime2-2017.12/lib/python3.5/site-packages/sklearn/utils/metaestimators.py", line 115, in <lambda>
    out = lambda *args, **kwargs: self.fn(obj, *args, **kwargs)
  File "/home/qiime2/miniconda/envs/qiime2-2017.12/lib/python3.5/site-packages/sklearn/pipeline.py", line 357, in predict_proba
    return self.steps[-1][-1].predict_proba(Xt)
  File "/home/qiime2/miniconda/envs/qiime2-2017.12/lib/python3.5/site-packages/sklearn/naive_bayes.py", line 104, in predict_proba
    return np.exp(self.predict_log_proba(X))
  File "/home/qiime2/miniconda/envs/qiime2-2017.12/lib/python3.5/site-packages/sklearn/naive_bayes.py", line 84, in predict_log_proba
    jll = self._joint_log_likelihood(X)
  File "/home/qiime2/miniconda/envs/qiime2-2017.12/lib/python3.5/site-packages/sklearn/naive_bayes.py", line 725, in _joint_log_likelihood
    return (safe_sparse_dot(X, self.feature_log_prob_.T) +
  File "/home/qiime2/miniconda/envs/qiime2-2017.12/lib/python3.5/site-packages/sklearn/utils/extmath.py", line 135, in safe_sparse_dot
    ret = a * b
  File "/home/qiime2/miniconda/envs/qiime2-2017.12/lib/python3.5/site-packages/scipy/sparse/base.py", line 360, in __mul__
    return self._mul_multivector(other)
  File "/home/qiime2/miniconda/envs/qiime2-2017.12/lib/python3.5/site-packages/scipy/sparse/compressed.py", line 507, in _mul_multivector
    other.dtype.char))
MemoryError

So… It all seems to boil down to insufficient memory allocated to my VM. It’s no wonder I’m getting nowhere with filtering my table seeing as my taxonomy file is not the proper file for my table.

I don’t know what I was thinking, silly me .

I’ll fix the memory issue then get back to it. Thanks a lot for the help and the patience .