classify-sklearn - Plugin error from feature-classifier: pop from empty list

Technical Support
, ,
1

Dear all,

I got this error while running feature-classifier classify-sklearn

Plugin error from feature-classifier:

pop from empty list

I am using qiime2-2020.2, and conda environment in Linux.

I trained my classifier using Silva128, 16S_97, v3-v4 trim, and taxonomy is 16S_97_consensus_taxonomy_7_levels, and ran under this.

#!/bin/sh
#$ -S /bin/sh
#$ -N train-classifier-silva-128-97-2020.2
#$ -l mem_req=256G
#$ -cwd

export PATH=$PATH:$HOME/.local/bin:$HOME/bin
export PATH=/home/miniconda3:$PATH
export PATH=/home/miniconda3/bin:$PATH

source ~/.bashrc

cd /home/qiime_data/reference

conda init bash
conda activate qiime2-2020.2
qiime feature-classifier fit-classifier-naive-bayes \
 --p-classify--chunk-size 10000 \
 --verbose \
 --output-dir /home/qiime_data/reference/classifier_training_error_silva_128_97_2020_2 \
 --i-reference-reads /home/qiime_data/reference/silva_128_16S_97_v3v4_ref_seq.qza \
 --i-reference-taxonomy SILVA_128_16S_97_consensus7.qza \
 --o-classifier /home/qiime_data/reference/naivebayes_silva128_97_v3v4classifier_2020_2.qza

Then I run

#!/bin/sh
#$ -S /bin/sh
#$ -N make-classify-sklearn-sliva97-2020-2
#$ -l mem_req=64G
#$ -cwd

export PATH=$PATH:$HOME/.local/bin:$HOME/bin
export PATH=/home/miniconda3:$PATH
export PATH=/home/miniconda3/bin:$PATH

source ~/.bashrc

cd /home/qiime_data/2019_tank_field

conda init bash
conda activate qiime2-2020.2

qiime feature-classifier classify-sklearn \
 --i-classifier naivebayes_silva128_97_v3v4classifier_2020_2.qza \
 --i-reads rep_all.qza \
 --o-classification 2019_Silva128_97_v3v4_2020_2_taxonomy.qza

Then I got the error, the detail of error message is

Traceback (most recent call last):
File "/home/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2cli/commands.py", line 328, in call
results = action(**arguments)
File "</home/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/decorator.py:decorator-gen-343>", line 2, in classify_sklearn
File "/home/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/qiime2/sdk/action.py", line 245, in bound_callable
output_types, provenance)
File "/home/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/qiime2/sdk/action.py", line 390, in callable_executor
output_views = self._callable(**view_args)
File "/home/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2_feature_classifier/classifier.py", line 220, in classify_sklearn
seq_ids, taxonomy, confidence = list(zip(*predictions))
File "/home/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2_feature_classifier/_skl.py", line 46, in predict
for calculated in workers(jobs):
File "/home/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/joblib/parallel.py", line 1004, in call
if self.dispatch_one_batch(iterator):
File "/home/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/joblib/parallel.py", line 835, in dispatch_one_batch
self._dispatch(tasks)
File "/home/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/joblib/parallel.py", line 754, in _dispatch
job = self._backend.apply_async(batch, callback=cb)
File "/home/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/joblib/_parallel_backends.py", line 209, in apply_async
result = ImmediateResult(func)
File "/home/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/joblib/_parallel_backends.py", line 590, in init
self.results = batch()
File "/home/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/joblib/parallel.py", line 256, in call
for func, args, kwargs in self.items]
File "/home/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/joblib/parallel.py", line 256, in
for func, args, kwargs in self.items]
File "/home/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2_feature_classifier/_skl.py", line 54, in _predict_chunk
return _predict_chunk_with_conf(pipeline, separator, confidence, chunk)
File "/home/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2_feature_classifier/_skl.py", line 81, in _predict_chunk_with_conf
classes = [cls for cls in classes if cls[0].pop(0) == level]
File "/home/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2_feature_classifier/_skl.py", line 81, in
classes = [cls for cls in classes if cls[0].pop(0) == level]
IndexError: pop from empty list

I used Greengene to run the whole process and it was successful. There was a post here about this error
It has been suggested that the problem appears to be an incomplete taxonomy, and if feeding --p-confidence -1 --p-read-orientation same may slove problem, but the new version can't feed -1 for the --p-confidence

Could anyone suggest how to slove this problem?
Thank you very much.
Sung

2

Hi @SY_Yang,

That's correct! You could try to find which feature ID has the missing taxonomic info... the 7-level taxonomy should not be missing this info but something is clearly wrong here, perhaps the original file became altered.

Use --p-confidence=disable instead (this was changed in a recent release to disambiguate what that does to the confidence setting)

Good luck!

3

Hi @Nicholas_Bokulich,

Thank you for the suggestion!
I take a look of the 7-level taxonomy of Silva, there are lots of taxonomic is missing the D_6, so I use the taxonomy all level.
However, even used the all level, I still need to use --p-confidence=disable , otherwise I still got the same error. This doesn’t happen in previous version of QIIME2.

Anyway I got my result successfully!
Thank you for the help!!
Sung

4

The 7-level should have 7 levels for everything, so I think something must have gone wrong, maybe someone altered this file somehow?

All level contains all levels, so will always have this issue, and is not recommended.

It is best to avoid using this setting if you can, as it just causes the classifier to choose the top hit without considering its similarity to other hits.

Something is going wrong here and I do not believe it to be a QIIME 2 version issue; I suspect it is a database version or format issue.