UNITE (fungal ITS) Taxonomy classifiers for use with q2-feature-classifier

Hi all,

I am using QIIME2 2023.7 to import the reference data from UNITE to train a classifier. I successfully trained the classifier, but when I attempted to use the classifier to obtain taxonomic classification results, it failed, and I received the error message below. Has anyone had a similar experience or do you have any ideas on how I should troubleshoot this?

qiime feature-classifier classify-sklearn --i-classifier unite-ver9-dynamic-all-classifier-25.07.2023.qza --i-reads asv-rep-seqs-ITS-all.qza --o-classification unite-ver9-ITS-taxonomy-asv-all-25.07.2023.qza --p-n-jobs -1 --verbose
joblib.externals.loky.process_executor._RemoteTraceback:
"""
Traceback (most recent call last):
File "/home/.conda/envs/qiime2-2023.7/lib/python3.8/site-packages/joblib/externals/loky/backend/queues.py", line 159, in feed
obj
= dumps(obj, reducers=reducers)
File "/home/.conda/envs/qiime2-2023.7/lib/python3.8/site-packages/joblib/externals/loky/backend/reduction.py", line 215, in dumps
dump(obj, buf, reducers=reducers, protocol=protocol)
File "/home/.conda/envs/qiime2-2023.7/lib/python3.8/site-packages/joblib/externals/loky/backend/reduction.py", line 208, in dump
_LokyPickler(file, reducers=reducers, protocol=protocol).dump(obj)
File "/home/.conda/envs/qiime2-2023.7/lib/python3.8/site-packages/joblib/externals/cloudpickle/cloudpickle_fast.py", line 632, in dump
return Pickler.dump(self, obj)
File "/home/.conda/envs/qiime2-2023.7/lib/python3.8/site-packages/joblib/_memmapping_reducer.py", line 446, in call
for dumped_filename in dump(a, filename):
File "/home/.conda/envs/qiime2-2023.7/lib/python3.8/site-packages/joblib/numpy_pickle.py", line 553, in dump
NumpyPickler(f, protocol=protocol).dump(value)
File "/home/.conda/envs/qiime2-2023.7/lib/python3.8/pickle.py", line 487, in dump
self.save(obj)
File "/home/.conda/envs/qiime2-2023.7/lib/python3.8/site-packages/joblib/numpy_pickle.py", line 352, in save
wrapper.write_array(obj, self)
File "/home/.conda/envs/qiime2-2023.7/lib/python3.8/site-packages/joblib/numpy_pickle.py", line 134, in write_array
pickler.file_handle.write(chunk.tobytes('C'))
OSError: [Errno 28] No space left on device
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/.conda/envs/qiime2-2023.7/lib/python3.8/site-packages/q2cli/commands.py", line 478, in call
results = self._execute_action(
File "/home/.conda/envs/qiime2-2023.7/lib/python3.8/site-packages/q2cli/commands.py", line 539, in _execute_action
results = action(**arguments)
File "", line 2, in classify_sklearn
File "/home/.conda/envs/qiime2-2023.7/lib/python3.8/site-packages/qiime2/sdk/action.py", line 342, in bound_callable
outputs = self.callable_executor(
File "/home/.conda/envs/qiime2-2023.7/lib/python3.8/site-packages/qiime2/sdk/action.py", line 566, in callable_executor
output_views = self._callable(**view_args)
File "/home/.conda/envs/qiime2-2023.7/lib/python3.8/site-packages/q2_feature_classifier/classifier.py", line 220, in classify_sklearn
seq_ids, taxonomy, confidence = list(zip(*predictions))
File "/home/.conda/envs/qiime2-2023.7/lib/python3.8/site-packages/q2_feature_classifier/_skl.py", line 46, in predict
for calculated in workers(jobs):
File "/home/.conda/envs/qiime2-2023.7/lib/python3.8/site-packages/joblib/parallel.py", line 1952, in call
return output if self.return_generator else list(output)
File "/home//.conda/envs/qiime2-2023.7/lib/python3.8/site-packages/joblib/parallel.py", line 1595, in _get_outputs
yield from self._retrieve()
File "/home/.conda/envs/qiime2-2023.7/lib/python3.8/site-packages/joblib/parallel.py", line 1699, in _retrieve
self._raise_error_fast()
File "/home/.conda/envs/qiime2-2023.7/lib/python3.8/site-packages/joblib/parallel.py", line 1734, in _raise_error_fast
error_job.get_result(self.timeout)
File "/home/.conda/envs/qiime2-2023.7/lib/python3.8/site-packages/joblib/parallel.py", line 736, in get_result
return self._return_or_raise()
File "/home/.conda/envs/qiime2-2023.7/lib/python3.8/site-packages/joblib/parallel.py", line 754, in _return_or_raise
raise self._result
_pickle.PicklingError: Could not pickle the task to send it to the workers.

Plugin error from feature-classifier:

Could not pickle the task to send it to the workers.

See above for debug info.

Hi @Shuai_Man

It looks like you are out of space on your device! Clear up your device and try again!
:turtle:

1 Like

Dear cherman2,

== this may also work for others who get Errno 28 with feature-classifier ==

I have the same issue with feature-classifier classify-sklearn, Qiime2 version 2023.7, set another tempdir in my home dir, still had the problem (>100 GB free space, RAM+swap > 32 GB, only user, small classifier database, "No space left on device" cannot be the real issue). The command does use ~5 GB of tempdir space, by the way.

What did the trick for me is (tempdir still in home dir):
--p-n-jobs 1
It is slower than using more cores but at least it works.

1 Like

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.