Plugin error from feature-classifier - No space left on device

User Support
,
1

Hello ,

not sure how to deal with this error. This was the command:

(qiime2-2018.2) mpb@virgo:/data3/mpb/MySeq2018_03_26_Mouse/demux$ qiime feature-classifier classify-sklearn \
>   --i-classifier /data/Qiime2_classifiers/silva-119-99-515-806-nb-classifier.qza \
>   --i-reads dataseqs.qza \
>   --p-n-jobs -1 \
>   --o-classification /data3/mpb/MySeq2018_03_26_Mouse/demux/taxonomy.qza

The classifier.qza is the one provided on the Qiime2 site, which is a 155Mb qza archive. The sequence file I am trying to run is tiny, 90kb. I have qiime2 installed on a partition disk that has 18Gb available. After I launch the command I can see how the temp files are filling up the disk space, all 18Gb and then gives the

Plugin error from feature-classifier:

  [Errno 28] No space left on device

Debug info has been saved to /tmp/qiime2-q2cli-err-0gm1j2x2.log

This is the debug log file:

Traceback (most recent call last):
  File "/home/mpb/miniconda3/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/commands.py", line 246, in __call__
    results = action(**arguments)
  File "<decorator-gen-272>", line 2, in classify_sklearn
  File "/home/mpb/miniconda3/envs/qiime2-2018.2/lib/python3.5/site-packages/qiime2/sdk/action.py", line 222, in bound_callable
    spec.view_type, recorder)
  File "/home/mpb/miniconda3/envs/qiime2-2018.2/lib/python3.5/site-packages/qiime2/sdk/result.py", line 261, in _view
    result = transformation(self._archiver.data_dir)
  File "/home/mpb/miniconda3/envs/qiime2-2018.2/lib/python3.5/site-packages/qiime2/core/transform.py", line 59, in transformation
    new_view = transformer(view)
  File "/home/mpb/miniconda3/envs/qiime2-2018.2/lib/python3.5/site-packages/q2_feature_classifier/_taxonomic_classifier.py", line 71, in _1
    tar.extractall(dirname)
  File "/home/mpb/miniconda3/envs/qiime2-2018.2/lib/python3.5/tarfile.py", line 1998, in extractall
    numeric_owner=numeric_owner)
  File "/home/mpb/miniconda3/envs/qiime2-2018.2/lib/python3.5/tarfile.py", line 2040, in extract
    numeric_owner=numeric_owner)
  File "/home/mpb/miniconda3/envs/qiime2-2018.2/lib/python3.5/tarfile.py", line 2110, in _extract_member
    self.makefile(tarinfo, targetpath)
  File "/home/mpb/miniconda3/envs/qiime2-2018.2/lib/python3.5/tarfile.py", line 2158, in makefile
    copyfileobj(source, target, tarinfo.size, ReadError)
  File "/home/mpb/miniconda3/envs/qiime2-2018.2/lib/python3.5/tarfile.py", line 244, in copyfileobj
    dst.write(buf)
OSError: [Errno 28] No space left on device

I dont understand why it needs such a huge temp space, is there a way to direct the temp directory for this plugin to a different location? I guess I could reinstall on another disk but this disk I am using has flash memory.

Thank you

Mircea Podar

3

Hi @mpodar, even though your dataset might be small, the feature classifier still needs a lot of space to do its work! This error definitely indicates you are running out of disk space :frowning:

QIIME 2 might be installed on this partition, but, most likely your temporary directory isn't using that partition, which is probably why you are running out of space. You can force Python to use a new tempdir location by doing something like the following (please change the names as necessary for your machine):

mkdir /data3/mbp/mpodars-new-temporary-directory
export TMPDIR='/data3/mbp/mpodars-new-temporary-directory'

This is assuming that /data3 is the partition that has enough disk space free. Let us know how that goes for you! :t_rex:

5

hi Matthew,

thanks. I already tried changing the environment variable in my .bashrc and force to a new temp directory on a bigger partition, as you indicated. Unfortunately that plugin (or something hardcoded in the python/conda?) does not want to abide to that redirecting and still does it (and crashes) the system tmp. The tmp is indeed on a system disk with 18G of space (not the conda/qiime disk, which has >2Tb) and that much space is being gulped out by the plugin (watching it go on the system monitor)… I found in another post that you guys are planning to enable the choice of using tmp other than the system default in some new releases (?) as others have had issues with other plugins that need lots of temp space. I’ll be watching for that update. In the meantime I installed it on an iMac that has a big hard drive (and few processors), lets see how long will take to do the taxonomy with that machine.

Mircea

7

Hi @mpodar

If you are setting the env var in your bashrc, then launching a new shell, then activating the QIIME 2 environment, it is very likely that you are rewriting the env var accidentally. After activating your qiime 2 env, please run the following and double-check that your new temp dir env var is set correctly:

env

You are looking for the TMPDIR variable to be set to your new temp dir location. Please copy-and-paste the output of this command here.

This is just a convenience config you are referring to --- the env var we are discussing right now is the canonical way to instruct python to use a temp dir location, this is all that convenience config setting would do behind the scenes.

1 Like
9

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.