Plugin error from feature-classifier: [Errno 28] No space left on device

User Support
1

Hey!

I have seen that this problem has come up several times in your forum and I tried to read through the different topics in order to solve the problem. Although, I unfortunately think I might need some different advice.

I am using QIIME2 2020.8 version. I am using this version on a centralized computing system so they have installed QIIME2. I am not sure how they have installed it unfortuantaly but I know that they use github easybuild to instal programs. All of the jobs are run in a queue system. I have successfully run this batch job:

#!/bin/bash -l
################### QIIME2 Assign Taxonomy ###################
# Section for defining queue-system variables:
#-------------------------------------
# SLURM-section
#SBATCH --account=nn9800k
#SBATCH --job-name=Assign_Taxonomy
#SBATCH --ntasks=2 --cpus-per-task=4
#SBATCH --mem-per-cpu=32G
#SBATCH --partition=bigmem
#SBATCH --time=10-0:0:0
#SBATCH --mail-type=ALL
##SBATCH -M [email protected]
######################################
# Section for defining job variables and settings:
#-------------------------------------
# Command must be ran in same directory where 5_denoise_dada2_seq_file.qza or 12_denoise_dada2_minfreq60_ASVs.qza file is located

# Must ensure path to classifier is correct for the directory you are in

module load QIIME2/2020.8

#export job_output_dir=/cluster/work/users/$USER/$SLURM_JOB_ID

# Creating aliases and moving files to scratch:
#submit=$SLURM_SUBMIT_DIR
#temp=$job_output_dir
#mkdir -p $temp

#cd $submit
#cp $input $temp
#cd $temp

# Running program, pay attention to command name:
# Assign_Taxonomy

time qiime feature-classifier classify-sklearn --i-reads 5_denoise_dada2_seq_file.qza --i-classifier …/…/…/Classifier_files/Classifiers_Silva138.1/18S/silva-138.1-ssu-nr99-139f_EukBr-classifier.qza --o-classification 14_Unfiltered_ASV_taxonomy.qza

qiime feature-classifier classify-sklearn --i-reads 12_denoise_dada2_minfreq60_ASVs.qza --i-classifier …/…/…/Classifier_files/Classifiers_Silva138.1/18S/silva-138.1-ssu-nr99-139f_EukBr-classifier.qza --o-classification 15_60Filtered_ASV_taxonomy.qza

qiime metadata tabulate --m-input-file 14_Unfiltered_ASV_taxonomy.qza --o-visualization 16_Unfiltered_ASV_taxonomy_visfile.qzv

qiime metadata tabulate --m-input-file 15_60Filtered_ASV_taxonomy.qza --o-visualization 17_60Filtered_ASV_taxonomy_visfile.qzv

# Cleaning up and moving files back to home if desirable:

#cp output.out $submitdir
#cp $input.changed $submitdir

exit 0

It ran successfully when I used a sequence file that was 19KB and 32KB against a trained classifier that was 12MB. But when I try to run a 76KB or a 205KB sequence file against a 160MB classifier I get this error:

Plugin error from feature-classifier:

** [Errno 28] No space left on device**

Debug info has been saved to /tmp/qiime2-q2cli-err-q0_5kqtx.log

If I create a temporary file I am not sure where I can send it. I am suppose to only use my project space. Seems that there is a 15GiB limit on /tmp on nodes. My question is what is the expected sizes of ouputs that is potentially placed in /tmp during calculations? How does the code behaves in relation to storage and file system usage? I believe this problem will have to be addressed with the managers of the centralized computing system but I am not completely sure what size files I am looking at. I was surprised to find that 15GB was not big enough for the tempoary file. This won’t even be the largest file that I will run so I need to know what kind of sizes to expect in order to request access to more. Also, if this is not the issue please let me know how else I might be able to address it. Thanks!

Hannah

2

@hdoris I had the same issue before. Using silva to generate taxonomy would need a lot of temporary space. I am not exactly sure how large it is. But you can set the TMPDIR environmental variable prior to the step to a place that you have permission.

export TMPDIR='/to/the/location/of/Tmpdir/'

On a centralized cluster, I would typically make a directory in my home directory and point that TMPDIR to that directory. I had no problem with that ever since.

Hope this helps.

5 Likes
3

Hi @hdoris,

One other thought is to go into the /tmp/ dir and just empty anything with a qiime2 in the name (rm -r /tmp/qiime2) if you’re the only qiime2 user on the system. I do this periodically on my projects because occasionally I do fill up my tmp dir.

Best,
Justine

4

I am likely not the only qiime2 user on the system but I will check. Thanks!

1 Like
5

Thank you so much I will try this!

1 Like
6

@hdoris,

Even if you’re not the only qiime2 user on the system, if you’re the only one currently running commands, it can be helpful to clean up as well.

Best,
Justine

7

@jwdebelius Good to know! Thanks!

1 Like
8

Sorry I am having a little trouble with this. I tried running this command:

qiime feature-classifier classify-sklearn --i-reads 5_denoise_dada2_seq_file.qza --i-classifier …/…/…/Classifier_files/Classifiers_Silva138.1/16S/silva-138.1-ssu-nr99-515f-806r-classifier.qza --o-classification 14_Unfiltered_ASV_taxonomy.qza TMPDIR = …/…/…/TempDir

It ran for a bit and then stopped with the error:

(1/1) Got unexpected extra arguments (TMPDIR= …/…/…/TempDir_Test/)

How am I suppose to create send it to my created temporary directory? Is there a build in temp directory setting that needs manual override for the feature-classifier command?

9

Is not a qiime command, it is a bash command. Essentially, you are setting the bash environmental variable TMPDIR. Qiime will use this environmental variable.

so basically:

export TMPDIR='/to/the/location/of/Tmpdir/'

qiime feature-classifier classify-sklearn --i-reads 5_denoise_dada2_seq_file.qza --i-classifier …/…/…/Classifier_files/Classifiers_Silva138.1/16S/silva-138.1-ssu-nr99-515f-806r-classifier.qza --o-classification 14_Unfiltered_ASV_taxonomy.qza
2 Likes
10

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.