QIIME2, Training a Classifier, Plugin error from rescript: [Errno 28] No space left on device, export TMPDIR
Hello all,
My name is Bryan and I'm a master's student studying fish diet using cytochrome oxidase I (COI) universal primers. I am following this Building a COI database from BOLD references tutorial on building a database and trained classifier for the target COI region (~180 bases). I have attempted to use evaluate-fit-classifier (rescript) and the q2-feature-classifier (QIIME2), both encounter the same error after many hours of running. evaluate-fit-classifier apparently requires more memory and storage because it includes a series of validation steps at the end. The issue appears to be that I run out of tmp directory storage, and the stated solution to this on other forums is to export to tmp directory to to a location with adequate storage. After doing so, the code still appears to attempting to write to the default location. I have successfully trained the classifier and used the classifier with q2-feature-classifier on a 0.2 subset of the dataset, and failed with a 0.25 subset. Below are the annotated commands and errors.
(this post was edited/updated on 04/26/2022 to reflect the exact code that was used when the problem occurred)
#attach to screen
-bash-4.1$ screen -r 12712.pts-3.khaleesi
#created bash subshell (potential issue)
-bash-4.1$ bash
#put conda' base (root) on PATH (potential issue)
bash-4.1$ conda activate
#activate conda environment
(base) bash-4.1$ conda activate qiime2-2022.2
#navigate to working directory
(qiime2-2022.2) bash-4.1$ cd /projects/pesticide_metagenomics/CLG_analysis_bv/QIIME2
#check current tmp dir storage
(qiime2-2022.2) bash-4.1$ df /tmp
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/vg_centos-lv_root
51475068 40293260 8560368 83% /
#export tmp directory to location with more space
(qiime2-2022.2) bash-4.1$ export TMPDIR = '/projects/temp'
#validate
(qiime2-2022.2) bash-4.1$ echo -k $TMPDIR
’/projects/temp’
#check storage in new location, should be plenty
(qiime2-2022.2) bash-4.1$ df /projects/temp
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/vg_data-projects
13639837696 11687559604 1952278092 86% /projects
#subset by 0.25
qiime rescript subsample-fasta --i-sequences bold_anml_seqs.qza --p-subsample-size 0.25 --p-random-seed 1 --o-sample-sequences bold_anml_seqs_sub0.25.qza --verbose
#attempt training of classifier
qiime feature-classifier fit-classifier-naive-bayes --i-reference-reads bold_anml_seqs_sub0.25.qza --i-reference-taxonomy bold_anml_taxa.qza --o-classifier classifier_q2_0.25.qza
#detatch from screen
Ctrl+N+D
[detached]
#evaluate job status
-bash-4.1$ htop
#error after hours of running
Plugin error from rescript: [Errno 28] No space left on device Debug info has been saved to /tmp/qiime2-q2cli-err-_j6y5a0l.log
I'm concerned that the .log file wrote to the default tmp directory location, which makes me think exporting the variable did not do the trick. Here is a screen grab of the log file error.
I know this is a subject that has been discussed before, but the solutions that I could find in the forum do not appear to solve it. Any help is greatly appreciated!
Best,
-Bryan