Hello,
I am trying to convert a midori2 database downloaded specifically in the qiime2 format (MIDORI2_UNIQ_NUC_GB261_srRNA_QIIME.fasta) to the qiime2 archive format, but I am having issues with space in the tmp folder. I saw that there was many discussions on this topic on the forum and I went through them but couldn't find a solution that worked. I am running Qiime2 on a computing cluster from the digital reasearch alliance of Canada.
Here are the commands I used and the error message I got:
#SBATCH --mem=150G
#SBATCH --cpus-per-task=31
#SBATCH --time=2:00:00
#SBATCH --account=def-mcristes
#SBATCH --mail-type=ALL
module load StdEnv/2023 qiime2/2024.5
cd /home/msalamon/projects/def-mcristes/msalamon/scripts/QIIME2_taxonomy/12SMimammal/
export TMPDIR='/home/msalamon/AMPtk/TEMP/'
qiime tools import
--input-path /lustre03/project/6006581/msalamon/scripts/QIIME2_taxonomy/12SMimammal/MIDORI2_UNIQ_NUC_GB261_srRNA_QIIME.fasta
--output-path /lustre03/project/6006581/msalamon/scripts/QIIME2_taxonomy/12SMimammal/MIDORI2_UNIQ_NUC_GB261_srRNA_QIIME.fasta.qza
--type 'FeatureData[Sequence]'
Matplotlib created a temporary cache directory at /tmp/matplotlib-qra2psto because the default path (/home/qiime2/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDI>
Traceback (most recent call last):
File "/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/q2cli/builtin/tools.py", line 852, in _import
artifact = qiime2.sdk.Artifact.import_data(
File "/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/sdk/result.py", line 332, in import_data
return cls.from_view(type, view, view_type, provenance_capture,
File "/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/sdk/result.py", line 360, in _from_view
result = transformation(view, validate_level)
File "/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/core/transform.py", line 70, in transformation
new_view = transformer(view)
File "/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/core/transform.py", line 224, in wrapped
new_view.file.write_data(file_view, self._wrapped_view_type)
File "/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/plugin/model/directory_format.py", line 93, in write_data
result.path._move_or_copy(self.path_maker(**kwargs))
File "/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/core/path.py", line 45, in _move_or_copy
return self._copy_dir_or_file(other)
File "/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/core/path.py", line 35, in _copy_dir_or_file
return shutil.copy(str(self), str(other))
File "/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/shutil.py", line 427, in copy
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/shutil.py", line 277, in copyfile
_fastcopy_sendfile(fsrc, fdst)
File "/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/shutil.py", line 166, in _fastcopy_sendfile
raise err from None
File "/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/shutil.py", line 152, in _fastcopy_sendfile
sent = os.sendfile(outfd, infd, offset, blocksize)
OSError: [Errno 28] No space left on device: '/lustre03/project/6006581/msalamon/scripts/QIIME2_taxonomy/12SMimammal/MIDORI2_UNIQ_NUC_GB261_srRNA_QIIME.fasta' -> '/tmp/q2-DNASequencesDirectoryFormat-qzk>
An unexpected error has occurred:
[Errno 28] No space left on device: '/lustre03/project/6006581/msalamon/scripts/QIIME2_taxonomy/12SMimammal/MIDORI2_UNIQ_NUC_GB261_srRNA_QIIME.fasta' -> '/tmp/q2-DNASequencesDirectoryFormat-qzkf0v3f/d>
See above for debug info.
Matplotlib created a temporary cache directory at /tmp/matplotlib-z8hrb7fq because the default path (/home/qiime2/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDI>
Usage: qiime feature-classifier classify-consensus-vsearch
(1/1) Invalid value for '--i-reference-reads': /lustre03/project/6006581/msal
amon/scripts/QIIME2_taxonomy/12SMimammal/MIDORI2_UNIQ_NUC_GB261_srRNA_QIIME.
fasta.qza does not exist.
I tried changing the TMPDIR in different ways to other directories including in the home directory which should have sufficient space:
export TMPDIR=$SLURM_TMPDIR
or
cd /home/msalamon/projects/def-mcristes/msalamon/scripts/QIIME2_taxonomy/12SMimammal/
mkdir -p TEMP
export TMPDIR='/home/msalamon/projects/def-mcristes/msalamon/scripts/QIIME2_taxonomy/12SMimammal/TEMP'
However I still got the same error message, so it doesn't seem like I was able to change the tmp directory. I also tried increasing the memory to 150G (the fasta file itself is ~110MB) but this didn't change the issue.
Thank you for your help,
Mathilde Salamon