Hello,
It looks like there was a change in behavior for the TMPDIR variable between QIIME 2 2022.2 and 2022.8. Identical command sets do not work across versions. This was first discovered by the standard microsetta processing pipeline that is largely unchanged for the past few versions of QIIME 2.
On our shared user environment, our admin has strongly advised we set TMPDIR to point to our scratch space as our /tmp is small (and not fast). Doing so with QIIME 2 2022.8 yields an exception:
# set a temporary directory
(qiime2-2022.8) 15:26:07 ([email protected]):[email protected]$ export TMPDIR=$PANFS/tmp
# run a QIIME 2 command
(qiime2-2022.8) 15:26:10 ([email protected]):[email protected]$ qiime diversity beta --i-table raw.minfeat.mindepth.even.biom.qza --o-distance-matrix foobar.qza --p-metric braycurtis
Plugin error from diversity:
[('/panfs/dtmcdonald/tmp/qiime2/dtmcdonald/data/8b14075f-ce90-4492-bed0-efa14e2a1be1/provenance/metadata.yaml', '/panfs/dtmcdonald/tmp/qiime2-provenance-5hqj1mr1/artifacts/8b14075f-ce90-4492-bed0-efa14e2a1be1/metadata.yaml', "[Errno 13] Permission denied: '/panfs/dtmcdonald/tmp/qiime2-provenance-5hqj1mr1/artifacts/8b14075f-ce90-4492-bed0-efa14e2a1be1/metadata.yaml'"), ('/panfs/dtmcdonald/tmp/qiime2/dtmcdonald/data/8b14075f-ce90-4492-bed0-efa14e2a1be1/provenance/VERSION', '/panfs/dtmcdonald/tmp/qiime2-provenance-5hqj1mr1/artifacts/8b14075f-ce90-4492-bed0-efa14e2a1be1/VERSION', "[Errno 13] Permission denied: '/panfs/dtmcdonald/tmp/qiime2-provenance-5hqj1mr1/artifacts/8b14075f-ce90-4492-bed0-efa14e2a1be1/VERSION'"), ('/panfs/dtmcdonald/tmp/qiime2/dtmcdonald/data/8b14075f-ce90-4492-bed0-efa14e2a1be1/provenance/citations.bib', '/panfs/dtmcdonald/tmp/qiime2-provenance-5hqj1mr1/artifacts/8b14075f-ce90-4492-bed0-efa14e2a1be1/citations.bib', "[Errno 13] Permission denied: '/panfs/dtmcdonald/tmp/qiime2-provenance-5hqj1mr1/artifacts/8b14075f-ce90-4492-bed0-efa14e2a1be1/citations.bib'"), ('/panfs/dtmcdonald/tmp/qiime2/dtmcdonald/data/8b14075f-ce90-4492-bed0-efa14e2a1be1/provenance/action/action.yaml', '/panfs/dtmcdonald/tmp/qiime2-provenance-5hqj1mr1/artifacts/8b14075f-ce90-4492-bed0-efa14e2a1be1/action/action.yaml', "[Errno 13] Permission denied: '/panfs/dtmcdonald/tmp/qiime2-provenance-5hqj1mr1/artifacts/8b14075f-ce90-4492-bed0-efa14e2a1be1/action/action.yaml'"), (<DirEntry 'action'>, '/panfs/dtmcdonald/tmp/qiime2-provenance-5hqj1mr1/artifacts/8b14075f-ce90-4492-bed0-efa14e2a1be1/action', "[Errno 13] Permission denied: '/panfs/dtmcdonald/tmp/qiime2-provenance-5hqj1mr1/artifacts/8b14075f-ce90-4492-bed0-efa14e2a1be1/action'"), ('/panfs/dtmcdonald/tmp/qiime2/dtmcdonald/data/8b14075f-ce90-4492-bed0-efa14e2a1be1/provenance', '/panfs/dtmcdonald/tmp/qiime2-provenance-5hqj1mr1/artifacts/8b14075f-ce90-4492-bed0-efa14e2a1be1', "[Errno 13] Permission denied: '/panfs/dtmcdonald/tmp/qiime2-provenance-5hqj1mr1/artifacts/8b14075f-ce90-4492-bed0-efa14e2a1be1'")]
Debug info has been saved to /panfs/dtmcdonald/tmp/qiime2-q2cli-err-almmlvfj.log
# the permissions denied statements are I believe a red herring as I am able to write to `$TMPDIR`, and notably, QIIME's debug info is saved there and accessible:
(qiime2-2022.8) 15:27:10 ([email protected]):[email protected]$ touch $TMPDIR/example
(qiime2-2022.8) 15:27:24 ([email protected]):[email protected]$ cat /panfs/dtmcdonald/tmp/qiime2-q2cli-err-almmlvfj.log
Traceback (most recent call last):
File "/home/mcdonadt/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/q2cli/commands.py", line 339, in __call__
results = action(**arguments)
File "<decorator-gen-359>", line 2, in beta
File "/home/mcdonadt/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/qiime2/sdk/action.py", line 210, in bound_callable
provenance.add_input(name, artifact)
File "/home/mcdonadt/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/qiime2/core/archive/provenance.py", line 425, in add_input
self.inputs[name] = self.add_ancestor(input)
File "/home/mcdonadt/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/qiime2/core/archive/provenance.py", line 167, in add_ancestor
shutil.copytree(
File "/home/mcdonadt/miniconda3/envs/qiime2-2022.8/lib/python3.8/shutil.py", line 557, in copytree
return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
File "/home/mcdonadt/miniconda3/envs/qiime2-2022.8/lib/python3.8/shutil.py", line 513, in _copytree
raise Error(errors)
shutil.Error: [('/panfs/dtmcdonald/tmp/qiime2/dtmcdonald/data/8b14075f-ce90-4492-bed0-efa14e2a1be1/provenance/metadata.yaml', '/panfs/dtmcdonald/tmp/qiime2-provenance-5hqj1mr1/artifacts/8b14075f-ce90-4492-bed0-efa14e2a1be1/metadata.yaml', "[Errno 13] Permission denied: '/panfs/dtmcdonald/tmp/qiime2-provenance-5hqj1mr1/artifacts/8b14075f-ce90-4492-bed0-efa14e2a1be1/metadata.yaml'"), ('/panfs/dtmcdonald/tmp/qiime2/dtmcdonald/data/8b14075f-ce90-4492-bed0-efa14e2a1be1/provenance/VERSION', '/panfs/dtmcdonald/tmp/qiime2-provenance-5hqj1mr1/artifacts/8b14075f-ce90-4492-bed0-efa14e2a1be1/VERSION', "[Errno 13] Permission denied: '/panfs/dtmcdonald/tmp/qiime2-provenance-5hqj1mr1/artifacts/8b14075f-ce90-4492-bed0-efa14e2a1be1/VERSION'"), ('/panfs/dtmcdonald/tmp/qiime2/dtmcdonald/data/8b14075f-ce90-4492-bed0-efa14e2a1be1/provenance/citations.bib', '/panfs/dtmcdonald/tmp/qiime2-provenance-5hqj1mr1/artifacts/8b14075f-ce90-4492-bed0-efa14e2a1be1/citations.bib', "[Errno 13] Permission denied: '/panfs/dtmcdonald/tmp/qiime2-provenance-5hqj1mr1/artifacts/8b14075f-ce90-4492-bed0-efa14e2a1be1/citations.bib'"), ('/panfs/dtmcdonald/tmp/qiime2/dtmcdonald/data/8b14075f-ce90-4492-bed0-efa14e2a1be1/provenance/action/action.yaml', '/panfs/dtmcdonald/tmp/qiime2-provenance-5hqj1mr1/artifacts/8b14075f-ce90-4492-bed0-efa14e2a1be1/action/action.yaml', "[Errno 13] Permission denied: '/panfs/dtmcdonald/tmp/qiime2-provenance-5hqj1mr1/artifacts/8b14075f-ce90-4492-bed0-efa14e2a1be1/action/action.yaml'"), (<DirEntry 'action'>, '/panfs/dtmcdonald/tmp/qiime2-provenance-5hqj1mr1/artifacts/8b14075f-ce90-4492-bed0-efa14e2a1be1/action', "[Errno 13] Permission denied: '/panfs/dtmcdonald/tmp/qiime2-provenance-5hqj1mr1/artifacts/8b14075f-ce90-4492-bed0-efa14e2a1be1/action'"), ('/panfs/dtmcdonald/tmp/qiime2/dtmcdonald/data/8b14075f-ce90-4492-bed0-efa14e2a1be1/provenance', '/panfs/dtmcdonald/tmp/qiime2-provenance-5hqj1mr1/artifacts/8b14075f-ce90-4492-bed0-efa14e2a1be1', "[Errno 13] Permission denied: '/panfs/dtmcdonald/tmp/qiime2-provenance-5hqj1mr1/artifacts/8b14075f-ce90-4492-bed0-efa14e2a1be1'")]
This error seems to be associated with the filesystem itself as setting TMPDIR=$HOME/foo works. That is not a solution though in our environment. Our scratch space is based off Panfs and I suspect there is something unusual going with that specific type of filesystem.
Any ideas?
Best,
Daniel