Hi, @lizgehret,
I am back now and resumed the original cache.py.
- Make sure you're setting
TMPDIR as an environment variable not just a shell variable (TMPDIR="/mnt/tmp" sets it as a shell variable). You'll need export TMPDIR="/mnt/tmp".
I used export TMPDIR="/mnt/tmp".
- You'll also need to make sure
/mnt/tmp exists (python doesn't like it if the dir you set as your tmp doesn't exist).
- You'll also need to make sure you have write access to
/mnt/tmp/
I manually created /mnt/tmp by mkdir /mnt/tmp.
Here comes the error message.
Traceback (most recent call last):
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/bin/qiime", line 11, in <module>
sys.exit(qiime())
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/click/core.py", line 1686, in invoke
sub_ctx = cmd.make_context(cmd_name, args, parent=ctx)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/click/core.py", line 943, in make_context
self.parse_args(ctx, args)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/q2cli/click/command.py", line 60, in parse_args
value, args = param.handle_parse_result(ctx, opts, args)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/click/core.py", line 2400, in handle_parse_result
value = self.process_value(ctx, value)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/q2cli/click/option.py", line 181, in process_value
return super().process_value(ctx, value)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/click/core.py", line 2356, in process_value
value = self.type_cast_value(ctx, value)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/q2cli/click/option.py", line 297, in type_cast_value
value = super().type_cast_value(ctx, value)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/click/core.py", line 2344, in type_cast_value
return convert(value)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/click/core.py", line 2316, in convert
return self.type(value, param=self, ctx=ctx)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/click/types.py", line 83, in __call__
return self.convert(value, param, ctx)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/q2cli/click/type.py", line 70, in convert
with get_used_artifact_cache():
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/q2cli/core/artifact_cache_global.py", line 81, in get_used_artifact_cache
return Cache() if _USED_ARTIFACT_CACHE is None else _USED_ARTIFACT_CACHE
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/qiime2/core/cache.py", line 417, in __init__
self.__init(path=path, process_pool_lifespan=process_pool_lifespan)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/qiime2/core/cache.py", line 421, in __init
temp_cache_path = pathlib.Path(_get_temp_path())
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/qiime2/core/cache.py", line 158, in _get_temp_path
raise ValueError(f"Directory '{cache_dir}' already exists without "
ValueError: Directory '/mnt/tmp/qiime2' already exists without proper permissions '0o41777' set. Current permissions are '0o40777.' This most likely means something other than QIIME 2 created the directory '/mnt/tmp/qiime2' or QIIME 2 failed between creating '/mnt/tmp/qiime2' and setting permissions on it.
According to the message, /mnt/tmp/qiime2 was already created by QIIME 2 (I only created /mnt/tmp) without setting proper permission while encountering the error.
I also noticed, in the source file of cache.py
"""
The cache is used to store unzipped data on disk in a predictable and user
controlled location. This allows us to skip constantly zipping and unzipping
large amounts of data and taking up CPU time when storage space is not an
issue. It also allows us to know exactly what data has been created and where.
By default, a cache will be created under $TMPDIR/qiime2/$USER and all
intermediate data created by QIIME 2 as it executes will be written into that
directory. This means QIIME 2 reserves usage of the $TMPDIR/qiime2 directory.
The user may also specify a new location to be used in place of this default
directory. This location must meet a few criteria.
**1.** It must be writable from any and all locations the QIIME 2 command
intending to use it will be running. This means that in an HPC context, the
location specified for the cache must be writable from the node QIIME 2 will be
executing on.
**2.** It must either not exist or already be a cache. The first time a
directory is specified to be used as a cache, it should not exist. QIIME 2 will
create a cache structure on disk at that location. Any existing directory you
attempt to use as a cache should have been created as a cache by QIIME 2.
"""
According to item 2, QIIME 2 accepts (or requires?) non-exisiting directory. However, setting TMPDIR in such case (not manually creating /mnt/tmp) failed to change the default behavior.
Looking forward to your help~
). I will make sure of these details after completing the current running process.