# still default behavior after setting TMPDIR
**URL:** https://forum.qiime2.org/t/still-default-behavior-after-setting-tmpdir/32926
**Category:** General Discussion
**Created:** [April 8, 2025, 10:31am UTC](https://forum.qiime2.org/t/still-default-behavior-after-setting-tmpdir/32926 "2025-04-08T10:31:06Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@b87401116](https://forum.qiime2.org/u/b87401116)
#### Post date: [April 8, 2025, 10:31am UTC](https://forum.qiime2.org/t/still-default-behavior-after-setting-tmpdir/32926/1 "2025-04-08T10:31:06Z")
Hi,
I experienced difficulties trying to assign TMPDIR to a large disk, similar to this post.
> [@Is there something unusual about how the TMPDIR is being set in 2023.2?](https://forum.qiime2.org/t/is-there-something-unusual-about-how-the-tmpdir-is-being-set-in-2023-2/26402):
>
> Hello, I'm running into an issue related to how the TMPDIR is being set in QIIME 2 2023.2. Here is the code I am running: source activate qiime2-2023.2 set -x set -e export TMPDIR=/panfs/jpshaffer/tmp2 qiime diversity beta-phylogenetic \ --i-table /projects/emp/emp-allergy/data/emp\_16s\_release2\_qiita\_with\_release1\_90bp\_gg2\_asv\_rar5k.qza \ --p-metric 'weighted\_unifrac' \ --p-threads 60 \ --i-phylogeny /databases/gg/2022.10/2022.10.phylogeny.asv.nwk.qza \ --o-distance-matrix /proje…
In brief, setting TMPDIR did not change the default behavior. I tried to debug and found that `tempfile.gettempdir()` in `cache.py` did not return TMPDIR.
```auto
(qiime2-amplicon-2024.10) yc@yc:/mnt/research/microbiota/analysis/202504_LSG$ echo $TMPDIR
/mnt/tmp
(qiime2-amplicon-2024.10) yc@yc:/mnt/research/microbiota/analysis/202504_LSG$ python
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tempfile
>>> tempfile.gettempdir()
'/tmp'
>>>
```
According to some post, `tempfile.gettempdir()` will validate directory assigned by TMPDIR. If the validation process failed, it will fall back to default.
I am not sure if it is the key problem causing the behavior but would ask for help whether this problem could be resolved.
Many thanks
YC
---
### Author:  [@lizgehret](https://forum.qiime2.org/u/lizgehret)
#### Post date: [April 8, 2025, 8:54pm UTC](https://forum.qiime2.org/t/still-default-behavior-after-setting-tmpdir/32926/4 "2025-04-08T20:54:23Z")
---
### Author:  [@lizgehret](https://forum.qiime2.org/u/lizgehret)
#### Post date: [April 9, 2025, 5:07pm UTC](https://forum.qiime2.org/t/still-default-behavior-after-setting-tmpdir/32926/5 "2025-04-09T17:07:06Z")
Hi @b87401116,
Thanks for reaching out! Here are a few suggestions:
1. Make sure you're setting `TMPDIR` as an environment variable not just a shell variable (`TMPDIR="/mnt/tmp"` sets it as a shell variable). You'll need `export TMPDIR="/mnt/tmp"`.
2. You'll also need to make sure `/mnt/tmp` exists (python doesn't like it if the dir you set as your tmp doesn't exist).
3. You'll also need to make sure you have write access to `/mnt/tmp/`.
Hope this helps! Let us know if you run into any further issues.
Cheers 🦎
---
### Author:  [@lizgehret](https://forum.qiime2.org/u/lizgehret)
#### Post date: [April 9, 2025, 5:07pm UTC](https://forum.qiime2.org/t/still-default-behavior-after-setting-tmpdir/32926/6 "2025-04-09T17:07:08Z")
---
### Author:  [@b87401116](https://forum.qiime2.org/u/b87401116)
#### Post date: [April 10, 2025, 2:40am UTC](https://forum.qiime2.org/t/still-default-behavior-after-setting-tmpdir/32926/7 "2025-04-10T02:40:50Z")
Hi @lizgehret,
Thank you for the reply. The computer is now running on a dada2 process (I modified `cache.py` to bypass this problem...) and I am not sure when it will finish (D3 now 😆). I will make sure of these details after completing the current running process.
Thank you!
---
### Author:  [@b87401116](https://forum.qiime2.org/u/b87401116)
#### Post date: [April 11, 2025, 4:09pm UTC](https://forum.qiime2.org/t/still-default-behavior-after-setting-tmpdir/32926/8 "2025-04-11T16:09:16Z")
Hi, @lizgehret,
I am back now and resumed the original `cache.py`.
> 1. Make sure you're setting `TMPDIR` as an environment variable not just a shell variable (`TMPDIR="/mnt/tmp"` sets it as a shell variable). You'll need `export TMPDIR="/mnt/tmp"`.
I used `export TMPDIR="/mnt/tmp"`.
> 1. You'll also need to make sure `/mnt/tmp` exists (python doesn't like it if the dir you set as your tmp doesn't exist).
> 2. You'll also need to make sure you have write access to `/mnt/tmp/`
I manually created `/mnt/tmp` by `mkdir /mnt/tmp`.
Here comes the error message.
```auto
Traceback (most recent call last):
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/bin/qiime", line 11, in
sys.exit(qiime())
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/click/core.py", line 1686, in invoke
sub_ctx = cmd.make_context(cmd_name, args, parent=ctx)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/click/core.py", line 943, in make_context
self.parse_args(ctx, args)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/q2cli/click/command.py", line 60, in parse_args
value, args = param.handle_parse_result(ctx, opts, args)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/click/core.py", line 2400, in handle_parse_result
value = self.process_value(ctx, value)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/q2cli/click/option.py", line 181, in process_value
return super().process_value(ctx, value)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/click/core.py", line 2356, in process_value
value = self.type_cast_value(ctx, value)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/q2cli/click/option.py", line 297, in type_cast_value
value = super().type_cast_value(ctx, value)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/click/core.py", line 2344, in type_cast_value
return convert(value)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/click/core.py", line 2316, in convert
return self.type(value, param=self, ctx=ctx)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/click/types.py", line 83, in __call__
return self.convert(value, param, ctx)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/q2cli/click/type.py", line 70, in convert
with get_used_artifact_cache():
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/q2cli/core/artifact_cache_global.py", line 81, in get_used_artifact_cache
return Cache() if _USED_ARTIFACT_CACHE is None else _USED_ARTIFACT_CACHE
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/qiime2/core/cache.py", line 417, in __init__
self.__init(path=path, process_pool_lifespan=process_pool_lifespan)
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/qiime2/core/cache.py", line 421, in __init
temp_cache_path = pathlib.Path(_get_temp_path())
File "/mnt/mambaforge/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/qiime2/core/cache.py", line 158, in _get_temp_path
raise ValueError(f"Directory '{cache_dir}' already exists without "
ValueError: Directory '/mnt/tmp/qiime2' already exists without proper permissions '0o41777' set. Current permissions are '0o40777.' This most likely means something other than QIIME 2 created the directory '/mnt/tmp/qiime2' or QIIME 2 failed between creating '/mnt/tmp/qiime2' and setting permissions on it.
```
According to the message, `/mnt/tmp/qiime2` was already created by QIIME 2 (I only created `/mnt/tmp`) without setting proper permission while encountering the error.
I also noticed, in the source file of `cache.py`
```auto
"""
The cache is used to store unzipped data on disk in a predictable and user
controlled location. This allows us to skip constantly zipping and unzipping
large amounts of data and taking up CPU time when storage space is not an
issue. It also allows us to know exactly what data has been created and where.
By default, a cache will be created under $TMPDIR/qiime2/$USER and all
intermediate data created by QIIME 2 as it executes will be written into that
directory. This means QIIME 2 reserves usage of the $TMPDIR/qiime2 directory.
The user may also specify a new location to be used in place of this default
directory. This location must meet a few criteria.
**1.** It must be writable from any and all locations the QIIME 2 command
intending to use it will be running. This means that in an HPC context, the
location specified for the cache must be writable from the node QIIME 2 will be
executing on.
**2.** It must either not exist or already be a cache. The first time a
directory is specified to be used as a cache, it should not exist. QIIME 2 will
create a cache structure on disk at that location. Any existing directory you
attempt to use as a cache should have been created as a cache by QIIME 2.
"""
```
According to item 2, QIIME 2 accepts (or requires?) non-exisiting directory. However, setting TMPDIR in such case (not manually creating `/mnt/tmp`) failed to change the default behavior.
Looking forward to your help~
---
### Author:  [@b87401116](https://forum.qiime2.org/u/b87401116)
#### Post date: [April 12, 2025, 4:32am UTC](https://forum.qiime2.org/t/still-default-behavior-after-setting-tmpdir/32926/9 "2025-04-12T04:32:39Z")
hi, @lizgehret,
I think I partially figured out the major problem. This is likely to occur due to filesystem. Previously I assigned TMPDIR to a filesystem of `NTFS`. After change the TMPDIR to another filesystem of `ext4` (also manually created a directory pointed by TMPDIR), QIIME 2 was able to create `qiime2` subdirectory with correct permission and is currently using the directory!
```auto
(base) yc@yc:/mnt/ex4_partition/yc/tmp$ ls -alh
total 24K
drwxrwxr-x 5 yc yc 4.0K Apr 12 03:35 .
drwxr-xr-x 22 yc yc 4.0K Apr 12 03:31 ..
drwx------ 2 yc yc 4.0K Apr 12 03:35 Rtmpmfkng8
drwxrwxrwt 3 yc yc 4.0K Apr 12 03:33 qiime2
-rw------- 1 yc yc 1.2K Apr 12 03:35 qiime2-q2cli-err-pbxao4xl.log
drwx------ 6 yc yc 4.0K Apr 12 03:35 tmprl7q0uu2
```
According to `chatGPT` (so please still have some doubt on this), directory in `NTFS` filesystem was not able to be set for appropriate permission. I tested and confirmed that `chmod 1666` will do nothing on setting permission without an error message.
Hope this will help!
---
### Author:  [@ebolyen](https://forum.qiime2.org/u/ebolyen)
#### Post date: [April 15, 2025, 7:09pm UTC](https://forum.qiime2.org/t/still-default-behavior-after-setting-tmpdir/32926/10 "2025-04-15T19:09:24Z")
Great troubleshooting @b87401116,
I would say ChatGPT has that basically right. NTFS is a pretty different concept from many Unix based filesystems, so it probably shouldn't surprise us that these permissions are over-fit and break on NTFS.
---
### Author:  [@system](https://forum.qiime2.org/u/system)
#### Post date: [May 17, 2025, 1:10am UTC](https://forum.qiime2.org/t/still-default-behavior-after-setting-tmpdir/32926/11 "2025-05-17T01:10:17Z")
This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.