Hey everyone!
We've just updated our release with a fix for OS X which would produce errors such as:
ValueError: Path: 'None' already exists and is not a cache
This release has been a trial-by-fire for a new feature that will be officially announced in November, called "Artifact Cache". We're sorry for the bumpy ride, but appreciate everyone's patience as we iron out these issues.
Briefly, this occured because we now create a single directory structure in your $TMPDIR
instead of littering it with lots of small directories. However, to make sure things worked right, we would include a VERSION
file at the root to double check that the directory is what we expect.
Unfortunately, OS X deletes temporary files that have not been accessed in at least 3 days, which meant if you closed your laptop for a weekend, there would be a good chance OS X had deleted this file and QIIME 2 would choose not to proceed with loading anything out of an abundance of concern for things no longer matching. (We now allow this mismatch for the temporary cache and just recreate it, you should see a user-warning whenever we do this.)
Some of you may recognize this unfortunate behavior if you've ever run a 3+ day long job on OS X . As QIIME 2 generally could not, depending on the specific action, rely on any of it's data to still exist, you would see some really disappointing error about a file not existing.
The really great news, is we are pretty sure we've fixed this long-standing bug as well. We now spawn a very light-weight thread which accesses each file in the new cache every 6 hours to defeat this behavior of OS X (or potentially a particularly aggressive sysadmin's scripts on linux). Once your action is done, the thread ends and OS X is free to destroy anything (although we've already cleaned up at that point).