Does QIIME2 have a per session tempdir that is managed by QIIME2 and accessable through a QIIME2 API that a aplugin should use? Or should a plugin set up and tear down its own tempdir? I'm looking for advice on what the recommended practice is.
1 Like
Hello @Adam_Rivers,
This question has a few potential answers; can you tell me more about what specifically you are trying to do with this tempdir?
Thank you
2 Likes
For each FASTQ pair in a .qza file, I create a temporary directory with Python's tempdir library then create a FASTQ file of merged reads, a de-replicated FASTA file, and an HMMSearch output file. Then I write all the output trimmed FASTQ files to one .qza file. then I delete the temporary directory with shutil.
1 Like
That sounds like a sane pattern to me, and I would keep on doing what you're doing for the time being. You prompted a bit of a discussion about this on our end, and I'll let you know if anything changes and if there are any new recommendations in the next release.
This is something we've been thinking about but have to be a bit careful with due to how often QIIME 2 is run on an HPC with an NFS. If we just set a tmpdir that could have been local to a compute node somewhere global to the NFS we could end up spending significantly longer on disk IO.
One thing I will point you at right now that you may or may not be interested in is qiime2.util.duplicate
1 Like