Temporary Files created from DADA2 denoise process

[jairideout (Jai Ram Rideout)]

Hi @moryfranklin22!

@moryfranklin22

Is there any way of designating an alternate temp file directory before running the denoise process? Or is there any way of redirecting the temp files to a specific directory within the qiime dada2 denoise-paired command itself?

We don't currently have a great way to do this in QIIME 2 -- in the future we'll have a configuration system that'll let users set a temporary directory and other host-related details. There's an open issue to track progress on the configuration system.

In the meantime, you can override the default (platform dependent) location by setting the TMPDIR environment variable to point to your temporary directory. Be sure to use an absolute path. For example:

export TMPDIR="/home/foo/my-temp-dir"

@moryfranklin22

Last, is there an estimate of disk storage required for processing of the 10 GB paired-end artifact file? Does the demultiplexed qza file copy itself into temporary storage?

Yes, loading the demux .qza will extract the ZIP file into a temporary directory, so that'll take up another 10GB of disk space (it'll get cleaned up after the analysis finishes). DADA2 will also take up disk space writing its output and there may be some other disk usage during the analysis. Overall, I'd expect to see around ~10GB of temporary storage being used during your DADA2 analysis. Please let us know if that's not the case!

Note: we've discussed mounting ZIP files (i.e. .qza and .qzv files) in order to avoid duplicating data in temporary storage. This may be possible with FUSE but I'm not sure when this feature will be available in QIIME 2.