I have a very large dataset that we will be running deblur on tomorrow. There are ~30,000 files, resulting in ~3 terrabytes of data. If I run QIIME2 deblur, will this be a duplication of space? Is there a limit within QIIME2 as to how much data it is willing to handle for this command? And finally, if it is too much, will the command fail immediately or would it make an attempt and fail when it runs out of memory or space?
Our alternative solution is to install and run deblur directly rather than via QIIME2, and we didn’t want to waste a lot of time discovering it wasn’t going to work if possible. Any insight is appreciated!