QIIME 2 is cool! I was able to use the excellent tutorials and docs to analyze data from start to finish!
I am wondering if commands/plugins can work on a stream. Instead of creating a qza file for every single step, is it possible to run the various commands through a pipe and generate fewer qza files? For instance, I would like to have one qza file that contains all data and artifacts from loading data, running deblur, calculating alignment and phylogenetic tree. I tried to stitch together two commands through a simple Unix stream/pipe but didn’t work. Please let me know.
My apologies if this question has been asked already and answered although I couldn’t find from searching the forum.
However, I may be able to point you in a similar but better direction. If you are familiar with programming in python, you can use QIIME2's artifact API to access all the functionality available in QIIME2 and its plugins but without operating through a command-line interface.
Using Artifact API would allow you to stitch together custom workflows and decide precisely what outputs you want to save to file.
It would be possible (with artifact API) to just save the terminal outputs, or select what outputs you want to save to file (avoiding saving intermediate steps), but it would not be possible to combine multiple artifacts into one file.
Thank you for answering all my questions promptly! Will look into QIIME 2's artifact API. I think it would be nice to have a feature to combine multiple artifacts into one qza file.
That's something we've bounced around before! Since every artifact is written inside of the .qza by their UUID it would be pretty trivial to combine them.
The harder part becomes selecting which one you wanted if you are using this combined artifact as input. Suppose you had two artifacts of the same type, how would you say which one you meant to use as input?
If they weren't the same type, then we could just declare them to be an intersection type (e.g. FeatureTable[Frequency] & FeatureData[Sequence]) and use the type to disambiguate, but even that can fall apart if a method accepted something like FeatureTable[Frequency] | FeatureData[Sequence] (not that this particular input makes a lot of sense) as you need to address which artifact was meant to be used again.
I wonder if a syntax like some_meta_artifact.qzm:c6 (where the c6 is just enough of the UUID c6b4c67a-b884-431e-a508-3c82810fd4d9 to tell it apart from the other artifacts) could work.
