Hello,
due to some decontam updates in q2-2024.5 (--i-rep-seqs), I installed q2-amplicon-2024.5 and started to update some community plugins, too.
The first one was empress, and I realized that the pip installer downgraded scikit-bio-0.5.9 scipy-1.10.1 to previous versions. I'm afraid that downgrading packages could affect the funcionality of the q2 core release. Therefore, I'm a bit unsure whether I should continue and install other community plugins as long as the compatibility with 2024.5 is not certain (e.g. gemelli, SCNIC, Greengenes, itsxpress).
I am aware of the plans of the q2 core team to re-think the support of the community plugins (as stated on the Q2 library page), and I can imagine that this is a big task! Thank you all for your great job! I am wondering whether there is already some kind of compatibility table of community plugins with recent q2 releases in order to avoid installing plugins that are not compatible to certain q2 releases. That would be a great help in the meantime.
It's not so much an issue with iow, but that QIIME 2 2024.5 depends on an out-of-date version. It's complicated further by iow not yet being compatible with Cython 3.
Thank you! I am not sure how Cython 3 plays into this; as far as I can tell, neither the 2024.2 nor 2024.5 releases of QIIME 2 impose any explicit restriction on the Cython version used (nor does scikit-bio, I think).
In any case—I agree that the iow version used by QIIME 2 should probably be updated for the next Q2 release. Because Q2 2024.5 pins iow=1.0.5 as well as scikit-bio=0.6.0, and because iow 1.0.5 requires scikit-bio >= 0.5.0, < 0.6.0, the Q2 2024.5 environment is inherently self-contradictory. (This is also true for deblur=1.1.1.) I guess this kind of "contradiction" explains why installing other packages that rely on iow causes pip to downgrade these things, in an attempt to retroactively fix the environment.
Just wanted to jump in here with some additional context- our dependency on iow is an indirect one (as are many of the dependencies present in our resolved environment files). If you're interested in seeing what our direct dependencies are (and their associated versions), you can find those in our seed environments within the distributions repository (under the QIIME 2 org). Here's what our seed environment contains for the amplicon distribution's 2024.10 release:
We do have a direct dependency on scikit-bio (which is currently pinned at 0.6.0) - but we are also limited in our pinned versions by what is available on anaconda (either under conda-forge or bioconda channels). The version of skbio we are using has only been on conda for ~6 months, and we updated this in the 2024.5 release.
All this to say - we do our best to keep our direct dependencies up to date, but getting a resolved environment can be trickier than it seems (with all of the downstream dependencies and available versions present on conda).
Thank you for the clarification! Knowing where the seed environment is located is super helpful (I was kind of assuming that the Q2 devs just put together the massive thousand-line YML file together manually lol).
For sure—managing all of this seems like a herculean task (thank you!) To my understanding, I think the workaround linked above of upgrading iow should probably be sufficient for the time being (for users who want to use Empress / Gemelli / etc in Q2 2024.5).
