My q2-SCNIC plugin depends on the fastspar package which is available in bioconda. It depends on blas. In the osx qiime2 environment file blas is not included at all and so fastspar can be installed fine in a qiime2-dev or qiime2-2018.6 environment on a mac using blas 1.1. In the linux environment file blas is pinned at 1.0 so I get a UnsatisfiableError from conda create. I tried pinning at blas 1.1 in the linux environment file and the environment was able to build fine. Would it be possible to update the blas pin to 1.1 in the linux environment or remove it like it is in the osx environment definition?
Yep, I am not disagreeing with you - the environment file has that pin. That doesn’t mean that a QIIME 2 plugin is directly setting it though — the environment files are built by fully resolving all dependencies, then saving them in the env file. So if q2-dada2 depends on dependency foo, and foo sets a pin on blas — we will inherit that in q2-dada2.
I just aged for blas in all of our plugin’s conda recipes and didn’t find anything. Next step is to trace condas resolution order to figure out why a certain version of blas is being declared somewhere.
I am looking into where the dependency is being set, but in the meantime, just to confirm:the release env files for 2018.6 do not have pins set for blas, only the 2018.8 staging files, for both linux and osx.
Thanks @thermokarst. I understand what you are saying now. It seems like my conda isn’t willing to downgrade or upgrade things that are set in the environment file used to create the environment right now. I’m trying to figure out why this is happening and happening different between my laptop, the server I use and in travis.
Okay, I just had to set my environment file to be the ‘normal’ environment file and not the dev version. Is there a way for me to peg to the newest release environment file or should I just update that as new qiime versions get released?
It looks like the version pin is coming in from xopen, which is a dependency of cutadapt, but, it is possible I have misread or misinterpreted to conda logs and package receipts.
Unfortunately not, but, I like the idea of being able to always grab the latest release - @ebolyen, thoughts?
I think this gets into our goal of making QIIME 2 work nicely with conda so we don't need an environment file at all. So I guess the answer is, stay tuned?
Thanks for the responses @ebolyen and @thermokarst. So do you have any advice about getting this fastspar dependency to work with 2018.8? Should I try to see if I can modify any of these recipes to loosen up these dependencies? q2-SCNIC works perfectly with 2018.6 and I got it conda installable but it feels like I’m up against a bit of a wall here.
Let me give the problem a shot and I'll report back. I'm sure something can be worked out. Although it might happen later in the 2018.10 cycle if we do have to re-arrange dependencies (which is something we're happy to do).
Thanks for the patience and sorry for the "dependency hell".
If you install -c conda-forge blas=1.1 in a q2 env it seems to swap the builds about (while retaining the versions) and allows you to install fastspar like q2-SCNIC needs.
This should be a safe thing to do. It would seem that software that needs blas (e.g. 1.1) needs to have openblas (part of the 1.1 metapackage), whereas other packages either don’t care or use MKL builds (1.0 of the blas metapackage).
As I’m looking into the situation it seems the situation with metapackages, the hackery with blas, and how conda update works, prevents a dependency (like fastspar) from upgrading blas like we might want.
For now forcing the version seems to work pretty well:
conda install -c conda-forge blas=1.1
Still looking into the root of the pinned version, since I don’t think packages get anything out of pinning 1.0 (unlike 1.1), so we may be able to request that be changed.
Thanks @ebolyen! I didn’t realize that if you expliciting install at a version it would override what was in the environment file. So I will add a note in my community tutorial that users with 2018.8 have to run conda install -c conda-forge blas=1.1 before installing q2-SCNIC. It would be great if the pin could be upped in 2018.10 or later.
