After the workshop, I successfully uploaded my .tar.bz2 file to anaconda.org. However, when I try to conda install the plugin, it can’t find the package and installing doesn’t work.
$ conda install -c cduvallet q2_perc_norm
PackagesNotFoundError: The following packages are not available from current channels:
- q2_perc_norm
It seems like conda doesn’t see that I have the package uploaded. If I do conda list -c cduvallet, I get nothing back. I’ve checked that my package is set to public on anaconda, and other than that am not sure how to fix this.
$ conda list -c cduvallet
$
Also a note that conda list -c cduvallet returns nothing on multiple systems (including my laptop in root environment, qiime2-2018.4, 2018.2, and dev environment, and our cluster which runs on linux).
Hey there @cduvallet! I wonder if this error is related to the current path you are in when running the conda install command above — if you are sitting in your q2-perc-normrepo, then there is a folder there called q2_perc_norm - is it possible conda thinks it should be looking in there to install the package from, instead of up towards the heavens (aka anaconda.org)?
On my dev machine I ran the following:
conda install -c cduvallet q2_perc_norm
Solving environment: done
==> WARNING: A newer version of conda exists. <==
current version: 4.5.2
latest version: 4.5.3
Please update conda by running
$ conda update -n base conda
## Package Plan ##
environment location: /Users/matthew/.conda/envs/q2dev
added / updated specs:
- q2_perc_norm
The following packages will be downloaded:
package | build
---------------------------|-----------------
scipy-0.19.1 | py35hf362045_3 14.6 MB
numpy-1.12.1 | py35h655160a_1 3.6 MB
blas-1.0 | mkl 6 KB
scikit-learn-0.19.1 | py35h2b554eb_0 4.8 MB
q2_perc_norm-1.0 | py35_0 11 KB cduvallet
certifi-2018.4.16 | py35_0 142 KB
------------------------------------------------------------
Total: 23.1 MB
The following NEW packages will be INSTALLED:
q2_perc_norm: 1.0-py35_0 cduvallet
The following packages will be UPDATED:
certifi: 2018.4.16-py35_0 conda-forge --> 2018.4.16-py35_0
openssl: 1.0.2o-0 conda-forge --> 1.0.2o-h26aff7b_0
The following packages will be DOWNGRADED:
blas: 1.1-openblas conda-forge --> 1.0-mkl
numpy: 1.12.1-py35_blas_openblas_200 conda-forge [blas_openblas] --> 1.12.1-py35h655160a_1
scikit-learn: 0.19.1-py35_blas_openblas_200 conda-forge [blas_openblas] --> 0.19.1-py35h2b554eb_0
scipy: 0.19.1-py35_blas_openblas_202 conda-forge [blas_openblas] --> 0.19.1-py35hf362045_3
Proceed ([y]/n)?
Downloading and Extracting Packages
scipy 0.19.1############################################################################################################################################################################################ | 100%
numpy 1.12.1############################################################################################################################################################################################ | 100%
blas 1.0################################################################################################################################################################################################ | 100%
scikit-learn 0.19.1##################################################################################################################################################################################### | 100%
q2_perc_norm 1.0######################################################################################################################################################################################## | 100%
certifi 2018.4.16####################################################################################################################################################################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Then, to check that the conda package was installed:
conda list | ag perc
q2_perc_norm 1.0 py35_0 cduvallet
One other thing that jumps out at me is that there is only a mac build of your plugin up on anaconda.org, that means if you are on a linux machine and try to install this package, conda will fail to resolve a valid linux-formatted package. Two ways around this come to mind:
Build a version of your package on a linux machine (more work, need to have a linux computer)
Build a noarch version of your package on your mac (less work).
The noarch package lets you upload one build that is compatible on multiple platforms - the caveat here is that it is up to you to know if that is the case or not (it looks like you should be safe to go with this option, I think )
It looks like the -c flag when running conda list is for displaying "canonical package names" (seriously? I think -c everywhere else in conda-land is for setting the channel...)
I think this might give you the info you are looking for:
conda search -c cduvallet q2_perc_norm flashcards e90e4e7 ✗
Loading channels: done
# Name Version Build Channel
q2_perc_norm 1.0 py35_0 cduvallet
Hm, okay. So it looks like conda install works on my Mac if I'm in a qiime2 environment (qiime2-2018.2 and qiime2-dev both work).
For posterity's sake, it seems like conda doesn't know where to install the various qiime2 modules from if I try to install my plugin from a non-qiime2 environment. But when I'm in the qiime2 environment, those modules are already installed so the installation of my plugin works. Is that a correct interpretation of what's going on?
Just out of curiosity, is there a way to tell conda where to install the various dependencies from? i.e. tell it to look in the qiime2, bioconda, etc channels for the various dependencies.
And I guess the lack of a linux build explains why installing failed on my cluster. I'll look into the noarch thing, thanks!
Dependency resolution and channels get into the magic inner-workings of conda, which remains a mystery to me! With that said, we have generally had great luck with using the -c $CHANNEL business that conda exposes. Apparently the newer versions ofconda support an install syntax that allows specifying the the channel on a per-package basis, ! I think that looks something like this:
But, I haven't really played with that myself, YMMV.
We use the following third-party channels for installing packages:
-c conda-forge
-c defaults
-c bioconda
-c biocore
And the way conda uses that, is it searches through each channel (in the order listed), for the dependency requested. If it doesn't find what it is looking for, it moves to the next channel.
Hope that helps - let us know if you need a hand with the noarch build!
)
! I think that looks something like this: