Conda repositories for contributed plugin packages

conda

(Adam Rivers) #1

I’ve developed a Qiime2 plugin, q2-itsxpress, which is available from Github, Pypi and the underlying tool is available from Github Pypi and Bioconda.

I’d like to offer a conda distribution of q2_itsxpress but I cannot designate channels in Bioconda so I can’t include q2_types and qiime2 in the recipe. I see that Qiime2 and Qiime core plugins are available through Anaconda Cloud. Would distributing my contributed plugin through Anaconda Cloud be the best solution?

Adam


Fungal ITSx and qiime2
(Evan Bolyen) #2

(Matthew Ryan Dillon) #3

Hey there @Adam_Rivers! Super excited to hear about q2-itsexpress!

Distributing a plugin through Anaconda Cloud would be a pretty straightforward option, although adding a recipe to bioconda or conda-forge is also a great option. @cduvallet wrote up a great tutorial on publish to anaconda (it will be published on our dev docs soon, too!).

This is where multiple channels comes in handy!

# Here is a snippet of our channel order we are currently using
-c qiime2 \
-c conda-forge \
-c defaults \
-c bioconda \
-c biocore \

Building conda packages can be a bit of a daunting task — if you get stuck feel free to ping us here, we have had a bit of experience with this platform ourselves! :qiime2: :t_rex:


(Matthew Ryan Dillon) #4

(Adam Rivers) #5

I should have given more specifics. My underlying module, itsxpress, is already available on bioconda https://github.com/bioconda/bioconda-recipes/blob/master/recipes/itsxpress/meta.yaml . @cduvallet’s guide was helpful when I was originally creating conda recipe.

We’ve written a recipe for the qiime plugin q2_itxpress too but it does not pass Circleci build tests for Bioconda because we have to run an import test on the recipe it cannot import q2_types and qiime2.plugin since qiime2 is not installed. Normally I would just add qiime2 to my list of channels then add Qiime2 to my list of required packages in my meta.yaml file

I can designate additional channels easily on the command line. But in the bioconda build workflow I cannot do that. See the dependencies section of this page: https://bioconda.github.io/guidelines.html for an explanation.

That’s why I was asking if it makes more sense to use a different channel for a Qiime Plugin. If there is some way to add the anaconda cloud qiime2/label/r2018.6 channel to in the setup.sh script that would be best since I’m already using bioconda. Alternatively, maybe I could wget the recipe and install it locally with the build.sh script then install.

I was just trying to get a sense of how other people have distributed non-core plugins.


(Matthew Ryan Dillon) #6

(Matthew Ryan Dillon) #7

Oops! Sorry, I completely misunderstood your question.

Reading the dependencies section of the guidelines:

There is currently no mechanism to define, in the meta.yaml file, that a particular dependency should come from a particular channel.

I wonder if conda-build supports the MatchSpec syntax released in conda 4.4? That would potentially allow you to define a channel in the recipe, but I am probably missing something there.

Otherwise, yeah, setting up your own channel would allow you to get around this, for now.


(Matthew Ryan Dillon) #8

(Adam Rivers) #9

I found a pretty simple way to solve this in Bioconda. Rather than writing two bioconda recipes, one for the base package ITSxpress and one for the QIIME2 plugin q2-itsxpress, I wrote only one Bioconda recipe for ITSxpress that installs the plugin from pip but along with all the other dependencies for the package.

Then my bioconda tests check that itsxpress is importable but not the plugin which has QIIME2 as a dependency. This allows everything to be installed by typing conda install itsxpress. I run more extensive tests on both the core package and the plugin with TravisCI so this shouldn’t impact my CI testing.


(Nicholas Bokulich) #10

(Matthew Ryan Dillon) #12