Issue with cmake when making EPA-NG conda recipe

plugin-development
conda

(Gavin Douglas) #1

Hi there,

I’m working on a plugin for PICRUSt2 (https://github.com/picrust/picrust2 - you can see rough details on the plugin here: https://github.com/picrust/picrust2/wiki/q2-picrust2-Tutorial).

I’d like to make the pre-requisites for PICRUSt2 available on conda to make the installation more straight-forward. EPA-NG (https://github.com/Pbdas/epa-ng) is one of the pre-requisities that I’m having trouble writing a conda recipe for. This tool has a number of build requirements including cmake and autotools-dev (which isn’t available through conda and may be related to this problem).

I’m getting this error in the circle CI report:

Running cmake
CMake Warning (dev) at CMakeLists.txt:119:
Syntax Warning in cmake code at column 71

Argument not separated from preceding token by whitespace.
This warning is for project developers. Use -Wno-dev to suppress it.

CMake Error at /opt/conda/conda-bld/epa-ng_1539368794807/_build_env/share/cmake-3.12/Modules/CMakeDetermineCXXCompiler.cmake:47 (message):
Could not find compiler set in environment variable CXX:

g++.

Call Stack (most recent call first):
CMakeLists.txt:23 (project)

CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage

This is what I have in the meta.yaml:

package:
name: epa-ng
version: “0.2.1_beta_dev”

build:
number: 1

source:
url: https://github.com/gavinmdouglas/epa-ng/archive/0.2.1-beta-dev.tar.gz
md5: 15e7ad94596afc5cb72d90dfc0fad3eb

requirements:
build:
- {{ compiler(‘c’) }}
- libtool
- flex
- bison
- cmake
- automake
- autoconf
run:

test:
commands:
- epa-ng -h > /dev/null

about:
home: https://github.com/Pbdas/epa-ng
license: GNU Affero General Public License v3.0
summary: Massively parallel phylogenetic placement of genetic sequences

extra:
identifiers:
- doi:10.1093/sysbio/syy054

And these are the commands in build.sh:

#!/bin/bash

make

mkdir -p $PREFIX/bin

cp bin/epa-ng $PREFIX/bin

Thanks in advance!


(Evan Bolyen) #2

Hey @gmdouglas!

Are you building this locally or in the context of something like conda-forge or bioconda (I’ll assume locally).

I think if you change the compiler:

To 'cxx' instead we’ll get some slightly different errors, as it looks like EPA-NG uses CMake and C++. I’m not sure how CMake factors into this (if at all), but it’s probably not surprising that conda isn’t setting a env-var for a C++ compiler.

We can also use conda render which will template out everything in the recipe to what it is going to use. This should show us the recipe conda is actually using, which should be helpful information.


(Gavin Douglas) #3

Hey @ebolyen, thanks for getting back to me!

I’m building this locally in the bioconda github repo using their docker environment (https://bioconda.github.io/contribute-a-recipe.html#circle-ci-client-method).

When using ‘cxx’ I get the same error actually:

CMake Error at /opt/conda/conda-bld/epa-ng_1539608768649/_build_env/share/cmake-3.12/Modules/CMakeDetermineCXXCompiler.cmake:47 (message):
Could not find compiler set in environment variable CXX:

g++.

Call Stack (most recent call first):
CMakeLists.txt:23 (project)

CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage

I wasn’t familiar with conda render, thanks for suggesting it! Here’s the output:

No handlers could be found for logger "libarchive"
--------------
Hash contents:
--------------
{'cxx_compiler': 'gxx'}
----------
meta.yaml:
----------
package:
    name: epa-ng
    version: 0.2.1_beta_dev
source:
    md5: 15e7ad94596afc5cb72d90dfc0fad3eb
    url: https://github.com/gavinmdouglas/epa-ng/archive/0.2.1-beta-dev.tar.gz
build:
    number: '1'
requirements:
    build:
        - krb5 1.16.1 hbb41f41_0
        - rhash 1.3.6 h470a237_1
        - xz 5.2.4 h470a237_1
        - binutils_impl_linux-64 2.31.1 h6176602_1
        - libstdcxx-ng 8.2.0 hdf63c60_1
        - curl 7.61.1 h74213dd_2
        - perl 5.26.2 h470a237_0
        - binutils_linux-64 2.31.1 h6176602_3
        - bzip2 1.0.6 h470a237_2
        - cmake 3.12.3 h011004d_0
        - libcurl 7.61.1 hbdb9355_2
        - automake 1.15.1 pl526_1001
        - m4 1.4.18 h470a237_1
        - tk 8.6.8 ha92aebf_0
        - gxx_impl_linux-64 7.3.0 hdf63c60_1
        - flex 2.6.4 hfc679d8_3
        - expat 2.2.5 hfc679d8_2
        - ncurses 6.1 hfc679d8_1
        - libssh2 1.8.0 h5b517e9_2
        - gcc_linux-64 7.3.0 h553295d_3
        - ca-certificates 2018.8.24 ha4d7672_0
        - libgcc-ng 8.2.0 hdf63c60_1
        - libtool 2.4.6 h470a237_2
        - gcc_impl_linux-64 7.3.0 habb00fd_1
        - libedit 3.1.20170329 haf1bffa_1
        - gxx_linux-64 7.3.0 h553295d_3
        - zlib 1.2.11 h470a237_3
        - openssl 1.0.2p h470a237_1
        - bison 3.0.5 hfc679d8_1
        - autoconf 2.69 pl526_7
        - libuv 1.23.2 h470a237_0
    host:
        - libstdcxx-ng 8.2.0 hdf63c60_1
        - libgcc-ng 8.2.0 hdf63c60_1
    run:
        - libstdcxx-ng >=7.3.0
        - libgcc-ng >=7.3.0
test:
    commands:
        - epa-ng -h > /dev/null
about:
    home: https://github.com/Pbdas/epa-ng
    license: GNU Affero General Public License v3.0
    summary: Massively parallel phylogenetic placement of genetic sequences
extra:
    copy_test_source_files: true
    final: true
    identifiers:
        - doi:10.1093/sysbio/syy054

Thanks for your help!


(Nicholas Bokulich) #4

(Evan Bolyen) #5

Hey @gmdouglas,

Sorry for the delayed response.

Since you are using bioconda, that makes things a bit harder, we’ve noticed their compiler toolchain doesn’t quite work like we expect, so it’s something we’re trying to learn as well.

I’ll spend a few hours reading up on it and see what I can do.

Do you have a repo with this recipe inside (or a pull request to bioconda)? I couldn’t find anything specifically.


(Gavin Douglas) #6

If you have the time that would be great @ebolyen, thanks!

I do have a pull-request here: https://github.com/bioconda/bioconda-recipes/pull/10902. I have mainly been testing it locally, but that was my first attempt at pushing the tool as a test. Two major problems could be that (1) EPA-NG hasn’t been released for Mac yet and (2) the requirement autotools-dev isn’t available through conda. I don’t think these issues explain this particular error message though!

Thanks again,

Gavin


(Evan Bolyen) #7

What a rabbit hole I fell into!

I was able to reproduce the problem on Circle CI. Basically, we need gcc7 during the build step because epa-ng uses C++14. conda-forge is in the middle of a complete overhaul to make this possible:

However I don’t know how to enable similar compilers on bioconda yet:


(Evan Bolyen) #8

(Evan Bolyen) #9

I guess the best answer I have is stay tuned? Either this is possible with some kind of machinery that we don’t know about just yet, or it will soon be possible (ideally by default).

Until then, I don’t think there’s anything we can do about this particular recipe.


(Evan Bolyen) #10

Great news! I was able to get the new compilers, and after some tweaking (and trying to figure out what exit code 141 means to bison) I was able to get a local build working in docker!

Here’s a PR against your branch with the changes:


(Gavin Douglas) #11

Fantastic - thanks so much for figuring this out! I’ll test this out tomorrow! :smiley: