Conda config overriding channel order - even when config file : None

Hey, in order for this release to work I had to add anaconda in the end of the channels section of the yml file

Could you give us some more details on your system, and any particular errors you were receiving? Thanks!

After wget ing the .yml file and running

conda env create -n qiime2-2018.6 --file qiime2-2018.6-py35-linux-conda.yml

I got this error:

ResolvePackageNotFound: 
 - intel-openmp 2018.0.3*

Since intel-openmp is available through the anaconda channel I added anaconda channel to the .yml file (right after the qiime/* channels). I guess it messed up the R installation since when I tried to run dada2 I got:

Traceback (most recent call last):
  File "/home/labs/elinav/mallyb/.conda/envs/qiime2-2018.6/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 229, in denoise_paired
    run_commands([cmd])
  File "/home/labs/elinav/mallyb/.conda/envs/qiime2-2018.6/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 36, in run_commands
    subprocess.run(cmd, check=True)
  File "/home/labs/elinav/mallyb/.conda/envs/qiime2-2018.6/lib/python3.5/subprocess.py", line 398, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['run_dada_paired.R', '/tmp/tmpjdz21amz/forward', '/tmp/tmpjdz21amz/reverse', '/tmp/tmpjdz21amz/output.tsv.biom', '/tmp/tmpjdz21amz/track.tsv', '/tmp/tmpjdz21amz/filt_f', '/tmp/tmpjdz21amz/filt_r', '250', '195', '0', '0', '2.0', '2', 'consensus', '1.0', '15', '1000000']' returned non-zero exit status 127

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/labs/elinav/mallyb/.conda/envs/qiime2-2018.6/lib/python3.5/site-packages/q2cli/commands.py", line 274, in __call__
    results = action(**arguments)
  File "<decorator-gen-380>", line 2, in denoise_paired
  File "/home/labs/elinav/mallyb/.conda/envs/qiime2-2018.6/lib/python3.5/site-packages/qiime2/sdk/action.py", line 232, in bound_callable
    output_types, provenance)
  File "/home/labs/elinav/mallyb/.conda/envs/qiime2-2018.6/lib/python3.5/site-packages/qiime2/sdk/action.py", line 367, in _callable_executor_
    output_views = self._callable(**view_args)
  File "/home/labs/elinav/mallyb/.conda/envs/qiime2-2018.6/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 244, in denoise_paired
    " and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code 127), please inspect stdout and stderr to learn more.

So, in order to prioritize dependencies, I moved the anaconda channel to the bottom of the channels section of the yml.

For now, it works!

1 Like

Hey there @uria — this is a strong indicator that your conda config is overriding the environment file’s settings.

Can you run the following and report back on what you find?

conda info

Thanks!

PS - this environment file has been tested dozens of times in the last few days on a few different systems and architectures — these test deployments did not need the addition of another channel.

2 Likes

Hey @thermokarst

There is no conda config file :face_with_raised_eyebrow:

As for what you asked:

There you go:

[[email protected] ~]$ conda info
Current conda install:

               platform : linux-64
               conda version : 4.3.24
               conda is private : False
               conda-env version : 4.3.24
               conda-build version : not installed
               python version : 3.6.1.final.0
               requests version : 2.14.2
               root environment : /apps/RH7U2/gnu/anaconda/4.4.0/python/3.6  (read only)
               default environment : /apps/RH7U2/gnu/anaconda/4.4.0/python/3.6
               envs directories : /home/labs/elinav/mallyb/.conda/envs
                                           /apps/RH7U2/gnu/anaconda/4.4.0/python/3.6/envs
               package cache : /apps/RH7U2/gnu/anaconda/4.4.0/python/3.6/pkgs
                                          /home/labs/elinav/mallyb/.conda/pkgs
               channel URLs : https://repo.continuum.io/pkgs/free/linux-64
                                         https://repo.continuum.io/pkgs/free/noarch
                                         https://repo.continuum.io/pkgs/r/linux-64
                                         https://repo.continuum.io/pkgs/r/noarch
                                         https://repo.continuum.io/pkgs/pro/linux-64
                                         https://repo.continuum.io/pkgs/pro/noarch
                config file : None
                netrc file : None
                offline mode : False
                user-agent : conda/4.3.24 requests/2.14.2 CPython/3.6.1 Linux/3.10.0-514.el7.x86_64 Red Hat Enterprise Linux Server/7.3 glibc/2.17
                UID:GID : #####:#####

The output of conda config --show is as follows:

[[email protected] ~]$ conda config --show
add_anaconda_token: True
add_pip_as_python_dependency: True
allow_non_channel_urls: True
allow_softlinks: True
always_copy: False
always_softlink: False
always_yes: False
anaconda_upload: None
auto_update_conda: True
changeps1: True
channel_alias: https://conda.anaconda.org
channel_priority: True
channels:
  - defaults
client_ssl_cert: None
client_ssl_cert_key: None
clobber: False
create_default_packages: []
custom_channels:
  pkgs/free: https://repo.continuum.io/
  pkgs/r: https://repo.continuum.io/
  pkgs/pro: https://repo.continuum.io/
custom_multichannels:
  defaults: ["https://repo.continuum.io/pkgs/free", "https://repo.continuum.io/pkgs/r", "https://repo.continuum.io/pkgs/pro"]
  local: []
default_channels:
  - https://repo.continuum.io/pkgs/free
  - https://repo.continuum.io/pkgs/r
  - https://repo.continuum.io/pkgs/pro
disallow: []
envs_dirs:
  - /home/labs/elinav/mallyb/.conda/envs
  - /apps/RH7U2/gnu/anaconda/4.4.0/python/3.6/envs
force: False
json: False
local_repodata_ttl: 1
migrated_channel_aliases: []
offline: False
path_conflict: clobber
pinned_packages: []
pkgs_dirs:
  - /apps/RH7U2/gnu/anaconda/4.4.0/python/3.6/pkgs
  - /home/labs/elinav/mallyb/.conda/pkgs
proxy_servers: {}
quiet: False
remote_connect_timeout_secs: 9.15
remote_max_retries: 3
remote_read_timeout_secs: 60.0
rollback_enabled: True
shortcuts: True
show_channel_urls: None
ssl_verify: True
track_features: []
use_pip: True
verbosity: 0

Sweet, thanks for the details @uria!

Looks like you are running an old version of conda (4.3.24) — the current version is 4.5.5, and, the config files that ship by default are quite different between the two versions. You are missing the main channels that ship by default now. The first section of the install guide mentions that you need to be running the latest version of conda in order to install these QIIME 2 environment files – check out the commands listed there to get set up!

Hope that helps! :qiime2: :t_rex:

@thermokarst
We use conda/anaconda to manage our python packages, and we can’t update global conda version each time a python package has a new release:

  • we work on hpcc so we don’t have full control over global installations and binaries
  • even if we had full control - it doesn’t seem a good idea since we are heavily dependent on conda to manage other stuff that we can’t harm.
  • The version we use was released a year ago, so it is far from being obsolete - even for the python community (Hey - people still write stuff in Python2.7!!)

So, if you can make that the installation works for all relevant conda releases - GREAT.
If not - I completely get it, I’ll send a pr when I’ll have some time to do it :slight_smile:

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.