QIIME2 conda install in multi-user environment

[Lela_Andrews]

I’m administering a multi-user linux environment. I have virtual environments for qiime1.9.1 and qiime2-2019.4 that are accessible to all users. Someone requested a new 2019.7 installation. Using the administrative account, I followed the instructions on the installing qiime natively page, but it installed the environment to that user’s local directory instead of the shared environment directory. Miniconda was installed to a shared location as per another post on this topic. I tried to set the env directory in the .condarc file as described here but it doesn’t seem to work. I’m a little confused about how I even set this up back in May, which I’m sure I did following an online guide. There is a user called conda who owns all of the shared directories. Seems like there is probably an obvious solution to someone more familiar with conda than I am. I eventually tried installing qiime to the wrong place and simply moving the install directory to where it should be (see below output). Upon moving the directory, conda immediately found the environment in the right place, but when I activated it, the qiime script complained about a bad python interpreter. I changed the path in the shebang line to correct this and the environment opens, but now it can’t find the default config file. Will this install work? Are there additional steps I will need to take? Is there a better way to do this with my existing system configuration? I would prefer to have a simple way to install updated environments without going through a lot of mess in the process. I can’t believe I didn’t copy down what I was doing when I made the installation, because otherwise it works perfectly.

Conda install location:
ecoss@QIIME-MACHINE:~$ ls -l /usr/bin/conda/conda/
total 8
drwxr-xr-x 1 conda conda 1268 Sep 20 11:01 bin
drwxr-xr-x 1 conda conda 16 May 17 14:30 compiler_compat
drwxr-xr-x 1 conda conda 10 Sep 20 10:59 condabin
drwxr-xr-x 1 conda conda 2456 Sep 20 11:01 conda-meta
drwxr-xr-x 1 conda conda 86 May 17 15:38 envs
drwxr-xr-x 1 conda conda 38 May 17 15:16 etc
drwxr-xr-x 1 conda conda 1950 Sep 20 11:01 include
drwxr-xr-x 1 conda conda 4038 Sep 20 11:01 lib
drwxr-xr-x 1 conda ecoss 6 Sep 20 10:59 libexec
-rw-r-xr-x 1 conda conda 4134 Apr 17 14:29 LICENSE.txt
drwxr-xr-x 1 conda ecoss 8 Sep 20 10:59 man
drwxr-xr-x 1 conda conda 82804 Sep 20 11:11 pkgs
drwxr-xr-x 1 conda conda 120 Sep 20 10:59 share
drwxr-xr-x 1 conda conda 16 May 17 14:30 shell
drwxr-xr-x 1 conda conda 168 Sep 20 10:59 ssl
drwxr-xr-x 1 conda conda 14 May 17 14:30 x86_64-conda_cos6-linux-gnu

Existing environments:

ecoss@QIIME-MACHINE:~$ conda env list  
# conda environments:  
#  
base                  *  /usr/bin/conda/conda  
qiime1.9.1               /usr/bin/conda/conda/envs/qiime1.9.1  
qiime2-2019.4            /usr/bin/conda/conda/envs/qiime2-2019.4  

Contents of .condarc
ecoss@QIIME-MACHINE:~$ cat .condarc
auto_activate_base: false
#CONDA_ENVS_PATH=~/my-envs:/usr/bin/conda/conda/envs

Activation of new environment after moving the directory (q2 is an alias for activation):
ecoss@QIIME-MACHINE:~$ q2
-bash: /usr/bin/conda/conda/envs/qiime2-2019.7/bin/qiime: /home/ecoss/.conda/envs/qiime2-2019.7/bin/python: bad interpreter: No such file or directory

Non-admin user activating qiime2-2019.7 environment after updating shebang line in qiime script:
lva123@QIIME-MACHINE:~ q2 QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment. Fontconfig error: Cannot load default config file (qiime2-2019.7) lva123@QIIME-MACHINE:~ grep q2 .bashrc
alias q2=“conda activate qiime2-2019.7”

[Lela_Andrews]

Ran into an issue already. Tried to install picrust, but it appears to be hidden from the average user, accessible only to the administrator. I have removed this installation and updated my .condarc thusly to try to force installation into the shared directory where the other installs reside:

ecoss@QIIME-MACHINE:~$ cat .condarc  
auto_activate_base: false  
envs_dirs:  
  - /usr/bin/conda/conda/envs  

Installing from the home directory seemed to try to put the install back into the default directory. I also tried installing from the conda directory (/usr/bin/conda/conda) but this also didn’t cause the installation to go to the intended location.

[ebolyen (Evan Bolyen)]

Hey @Lela_Andrews,

Sorry for the very late response from me.

Here’s what I think is happening based on your description (some of this will just be restatement).

1. You have a conda user named conda, it owns the subdirectory of /usr/bin/conda
   (as an aside, I think it is more common to put conda in /opt/ on linux, you’ll find a few of us out there, but it really shouldn’t matter)

2. All users have read and execute privileges for this subdirectory, however they do not have write (which is the goal).

3. When you installed the new environment it chose the “user” install path rather than the “global” install path. Meaning the env was only accessible to conda (or whichever user ran the install).

4. When conda installs something, it has to know what the prefix is that it is installing to. This is because conda creates these “fake” dynamically, but actually statically, linked libraries by rewriting all of the static links on install. When you copied the environment to the new location, this did not patch all of the binaries to look in the new path, which is why your interpretter thinks it is corrupted (it is!).

---

To fix this, it is actually pretty simple, first make sure you delete the bad environment from wherever it may be, and then we are going to use an odd flag in conda:

-p /usr/bin/conda/conda/envs/qiime2-2019.7

This will force the prefix that conda installs into to be that filepath.
If the path is in the normal “search” directories, then it will be available for activation with the normal name (qiime2-2019.7).

Important: when using -p you’ll want to skip any of the other flags which set the name (-n for example). So the full q2-install would look something like this:

conda env create -p /usr/bin/conda/conda/envs/qiime2-2019.7 --file qiime2-2019.7-py36-linux-conda.yml

Where the -n is now a -p.

---

Let me know if that helps! The prefix flag is a little bit subtle and not highly advertised, but we use it a lot for our automated testing systems.

[Lela_Andrews]

@ebolyen Thanks! Install seemed to go smoothly (install as user “conda” in this case) and I was able to call the new environment with my test user account. I asked the user who requested this to give it a try and let me know how things go.