installing plugins within qiime2 singularity

Technical Support
1

Hi,
I am looking for some help with installing plugins within a qiime2 singularity container.

I would like to add DEICODE and rescript, but although I was kindly directed to instructions for doing this in a docker container [Installing Community Plugins in Docker], the cluster I have access to only allows singularity.

I built a singularity file using "singularity pull docker://quay.io/qiime2/core:2023.2" (also previously 2021.11) - they work well. I did try to install deicode using "conda install -c conda-forge deicode" once I activated the qiime2 interactive mode with "singularity shell qiime2-2023.2.sif" but I ran into dependency and permissions issues. It failed in the end. A bit of a novice when it comes to working within containers.

Details below. Any help appreciated!

version qiime: qiime2-2023.2 (singularity)

plugin install commands tried:

  1. conda install -c conda-forge deicode

outcome: "Downloading and Extracting Packages
Preparing transaction: done
Verifying transaction: failed

              EnvironmentNotWritableError: The current user does not have write 
              permissions to the target environment"
  1. conda install -c conda-forge deicode --no-update-deps

outcome: "Solving environment: failed with initial frozen solve. Retrying with flexible solve.", then "Solving environment: | CPU time limit exceeded"

  1. conda install -c conda-forge deicode conda=23.1.0

outcome: same as in #1

  1. installing it within its own conda environment with "pip install deicode" returned:
    "Recursion error: maximum recursion depth exceeded while calling a Python object"
1 Like
2

Hi @anapopov!

I am not super familiar with using singularity (@colinbrislawn might have more insight on this) - but based on the error you're receiving it seems like a user permissions issue related to your conda configuration. Do you have access to update the conda config on this cluster?

3

This could also be related to the user inside the container image. I've not checked how Qiime2 does it, but often when the container disk image is being built, certain build steps take place with different user scopes. Also, image layers are immutable (by design)

This means singularity can make unwritable conda stuff. See:

While I'm here,

I appreciate your tenacity in working with containers! That's awesome!

Whoever made the 'singularity only' rule on this cluster could/should also help you use singularity to get this running. :whale: :rocket:

1 Like
4

Okay, thanks @colinbrislawn, and also @lizgehret !
I'll read up some more and try the cluster tech team.
:slight_smile:

1 Like
5

In case someone else is using qiime with singularity/apptainer, the instructions below were kindly provided by my cluster tech support for installing plugins:

Singularity (now is rebranded to apptainer) uses sif which is a
read-only image. To update an apptainer image requires a few steps.

  1. build a sandbox which is a directory ("sb") in which you can modify the content
    apptainer build --sandbox sb docker://quay.io/qiime2/core:2023.2

  2. ensure the directory (sb/opt) is writeable and/or change permissions

  3. activate this sandbox, as your container
    apptainer shell --contain --writable sb

  4. update the sandbox
    Apptainer> conda install -c conda-forge deicode

  5. pack the sandbox as a sif
    apptainer build my_updated.sif sb

Note you may need to remove/rename your $HOME/.cache directory in
order for the above conda to proceed.

2 Likes
6

This is helpful, thanks for following up with this info @anapopov! :slightly_smiling_face:

2 Likes
7

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.