How can I build a custom docker image from qiime2-amplicon distribution?

I'm trying to build a custom docker container with specific volume mount locations using the qiime2-amplicon-2024.2 distribution.

Here's my docker file:

FROM mambaorg/micromamba:1.5.6

SHELL ["/usr/local/bin/_dockerfile_shell.sh"]

WORKDIR /tmp/

# Data
USER root
RUN mkdir -p /volumes/
RUN mkdir -p /volumes/input
RUN mkdir -p /volumes/output
RUN mkdir -p /volumes/database

# Retrieve repository
USER $MAMBA_USER
RUN micromamba install -y -n base -c conda-forge wget
ARG MAMBA_DOCKERFILE_ACTIVATE=1
RUN wget https://data.qiime2.org/distro/amplicon/qiime2-amplicon-2024.2-py38-linux-conda.yml

# Install dependencies
RUN micromamba install -y -n base -f /tmp/qiime2-amplicon-2024.2-py38-linux-conda.yml && \
    micromamba clean -a -y -f

RUN rm -rf /tmp/qiime2-amplicon-2024.2-py38-linux-conda.yml

ENTRYPOINT ["/usr/local/bin/_entrypoint.sh"]

However, I'm getting errors like this:

(base) Joshs-MBP:Containers jolespin$ docker build -t jolespin/qiime2-amplicon:2024.2 -f Dockerfile_qiime2-amplicon-2024.2 .
[+] Building 7.3s (13/14)                                                                                                          docker:desktop-linux
 => [internal] load build definition from Dockerfile_qiime2-amplicon-2024.2                                                                        0.0s
 => => transferring dockerfile: 1.34kB                                                                                                             0.0s
 => [internal] load metadata for docker.io/mambaorg/micromamba:1.5.6                                                                               0.8s
 => [auth] mambaorg/micromamba:pull token for registry-1.docker.io                                                                                 0.0s
 => [internal] load .dockerignore                                                                                                                  0.0s
 => => transferring context: 2B                                                                                                                    0.0s
 => [ 1/10] FROM docker.io/mambaorg/micromamba:1.5.6@sha256:4d9619e648abe569fd72fa30011c22bab13dd8494a7745295f512303de048989                       0.0s
 => CACHED [ 2/10] WORKDIR /tmp/                                                                                                                   0.0s
 => CACHED [ 3/10] RUN mkdir -p /volumes/                                                                                                          0.0s
 => CACHED [ 4/10] RUN mkdir -p /volumes/input                                                                                                     0.0s
 => CACHED [ 5/10] RUN mkdir -p /volumes/output                                                                                                    0.0s
 => CACHED [ 6/10] RUN mkdir -p /volumes/database                                                                                                  0.0s
 => CACHED [ 7/10] RUN micromamba install -y -n base -c conda-forge wget                                                                           0.0s
 => CACHED [ 8/10] RUN wget https://data.qiime2.org/distro/amplicon/qiime2-amplicon-2024.2-py38-linux-conda.yml                                    0.0s
 => ERROR [ 9/10] RUN micromamba install -y -n base -c qiime2 -c conda-forge -c bioconda -f /tmp/qiime2-amplicon-2024.2-py38-linux-conda.yml &&    6.5s
------
 > [ 9/10] RUN micromamba install -y -n base -c qiime2 -c conda-forge -c bioconda -f /tmp/qiime2-amplicon-2024.2-py38-linux-conda.yml &&     micromamba clean -a -y -f:
0.184 conda-forge/linux-aarch64                                   Using cache
0.184 conda-forge/noarch                                          Using cache
6.484 error    libmamba Could not solve for environment specs
6.484     The following packages are incompatible
6.484     โ”œโ”€ alsa-lib 1.2.8**  is requested and can be installed;
6.484     โ”œโ”€ bioconductor-ancombc 2.0.1**  is not installable because it requires
6.484     โ”‚  โ””โ”€ bioconductor-s4vectors >=0.36.0,<0.37.0 , which does not exist (perhaps a missing channel);
6.484     โ”œโ”€ bioconductor-beachmat 2.14.0**  does not exist (perhaps a typo or a missing channel);
6.484     โ”œโ”€ bioconductor-biobase 2.58.0**  does not exist (perhaps a typo or a missing channel);

I've tried manually adding the channel to the condarc but it still didn't work:

#RUN echo "channel_priority: flexible" >> ~/.condarc
#RUN echo "channels:" >> ~/.condarc
#RUN echo "  - qiime2" >> ~/.condarc
#RUN echo "  - conda-forge" >> ~/.condarc
#RUN echo "  - bioconda" >> ~/.condarc

Any suggestions?

I'm seeing this Dockerfile here but miniconda is waaaay bigger than micromamba: vm-playbooks/docker/Dockerfile at master ยท qiime2/vm-playbooks ยท GitHub

Hello Josh,

conda-forge/linux-aarch64

aarch64 is supported by conda-forge, but not bioconda.

Are you trying to run this on ARM or should this be an x86 machine?

Iโ€™ve made some progress on this and I need to update this issue when I get home. I added the full url for conda channel like in the official dockerfile which helped. However, the last issue I ran into was getting past outdated certificates during the install command within the dockerfile. Iโ€™ll get exact details in a bit.

Getting this to work with micromamba should really reduce the file size by quite a bit for the container.

For context, I am building this an Apple silicon machine (m2 pro) using Rosetta in my terminal (emulating intel). I plan on using the image on AWS batch.

1 Like

Sup Josh,

Your timing is impeccable; it looks like bioconda just rolled out aarch64 support for Linux, though there's no osx-arm64 support yet.

If you are using AWS batch with x86, it probably makes more sense to build on that instead of trying to build on osx-arm64 and then ship the container...

(Our lead deployment engineer is out right now, so I appreciate your patience. Once she is back, we can get her advice about this topic.)

This did the trick for me:

# v2024.3.4
# =============================
FROM --platform=linux/amd64 mambaorg/micromamba:1.5.6

SHELL ["/usr/local/bin/_dockerfile_shell.sh"]

WORKDIR /tmp/

# Data
USER root
RUN mkdir -p /volumes/
RUN mkdir -p /volumes/input
RUN mkdir -p /volumes/output
RUN mkdir -p /volumes/database

ENV LC_ALL C.UTF-8
ENV LANG C.UTF-8
ENV MPLBACKEND agg
ENV XDG_CONFIG_HOME /home/qiime2


# Retrieve repository
USER $MAMBA_USER
RUN micromamba install -y -n base -c conda-forge wget ca-certificates

ARG MAMBA_DOCKERFILE_ACTIVATE=1
RUN wget --no-check-certificate https://data.qiime2.org/distro/amplicon/qiime2-amplicon-2024.2-py38-linux-conda.yml

# Install dependencies
RUN micromamba install -y -n base \
    -c https://packages.qiime2.org/qiime2/2024.2/amplicon/released \
    -c bioconda \
    -c conda-forge \
    -c defaults \
    -f /tmp/qiime2-amplicon-2024.2-py38-linux-conda.yml && \ 
    micromamba clean -a -y -f

RUN rm -rf /tmp/qiime2-amplicon-2024.2-py38-linux-conda.yml
# RUN qiime dev refresh-cache


ENTRYPOINT ["/usr/local/bin/_entrypoint.sh"]

However, I had to remove qiime dev refresh-cache because it was taking over a day to run just that step.

The micromamba installation is a little smaller but not as much as I thought it would be at 5.13GB compared to 5.92GB.

1 Like

Great!

Would you like me to leave this thread open to get comments from the Qiime devs, or close it for now?

I'm personally curious about the need for custom containers instead of the default ones on Quay, if you are willing to share more context.

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.