failed to do download functional database like egg in moshpit

Technical Support
1

I am using the latest version of docker images moshpit 2025.10, failed to download egg database by using mosh annotate fetch-diamond-db or mosh annotate fetch-eggnog-db, it seems the URL was unvailable, how to built databese locally?

mosh annotate fetch-diamond-db \
    --o-db cache:diamond_db \
    --verbose

error like this

Starting download...
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: wget -e robots=off -O cache/processes/1816-1763689293.85@root/tmp/q2-OutPath-egcebyp1/ref_db.dmnd.gz http://eggnog5.embl.de/download/emapperdb-5.0.2/eggnog_proteins.dmnd.gz

--2025-11-21 01:43:56--  http://eggnog5.embl.de/download/emapperdb-5.0.2/eggnog_proteins.dmnd.gz
Resolving eggnog5.embl.de (eggnog5.embl.de)... 194.94.44.170
Connecting to eggnog5.embl.de (eggnog5.embl.de)|194.94.44.170|:80... connected.
HTTP request sent, awaiting response... 500 reading HTTP response body: unexpected EOF
2025-11-21 01:56:25 ERROR 500: reading HTTP response body: unexpected EOF.

Traceback (most recent call last):
  File "/opt/conda/envs/qiime2-moshpit-2025.10/lib/python3.10/site-packages/q2cli/commands.py", line 567, in __call__
    results = self._execute_action(
  File "/opt/conda/envs/qiime2-moshpit-2025.10/lib/python3.10/site-packages/q2cli/commands.py", line 639, in _execute_action
    results = action(**arguments)
  File "<decorator-gen-575>", line 2, in fetch_diamond_db
  File "/opt/conda/envs/qiime2-moshpit-2025.10/lib/python3.10/site-packages/qiime2/sdk/action.py", line 265, in bound_callable
    outputs = self._callable_executor_(
  File "/opt/conda/envs/qiime2-moshpit-2025.10/lib/python3.10/site-packages/qiime2/sdk/action.py", line 434, in _callable_executor_
    output_views = self._callable(**view_args)
  File "/opt/conda/envs/qiime2-moshpit-2025.10/lib/python3.10/site-packages/q2_annotate/eggnog/dbs.py", line 127, in fetch_diamond_db
    run_command(
  File "/opt/conda/envs/qiime2-moshpit-2025.10/lib/python3.10/site-packages/q2_annotate/_utils.py", line 45, in run_command
    subprocess.run(cmd, check=True, **kwargs)
  File "/opt/conda/envs/qiime2-moshpit-2025.10/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['wget', '-e', 'robots=off', '-O', 'cache/processes/1816-1763689293.85@root/tmp/q2-OutPath-egcebyp1/ref_db.dmnd.gz', 'http://eggnog5.embl.de/download/emapperdb-5.0.2/eggnog_proteins.dmnd.gz']' returned non-zero exit status 8.

Plugin error from annotate:

  Command '['wget', '-e', 'robots=off', '-O', 'cache/processes/1816-1763689293.85@root/tmp/q2-OutPath-egcebyp1/ref_db.dmnd.gz', 'http://eggnog5.embl.de/download/emapperdb-5.0.2/eggnog_proteins.dmnd.gz']' returned non-zero exit status 8.

See above for debug info.

mosh annotate fetch-eggnog-db \
    --o-db cache:eggnog_db \
    --verbose

error like this

The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: download_eggnog_data.py -y -D --data_dir /tmp/qiime2/root/processes/2240-1763701912.51@root/tmp/q2-OutPath-503xg5n2

Downloading "eggnog.db" at /tmp/qiime2/root/processes/2240-1763701912.51@root/tmp/q2-OutPath-503xg5n2...
cd /tmp/qiime2/root/processes/2240-1763701912.51@root/tmp/q2-OutPath-503xg5n2 && wget -nH --user-agent=Mozilla/5.0 --relative --no-parent --reject "index.html*" --cut-dirs=4 -e robots=off -O eggnog.db.gz http://eggnogdb.embl.de/download/emapperdb-5.0.2/eggnog.db.gz && echo Decompressing... && gunzip eggnog.db.gz 
--2025-11-21 05:14:21--  http://eggnogdb.embl.de/download/emapperdb-5.0.2/eggnog.db.gz
Resolving eggnogdb.embl.de (eggnogdb.embl.de)... 194.94.44.170
Connecting to eggnogdb.embl.de (eggnogdb.embl.de)|194.94.44.170|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2025-11-21 05:14:27 ERROR 404: Not Found.

Downloading "eggnog.taxa.db" at /tmp/qiime2/root/processes/2240-1763701912.51@root/tmp/q2-OutPath-503xg5n2...
cd /tmp/qiime2/root/processes/2240-1763701912.51@root/tmp/q2-OutPath-503xg5n2 && wget -nH --user-agent=Mozilla/5.0 --relative --no-parent --reject "index.html*" --cut-dirs=4 -e robots=off -O eggnog.taxa.tar.gz http://eggnogdb.embl.de/download/emapperdb-5.0.2/eggnog.taxa.tar.gz && echo Decompressing... && tar -zxf eggnog.taxa.tar.gz && rm eggnog.taxa.tar.gz
--2025-11-21 05:14:27--  http://eggnogdb.embl.de/download/emapperdb-5.0.2/eggnog.taxa.tar.gz
Resolving eggnogdb.embl.de (eggnogdb.embl.de)... 194.94.44.170, 194.94.44.170
Connecting to eggnogdb.embl.de (eggnogdb.embl.de)|194.94.44.170|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2025-11-21 05:14:33 ERROR 404: Not Found.

Skipping diamond database (or already present). Use -f to force download
Skipping novel families diamond and annotation databases (or already present). Use -F and -f to force download
Skipping Pfam database (or already present). Use -P and -f to force download
Skipping MMseqs2 database (or already present). Use -M and -f to force download
No HMMER database requested. Use "-H -d taxid" to download the hmmer database for taxid
Finished.
Traceback (most recent call last):
  File "/opt/conda/envs/qiime2-moshpit-2025.10/lib/python3.10/site-packages/q2cli/commands.py", line 567, in __call__
    results = self._execute_action(
  File "/opt/conda/envs/qiime2-moshpit-2025.10/lib/python3.10/site-packages/q2cli/commands.py", line 639, in _execute_action
    results = action(**arguments)
  File "<decorator-gen-572>", line 2, in fetch_eggnog_db
  File "/opt/conda/envs/qiime2-moshpit-2025.10/lib/python3.10/site-packages/qiime2/sdk/action.py", line 265, in bound_callable
    outputs = self._callable_executor_(
  File "/opt/conda/envs/qiime2-moshpit-2025.10/lib/python3.10/site-packages/qiime2/sdk/action.py", line 451, in _callable_executor_
    self.signature.coerce_given_outputs(output_views, output_types,
  File "/opt/conda/envs/qiime2-moshpit-2025.10/lib/python3.10/site-packages/qiime2/core/type/signature.py", line 498, in coerce_given_outputs
    output = self._create_output_artifact(
  File "/opt/conda/envs/qiime2-moshpit-2025.10/lib/python3.10/site-packages/qiime2/core/type/signature.py", line 520, in _create_output_artifact
    artifact = qiime2.sdk.Artifact._from_view(
  File "/opt/conda/envs/qiime2-moshpit-2025.10/lib/python3.10/site-packages/qiime2/sdk/result.py", line 700, in _from_view
    result = transformation(view, validate_level)
  File "/opt/conda/envs/qiime2-moshpit-2025.10/lib/python3.10/site-packages/qiime2/core/transform.py", line 68, in transformation
    self.validate(view, level=validate_level)
  File "/opt/conda/envs/qiime2-moshpit-2025.10/lib/python3.10/site-packages/qiime2/core/transform.py", line 143, in validate
    view.validate(level)
  File "/opt/conda/envs/qiime2-moshpit-2025.10/lib/python3.10/site-packages/qiime2/plugin/model/directory_format.py", line 183, in validate
    raise ValidationError("Unrecognized file (%s) for %s."
qiime2.core.exceptions.ValidationError: Unrecognized file (/tmp/qiime2/root/processes/2240-1763701912.51@root/tmp/q2-OutPath-503xg5n2/eggnog.taxa.tar.gz) for EggnogRefDirFmt.

Plugin error from annotate:

  Unrecognized file (/tmp/qiime2/root/processes/2240-1763701912.51@root/tmp/q2-OutPath-503xg5n2/eggnog.taxa.tar.gz) for EggnogRefDirFmt.

See above for debug info.
2

Hi @chaokang2012 ,

Thanks for using moshpit and for reporting this issue!

Unfortunately this is an issue upstream in eggnog-mapper, see the issue report here. It appears that the eggnog databases were recently migrated to a different URL and the eggnog-mapper tool (which q2-annotate is using to grab the databases as well as for functional annotation) was not yet updated to use the correct URLs.

Fortunately, there is a workaround to download the databases directly from the new URLs and import as QIIME 2 artifacts (see the discussion in the forum topic linked below)

This issue was reported previously, see more discussion here:

I hope that helps!

2 Likes
3

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.