Fail to built kraken2_db in moshipit

fail to built kraken2_db by moshipit

I use the Docker image from Quay, When building the Kraken2 and Bracken databases, it always fails to download the files :broken_heart: . This command line seems to not support resumable downloads.

mosh annotate build-kraken-db \
    --p-collection pluspf \
    --p-use-ftp \
    --p-threads 60 \
    --o-kraken2-db ./cache:kraken2_db \
    --o-bracken-db ./cache:bracken_db \
    --use-cache /data/mydata/myProjects/meta/mymoshpit/cache \
    --verbose

output like this

Found the latest "pluspf" database: kraken/k2_pluspf_20251015.tar.gz.
Downloading the "kraken/k2_pluspf_20251015.tar.gz" database:  72%|▋| 56.0G/77.5GTraceback (most recent call last):
  File "/opt/conda/envs/qiime2-moshpit-2025.7/lib/python3.10/site-packages/urllib3/response.py", line 779, in _error_catcher
    yield
  File "/opt/conda/envs/qiime2-moshpit-2025.7/lib/python3.10/site-packages/urllib3/response.py", line 925, in _raw_read
    raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
urllib3.exceptions.IncompleteRead: IncompleteRead(60099215657 bytes read, 23110581710 more expected)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/envs/qiime2-moshpit-2025.7/lib/python3.10/site-packages/requests/models.py", line 820, in generate
    yield from self.raw.stream(chunk_size, decode_content=True)
  File "/opt/conda/envs/qiime2-moshpit-2025.7/lib/python3.10/site-packages/urllib3/response.py", line 1091, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/opt/conda/envs/qiime2-moshpit-2025.7/lib/python3.10/site-packages/urllib3/response.py", line 1008, in read
    data = self._raw_read(amt)
  File "/opt/conda/envs/qiime2-moshpit-2025.7/lib/python3.10/site-packages/urllib3/response.py", line 903, in _raw_read
    with self._error_catcher():
  File "/opt/conda/envs/qiime2-moshpit-2025.7/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/opt/conda/envs/qiime2-moshpit-2025.7/lib/python3.10/site-packages/urllib3/response.py", line 803, in _error_catcher
    raise ProtocolError(arg, e) from e
urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(60099215657 bytes read, 23110581710 more expected)', IncompleteRead(60099215657 bytes read, 23110581710 more expected))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/envs/qiime2-moshpit-2025.7/lib/python3.10/site-packages/q2cli/commands.py", line 529, in __call__
    results = self._execute_action(
  File "/opt/conda/envs/qiime2-moshpit-2025.7/lib/python3.10/site-packages/q2cli/commands.py", line 601, in _execute_action
    results = action(**arguments)
  File "<decorator-gen-665>", line 2, in build_kraken_db
  File "/opt/conda/envs/qiime2-moshpit-2025.7/lib/python3.10/site-packages/qiime2/sdk/action.py", line 221, in bound_callable
    outputs = self._callable_executor_(
  File "/opt/conda/envs/qiime2-moshpit-2025.7/lib/python3.10/site-packages/qiime2/sdk/action.py", line 362, in _callable_executor_
    output_views = self._callable(**view_args)
  File "/opt/conda/envs/qiime2-moshpit-2025.7/lib/python3.10/site-packages/q2_annotate/kraken2/database.py", line 331, in build_kraken_db
    _fetch_prebuilt_dbs(bracken_db, kraken2_db, collection, tmp)
  File "/opt/conda/envs/qiime2-moshpit-2025.7/lib/python3.10/site-packages/q2_annotate/kraken2/database.py", line 270, in _fetch_prebuilt_dbs
    _fetch_db_collection(collection=collection, tmp_dir=tmp)
  File "/opt/conda/envs/qiime2-moshpit-2025.7/lib/python3.10/site-packages/q2_annotate/kraken2/database.py", line 243, in _fetch_db_collection
    for chunk in response.iter_content(chunk_size=CHUNK_SIZE):
  File "/opt/conda/envs/qiime2-moshpit-2025.7/lib/python3.10/site-packages/requests/models.py", line 822, in generate
    raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(60099215657 bytes read, 23110581710 more expected)', IncompleteRead(60099215657 bytes read, 23110581710 more expected))

Plugin error from annotate:

  ('Connection broken: IncompleteRead(60099215657 bytes read, 23110581710 more expected)', IncompleteRead(60099215657 bytes read, 23110581710 more expected))

See above for debug info.
Downloading the "kraken/k2_pluspf_20251015.tar.gz" database:  72%|▋| 56.0G/77.5G

How to solve this problem?
How can I use the already downloaded database to locally build the cache: kraken2_db and bracken_db?

1 Like

Hi @chaokang2012,
Sorry you're having trouble with the download! See this forum post for some instruction on how to do the import of already downloaded files.

1 Like

Really helpful, it works @gregcaporaso

Many thanks!!!

2 Likes