rescript get-ncbi-data

Community Plugin Support
1

Dear qiime team,

I am trying using the code below to download functional gene sequence by accession number, but it can only do 1000 number, if more than that, the timeout error will appear, how can i solve this problem;
Thank you very much.

qiime rescript get-ncbi-data \
   --m-accession-ids-file "/home/rodrigues-lab/Laibin_Project/data/urec_accnum.txt" \
   --o-sequences /home/rodrigues-lab/Laibin_Project/data/urec-refseqs.qza \
   --o-taxonomy /home/rodrigues-lab/Laibin_Project/data/urec-tax.qza 

RuntimeError: Maximum retries (10) exceeded for HTTP request. Persistent trouble downloading from NCBI. Last exception was
ReadTimeout: HTTPSConnectionPool(host='eutils.ncbi.nlm.nih.gov', port=443): Read timed out. (read timeout=10)

The detail:

Traceback (most recent call last):
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/q2cli/commands.py", line 329, in __call__
    results = action(**arguments)
  File "<decorator-gen-179>", line 2, in get_ncbi_data
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/qiime2/sdk/action.py", line 245, in bound_callable
    output_types, provenance)
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/qiime2/sdk/action.py", line 390, in _callable_executor_
    output_views = self._callable(**view_args)
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/rescript/ncbi.py", line 102, in get_ncbi_data
    accs, logging_level, n_jobs, request_lock, _entrez_delay)
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/rescript/ncbi.py", line 326, in get_nuc_for_accs
    True, entrez_delay)
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/rescript/ncbi.py", line 312, in _get_for_ids
    for chunk in range(0, len(ids), 5000))
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/joblib/parallel.py", line 1029, in __call__
    if self.dispatch_one_batch(iterator):
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/joblib/parallel.py", line 847, in dispatch_one_batch
    self._dispatch(tasks)
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/joblib/parallel.py", line 765, in _dispatch
    job = self._backend.apply_async(batch, callback=cb)
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/joblib/_parallel_backends.py", line 206, in apply_async
    result = ImmediateResult(func)
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/joblib/_parallel_backends.py", line 570, in __init__
    self.results = batch()
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/joblib/parallel.py", line 253, in __call__
    for func, args, kwargs in self.items]
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/joblib/parallel.py", line 253, in <listcomp>
    for func, args, kwargs in self.items]
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/rescript/ncbi.py", line 289, in _get_id_chunk
    params, ids_chunk, request_lock, logging_level, entrez_delay)
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/rescript/ncbi.py", line 201, in _epost
    return _robustify(request, logger, params)
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/rescript/ncbi.py", line 167, in _robustify
    type(last_exception).__name__ + ': ' + str(last_exception))
RuntimeError: Maximum retries (10) exceeded for HTTP request. Persistent trouble downloading from NCBI. Last exception was
ReadTimeout: HTTPSConnectionPool(host='eutils.ncbi.nlm.nih.gov', port=443): Read timed out. (read timeout=10)
2

Hi @laibinhuang,

The help text from qiime rescript get-ncbi-data --help should help answer that for you.

If you search the forum you'll come across the following helpful threads:

-Cheers!
-Mike

3

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.