# rescript get-ncbi-data

**URL:** https://forum.qiime2.org/t/rescript-get-ncbi-data/20788
**Category:** Library Support
**Tags:** feature-classifier
**Created:** [September 18, 2021, 12:05am UTC](https://forum.qiime2.org/t/rescript-get-ncbi-data/20788 "2021-09-18T00:05:09Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![laibinhuang](https://forum.qiime2.org/user_avatar/forum.qiime2.org/laibinhuang/32/7698_2.png) [@laibinhuang](https://forum.qiime2.org/u/laibinhuang)
#### Post date: [September 18, 2021, 12:05am UTC](https://forum.qiime2.org/t/rescript-get-ncbi-data/20788/1 "2021-09-18T00:05:09Z")

</div>

Dear qiime team,

I am trying using the code below to download functional gene sequence by accession number, but it can only do 1000 number, if more than that, the timeout error will appear, how can i solve this problem;  
Thank you very much.

```auto
qiime rescript get-ncbi-data \
   --m-accession-ids-file "/home/rodrigues-lab/Laibin_Project/data/urec_accnum.txt" \
   --o-sequences /home/rodrigues-lab/Laibin_Project/data/urec-refseqs.qza \
   --o-taxonomy /home/rodrigues-lab/Laibin_Project/data/urec-tax.qza 

```

```auto
RuntimeError: Maximum retries (10) exceeded for HTTP request. Persistent trouble downloading from NCBI. Last exception was
ReadTimeout: HTTPSConnectionPool(host='eutils.ncbi.nlm.nih.gov', port=443): Read timed out. (read timeout=10)

```

The detail:

```auto
Traceback (most recent call last):
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/q2cli/commands.py", line 329, in __call__
    results = action(**arguments)
  File "<decorator-gen-179>", line 2, in get_ncbi_data
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/qiime2/sdk/action.py", line 245, in bound_callable
    output_types, provenance)
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/qiime2/sdk/action.py", line 390, in _callable_executor_
    output_views = self._callable(**view_args)
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/rescript/ncbi.py", line 102, in get_ncbi_data
    accs, logging_level, n_jobs, request_lock, _entrez_delay)
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/rescript/ncbi.py", line 326, in get_nuc_for_accs
    True, entrez_delay)
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/rescript/ncbi.py", line 312, in _get_for_ids
    for chunk in range(0, len(ids), 5000))
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/joblib/parallel.py", line 1029, in __call__
    if self.dispatch_one_batch(iterator):
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/joblib/parallel.py", line 847, in dispatch_one_batch
    self._dispatch(tasks)
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/joblib/parallel.py", line 765, in _dispatch
    job = self._backend.apply_async(batch, callback=cb)
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/joblib/_parallel_backends.py", line 206, in apply_async
    result = ImmediateResult(func)
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/joblib/_parallel_backends.py", line 570, in __init__
    self.results = batch()
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/joblib/parallel.py", line 253, in __call__
    for func, args, kwargs in self.items]
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/joblib/parallel.py", line 253, in <listcomp>
    for func, args, kwargs in self.items]
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/rescript/ncbi.py", line 289, in _get_id_chunk
    params, ids_chunk, request_lock, logging_level, entrez_delay)
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/rescript/ncbi.py", line 201, in _epost
    return _robustify(request, logger, params)
  File "/SSD/shared/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/rescript/ncbi.py", line 167, in _robustify
    type(last_exception). __name__ + ': ' + str(last_exception))
RuntimeError: Maximum retries (10) exceeded for HTTP request. Persistent trouble downloading from NCBI. Last exception was
ReadTimeout: HTTPSConnectionPool(host='eutils.ncbi.nlm.nih.gov', port=443): Read timed out. (read timeout=10)

```

---

<div class="post-metadata">

### Author: ![SoilRotifer](https://forum.qiime2.org/user_avatar/forum.qiime2.org/soilrotifer/32/21071_2.png) [@SoilRotifer](https://forum.qiime2.org/u/SoilRotifer)
#### Post date: [September 21, 2021, 5:24pm UTC](https://forum.qiime2.org/t/rescript-get-ncbi-data/20788/2 "2021-09-21T17:24:59Z")

</div>

Hi @laibinhuang,

The help text from `qiime rescript get-ncbi-data --help` should help answer that for you.

If you search the forum you'll come across the following helpful threads:

> [@RESCRIPT error: Download did not finish. Reason unknown](https://forum.qiime2.org/t/rescript-error-download-did-not-finish-reason-unknown/18928/7):
>
> @avtober, I forgot to mention that you should be aware that the help text also has other helpful information, e.g.: Please be aware of the NCBI Disclaimer and Copyright notice ([Policies and Disclaimers - NCBI](https://www.ncbi.nlm.nih.gov/home/about/policies/)), particularly "run retrieval scripts on weekends or between 9 pm and 5 am Eastern Time weekdays for any series of more than 100 requests". As a rough guide, if you are downloading more than 125,000 sequences, only run this method at those times... Which could also impact your abil…

> [@Error while getting plenty of data from NCBI](https://forum.qiime2.org/t/error-while-getting-plenty-of-data-from-ncbi/17635/3):
>
> It looks like you figured this out as per: Your next error: can be problematic to figure out. There are many "out of our hands" reasons that fetching data from NCBI can fail: I'd suggest trying to add more selective terms to your query, or try to download in batches. This is a great place to start: Good luck!

-Cheers!  
-Mike

---

<div class="post-metadata">

### Author: ![system](https://forum-qiime2-org.s3.dualstack.us-west-2.amazonaws.com/original/3X/2/1/21af5fe23cb6f4579467c66a9ed94e55274ca7bd.svg) [@system](https://forum.qiime2.org/u/system)
#### Post date: [October 22, 2021, 11:25pm UTC](https://forum.qiime2.org/t/rescript-get-ncbi-data/20788/3 "2021-10-22T23:25:30Z")

</div>

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.