Redbiom: OSError: [Errno 51] Network is unreachable

Other Bioinformatics Tools
1

Hello everyone,

I have determined the differentially abundant features in my dataset using both ANCOM and Songbirds and now I’d like to proceed with redbiom to learn more about the characteristics of the discovered features.

I followed the instruction from [Querying for public microbiome data in Qiita using redbiom], but I encountered an issue with the command redbiom summarize contexts, it returned a bunch of error code with which I am not sure how to troubleshoot (please see below). Is this a transient issue with the Qiita server or something wrong in my end?

PS: I processed essentially all of my data in an HPC cluster using a qiime2 docker image, therefore all of the generated artifacts are stored in the cluster. To execute redbiom, I first downloaded the featuretable[frequency] artifact and the metadata file, then install Qiime2 locally and executed the commands within the qiime2 Conda environmental on my computer.

Thank you for your help!

Rui

(qiime2-2020.8) Rui-Mac:~ rui$ redbiom summarize contexts
Traceback (most recent call last):
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/urllib3/connection.py", line 160, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/urllib3/util/connection.py", line 84, in create_connection
raise err
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/urllib3/util/connection.py", line 74, in create_connection
sock.connect(sa)
OSError: [Errno 51] Network is unreachable

During handling of the above exception, another excepv tion occurred:

Traceback (most recent call last):
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen
chunked=chunked,
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/urllib3/connectionpool.py", line 392, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/http/client.py", line 1272, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/http/client.py", line 1318, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/http/client.py", line 1267, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/http/client.py", line 1038, in _send_output
self.send(msg)
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/http/client.py", line 976, in send
self.connect()
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/urllib3/connection.py", line 187, in connect
conn = self._new_conn()
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/urllib3/connection.py", line 172, in _new_conn
self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7febac8cdc88>: Failed to establish a new connection: [Errno 51] Network is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/urllib3/connectionpool.py", line 727, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/urllib3/util/retry.py", line 439, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='qiita.ucsd.edu', port=7329): Max retries exceeded with url: /HGETALL/state:contexts (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7febac8cdc88>: Failed to establish a new connection: [Errno 51] Network is unreachable',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/rui/anaconda3/envs/qiime2-2020.8/bin/redbiom", line 8, in
sys.exit(cli())
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/click/core.py", line 829, in call
return self.main(*args, **kwargs)
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/redbiom/commands/summarize.py", line 16, in summarize_caches
contexts = redbiom.summarize.contexts()
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/redbiom/summarize.py", line 29, in contexts
contexts = get('state', 'HGETALL', 'contexts')
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/redbiom/_requests.py", line 90, in f
return _parse_validate_request(s.get(url), cmd)
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/requests/sessions.py", line 543, in get
return self.request('GET', url, **kwargs)
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "/Users/rui/anaconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='qiita.ucsd.edu', port=7329): Max retries exceeded with url: /HGETALL/state:contexts (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7febac8cdc88>: Failed to establish a new connection: [Errno 51] Network is unreachable',))

2

Hi @Rui, I reclassified this post to “Other Bioinformatics Tools,” since this support request isn’t directly related to QIIME 2. Perhaps @wasade can comment on this, but we can’t make any support guarantees for these kinds of (non QIIME 2) questions.

1 Like
3

Hi @Rui,

Thank you for reaching out. I can confirm the issue, and am in communication with our systems team right now. I’ll follow up as I know more.

Best,
Daniel

1 Like
4

Hi @Rui,

We had a server issue, but it should be resolved now. When you have a second, could you retry the interaction with redbiom?

Best,
Daniel

1 Like
5

Hi @wasade! Thank you for your help, it works now!

But now a new issue emerges.... I am having a hard time picking the right context, so the sequencing technology I used was Illumina Miseq PE150 targeted the V4 region of the 16S rDNA, and the data were denoised with DADA2, which produced ASVs. The database I compared my samples against was SILVA 138 SSU. There are a few reasonable contexts that I can choose. I think the first one is probably the best one, Could you confirm? Much appreciated :pray:

Pick_closed-reference_OTUs-SILVA-Illumina-16S-V4-150nt-96e6a9 1306 41654 Pick closed-reference OTUs (reference-seq: |projects|qiita_data|reference|silva_119_Silva_119_rep_set97.fna) | Trimming (length: 150)

Pick_closed-reference_OTUs-SILVA-Illumina-16S-V4-54d83f 1585 22836 Pick closed-reference OTUs (reference-seq: |projects|qiita_data|reference|silva_119_Silva_119_rep_set97.fna) | Split libraries FASTQ

Pick_closed-reference_OTUs-SILVA-Illumina-16S-V4-d1cf1d 20 5361 Pick closed-reference OTUs (reference-seq: |projects|qiita_test_data|reference|Greengenes_13_8_gg_13_8-97_otus.fasta) | Split libraries FASTQ

6

Hi @Rui,

You might want to consider the Deblur 150nt V4 context as it has substantially more samples represented (> 100,000). The ASVs from Deblur and DADA2 are themselves database agnostic. Note that a 150nt context is composed of features which are all 150nt, and is not using reverse read. To ensure an overlap, I would only use the first 150nt of your data without a left trim (unless there are adapters).

Unfortunately, I just tried to pull up the summarized contexts to provide the exact name and am receiving a timeout message. That is very weird, as the system should be quite stable – I just contacted our system administrator again to check in.

Best,
Daniel

1 Like
7

Hi @Rui,

It looks like redbiom will be offline overnight. The server went down unexpectedly again, and it smells like there may be a hardware issue. We’ll be diagnosing it tomorrow morning. I apologize for the inconvenience.

Best,
Daniel

2 Likes
8

Thank you for your help! And no worries, I’ll wait til it’s fixed. :slightly_smiling_face:

Rui

1 Like
9

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.