Dada2 "No such file or directory" and "Error in open.connection" message

Thanks for the offer! I think I'm going to try and reproduce with a smaller dataset (since it looks like a hardware problem). I haven't had a chance to really get started on this yet, but I'll let you know how it goes.

Hi,
I cannot solve that error in my iMAC so I changed to Virtual Machine in windows. Just got the following error:

qiime2@qiime2core2017-4:~/ShareWindows/vm$ qiime dada2 denoise-paired --i-demultiplexed-seqs demux-paired-end.qza --p-trim-left-f 0 --p-trunc-len-f 270 --p-trim-left-r 0 --p-trunc-len-r 215 --o-table bing-dada2-table.qza --o-representative-sequences bing-data2-rep-seqs.qza --verbose

Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_paired.R /tmp/tmpftxl8adr/forward /tmp/tmpftxl8adr/reverse /tmp/tmpftxl8adr/output.tsv.biom /tmp/tmpftxl8adr/filt_f /tmp/tmpftxl8adr/filt_r 270 215 0 0 2.0 2 pooled 1.0 1 1000000

R version 3.3.1 (2016-06-21) 
Loading required package: Rcpp
Warning messages:
1: multiple methods tables found for ‘arbind’ 
2: multiple methods tables found for ‘acbind’ 
3: replacing previous import ‘IRanges::arbind’ by ‘SummarizedExperiment::arbind’ when loading ‘GenomicAlignments’ 
4: replacing previous import ‘IRanges::acbind’ by ‘SummarizedExperiment::acbind’ when loading ‘GenomicAlignments’ 
5: multiple methods tables found for ‘left’ 
6: multiple methods tables found for ‘right’ 
DADA2 R package version: 1.1.7 
1) Filtering ..............Traceback (most recent call last):
  File "/miniconda3/lib/python3.5/site-packages/q2cli/commands.py", line 218, in __call__
    results = action(**arguments)
  File "<decorator-gen-225>", line 2, in denoise_paired
  File "/miniconda3/lib/python3.5/site-packages/qiime2/sdk/action.py", line 171, in callable_wrapper
    output_types, provenance)
  File "/miniconda3/lib/python3.5/site-packages/qiime2/sdk/action.py", line 248, in _callable_executor_
    output_views = callable(**view_args)
  File "/miniconda3/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 163, in denoise_paired
    run_commands([cmd])
  File "/miniconda3/lib/python3.5/site-packages/q2_dada2/_plot.py", line 26, in run_commands
    subprocess.run(cmd, check=True)
  File "/miniconda3/lib/python3.5/subprocess.py", line 398, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['run_dada_paired.R', '/tmp/tmpftxl8adr/forward', '/tmp/tmpftxl8adr/reverse', '/tmp/tmpftxl8adr/output.tsv.biom', '/tmp/tmpftxl8adr/filt_f', '/tmp/tmpftxl8adr/filt_r', '270', '215', '0', '0', '2.0', '2', 'pooled', '1.0', '1', '1000000']' returned non-zero exit status -9

Plugin error from dada2:

  Command '['run_dada_paired.R', '/tmp/tmpftxl8adr/forward',
  '/tmp/tmpftxl8adr/reverse', '/tmp/tmpftxl8adr/output.tsv.biom',
  '/tmp/tmpftxl8adr/filt_f', '/tmp/tmpftxl8adr/filt_r', '270', '215',
  '0', '0', '2.0', '2', 'pooled', '1.0', '1', '1000000']' returned non-
  zero exit status -9

See above for debug info.

Please help me and thanks!

I can’t say I’ve seen -9 as an exit code before, but I wonder if it’s not a response from SIGKILL. There’s not much else in that output, so I can’t really tell what, if anything, went wrong there. Out of curiosity, is ~/ShareWindows a mounted directory from your host OS? Does anything change when you work out of $HOME (.qza's and all)?

Related-ly, I went back through this entire thread and I completely missed how you had re-run on your iMac’s internal drive. So I don’t think it has anything to do with multiple devices (which is the bug we’re sorting out atm). I’m kind of back to thinking this is a memory situation (for the iMac).

1 Like

Also, I’ve been unable to reproduce this issue with a USB-drive locally :frowning2:, so that’s also part of my pivot back towards this being memory-related.

@ebolyen [quote="ebolyen, post:43, topic:412"]
Out of curiosity, is ~/ShareWindows a mounted directory from your host OS? Does anything change when you work out of $HOME (.qza's and all)?
[/quote]

Yes, it is a mounted directory from the drive in the windows desktop. The problem keeps happening in my iMac so I decide to give it a try in the windows' desktop!!! I have not tried the $home yet!

I feel frustrated about this problem. It got successful once and then suddenly not working any more!! Hopefully the memory extension will solve the problem~!!!

When you run a dataset, what is the minimum memory required for the 10GB dataset?

Thanks,
Bing

That's pretty much how I feel too. These are some very strange issues we've seen in this thread. (Even still, this is probably my favorite thread on the forum so far, thanks for all of the help!)

I'm not sure anyone actually knows a concrete answer for that.

2 Likes

@ebolyen

Thank you for the patient and continuous support. I finally noticed that memory is an issue in this case. I have shifted all my analysis to AWS and it works fast and no error at all!!

Thanks,
Bing

5 Likes

Awesome, thanks for letting us know what the solution was, and for all your patience with debugging! This forum topic will be really useful to others that might have the issue in the future.

1 Like

Gentlemen,

I am using the latest Qiime version 2020.2 and I have been getting the following error after the denoising step:

===============
Plugin error from dada2:

An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

Debug info has been saved to /var/folders/0_/kqg94rwj0hn532c1t4jlgtlh1__xx4/T/qiime2-q2cli-err-iua5kczp.log

I gave the following command

===================
qiime dada2 denoise-paired \

--i-demultiplexed-seqs demux.qza
--p-trim-left-f 5
--p-trim-left-r 5
--p-trunc-len-f 240
--p-trunc-len-r 240
--o-table table.qza
--o-representative-sequences rep-seqs.qza
--o-denoising-stats denoising-stats.qza
======================

I have used the following code to open the log file mentioned above "cat /var/folders/0_/kqg94rwj0hn532c1t4jlgtlh1__xx4/T/qiime2-q2cli-err-iua5kczp.log"

And I got the following log file

=======================

Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_paired.R /var/folders/0_/kqg94rwj0hn532c1t4jlgtlh1__xx4/T/tmpmryfu4tc/forward /var/folders/0_/kqg94rwj0hn532c1t4jlgtlh1__xx4/T/tmpmryfu4tc/reverse /var/folders/0_/kqg94rwj0hn532c1t4jlgtlh1__xx4/T/tmpmryfu4tc/output.tsv.biom /var/folders/0_/kqg94rwj0hn532c1t4jlgtlh1__xx4/T/tmpmryfu4tc/track.tsv /var/folders/0_/kqg94rwj0hn532c1t4jlgtlh1__xx4/T/tmpmryfu4tc/filt_f /var/folders/0_/kqg94rwj0hn532c1t4jlgtlh1__xx4/T/tmpmryfu4tc/filt_r 240 240 5 5 2.0 2.0 2 consensus 1.0 1 1000000

R version 3.5.1 (2018-07-02)
Loading required package: Rcpp
DADA2: 1.10.0 / Rcpp: 1.0.3 / RcppParallel: 4.4.4

  1. Filtering ................................................................................................................................................
  2. Learning Error Rates
    315475515 total bases in 1342449 reads from 3 samples will be used for learning the error rates.
    315475515 total bases in 1342449 reads from 3 samples will be used for learning the error rates.
  3. Denoise remaining samples .................................................................Error in open.connection(con, "rb") : cannot open the connection
    Calls: derepFastq ... FastqStreamer -> FastqStreamer -> open -> open.connection
    In addition: Warning message:
    In open.connection(con, "rb") :
    cannot open file '/var/folders/0_/kqg94rwj0hn532c1t4jlgtlh1__xx4/T/tmpmryfu4tc/filt_f/28_11_L001_R1_001.fastq.gz': No such file or directory
    Execution halted
    Traceback (most recent call last):
    File "/Users/elolimyahmed/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2_dada2/denoise.py", line 257, in denoise_paired
    run_commands([cmd])
    File "/Users/elolimyahmed/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2_dada2/denoise.py", line 36, in run_commands
    subprocess.run(cmd, check=True)
    File "/Users/elolimyahmed/miniconda3/envs/qiime2-2020.2/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
    subprocess.CalledProcessError: Command '['run_dada_paired.R', '/var/folders/0
    /kqg94rwj0hn532c1t4jlgtlh1__xx4/T/tmpmryfu4tc/forward', '/var/folders/0
    /kqg94rwj0hn532c1t4jlgtlh1__xx4/T/tmpmryfu4tc/reverse', '/var/folders/0_/kqg94rwj0hn532c1t4jlgtlh1__xx4/T/tmpmryfu4tc/output.tsv.biom', '/var/folders/0_/kqg94rwj0hn532c1t4jlgtlh1__xx4/T/tmpmryfu4tc/track.tsv', '/var/folders/0_/kqg94rwj0hn532c1t4jlgtlh1__xx4/T/tmpmryfu4tc/filt_f', '/var/folders/0_/kqg94rwj0hn532c1t4jlgtlh1__xx4/T/tmpmryfu4tc/filt_r', '240', '240', '5', '5', '2.0', '2.0', '2', 'consensus', '1.0', '1', '1000000']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/elolimyahmed/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2cli/commands.py", line 328, in call
results = action(**arguments)
File "</Users/elolimyahmed/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/decorator.py:decorator-gen-455>", line 2, in denoise_paired
File "/Users/elolimyahmed/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/qiime2/sdk/action.py", line 245, in bound_callable
output_types, provenance)
File "/Users/elolimyahmed/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/qiime2/sdk/action.py", line 390, in callable_executor
output_views = self._callable(**view_args)
File "/Users/elolimyahmed/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2_dada2/_denoise.py", line 272, in denoise_paired
" and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

ANY HELP IS APPRECIATED :slight_smile:

Welcome to the forum, @elolimy!

While I do happen to be a gentleman, it's worth noting that the :qiime2: community takes pride in its diversity! :grinning: :woman_scientist: :man_scientist: :rainbow:

This is a bit of an odd error which has, fortunately for us, come up on the forum before. Please review the existing discussions (links below). I suspect you will find everything you need there. But if not, don't hesitate to open a new topic with details about how your problem differs from those that have already been solved.

Firstly, the topic I have merged your post into: Dada2 "No such file or directory" and "Error in open.connection" message - #47 by Bing

And for (much) more context:

Thanks for your question, @elolimy!

Best,
Andrew

1 Like

This topic was automatically closed after 19 days. New replies are no longer allowed.