DADA2 errors - return code 1 and 11

Hello,

I have been having a couple of issues while running DADA2, and would appreciate your input.

So far I have received 2 different errors. The first time around I was using qiime2-2017.11 and got the following error:

Plugin error from dada2:

An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

Debug info has been saved to /var/folders/sn/92jxwrcd3qn6xp_y7c5rfdkc0000gn/T/qiime2-q2cli-err-bp_rhda1.log

For some reason I was not able to find the log file above, so I went through the following steps:

Actions:

  • upgrade qiime2
  • run tutorial data to ensure DADA2 works
  • run commmand with verbose

(qiime2-2017.12) dhcp-morris-2820:Rumen ErikaGanda$ qiime dada2 denoise-single \

--i-demultiplexed-seqs rumen-demux.qza
--p-trim-left 0
--p-trunc-len 270
--o-representative-sequences rumen-rep-seqs-dada2.qza
--o-table rumen-table-dada2.qza
--verbose

Plugin error from dada2:

An error was encountered while running DADA2 in R (return code -11), please inspect stdout and stderr to learn more.

Debug info has been saved to /var/folders/sn/92jxwrcd3qn6xp_y7c5rfdkc0000gn/T/qiime2-q2cli-err-ba9jr96p.log

This time I could get the log file, which is below:

dhcp-morris-2820:T ErikaGanda$ cat qiime2-q2cli-err-ba9jr96p.log
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_single.R /var/folders/sn/92jxwrcd3qn6xp_y7c5rfdkc0000gn/T/qiime2-archive-7fzn9g5n/a6d39a9d-42e3-4f5d-a135-8c762b4c1ae7/data /var/folders/sn/92jxwrcd3qn6xp_y7c5rfdkc0000gn/T/tmptz5bcgwy/output.tsv.biom /var/folders/sn/92jxwrcd3qn6xp_y7c5rfdkc0000gn/T/tmptz5bcgwy 270 0 2.0 2 consensus 1.0 1 1000000

R version 3.4.1 (2017-06-30)
Loading required package: Rcpp
DADA2 R package version: 1.6.0

  1. Filtering ...............................
  2. Learning Error Rates
    Initializing error rates to maximum possible estimate.
    Sample 1 - 91807 reads in 56092 unique sequences.

*** caught segfault ***
address 0x8, cause 'memory not mapped'

Traceback:
1: .Call("_dada2_dada_uniques", PACKAGE = "dada2", seqs, abundances, err, quals, score, gap, use_kmers, kdist_cutoff, band_size, omegaA, max_clust, min_fold, min_hamming, min_abund, use_quals, final_consensus, vectorized_alignment, homo_gap, multithread, verbose, SSE)
2: dada_uniques(names(derep[[i]]$uniques), unname(derep[[i]]$uniques), err, qi, opts[["SCORE_MATRIX"]], opts[["GAP_PENALTY"]], opts[["USE_KMERS"]], opts[["KDIST_CUTOFF"]], opts[["BAND_SIZE"]], opts[["OMEGA_A"]], if (initializeErr) { 1 } else { opts[["MAX_CLUST"]] }, opts[["MIN_FOLD"]], opts[["MIN_HAMMING"]], opts[["MIN_ABUNDANCE"]], TRUE, FALSE, opts[["VECTORIZED_ALIGNMENT"]], opts[["HOMOPOLYMER_GAP_PENALTY"]], multithread, (verbose >= 2), opts[["SSE"]])
3: dada(drps[1:i], err = NULL, selfConsist = TRUE, multithread = multithread, VECTORIZED_ALIGNMENT = FALSE, SSE = 2)
An irrecoverable exception occurred. R is aborting now ...
Traceback (most recent call last):
File "/Users/ErikaGanda/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 126, in denoise_single
run_commands([cmd])
File "/Users/ErikaGanda/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 35, in run_commands
subprocess.run(cmd, check=True)
File "/Users/ErikaGanda/miniconda3/envs/qiime2-2017.12/lib/python3.5/subprocess.py", line 398, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['run_dada_single.R', '/var/folders/sn/92jxwrcd3qn6xp_y7c5rfdkc0000gn/T/qiime2-archive-7fzn9g5n/a6d39a9d-42e3-4f5d-a135-8c762b4c1ae7/data', '/var/folders/sn/92jxwrcd3qn6xp_y7c5rfdkc0000gn/T/tmptz5bcgwy/output.tsv.biom', '/var/folders/sn/92jxwrcd3qn6xp_y7c5rfdkc0000gn/T/tmptz5bcgwy', '270', '0', '2.0', '2', 'consensus', '1.0', '1', '1000000']' returned non-zero exit status -11

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/ErikaGanda/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2cli/commands.py", line 224, in call
results = action(**arguments)
File "", line 2, in denoise_single
File "/Users/ErikaGanda/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/qiime2/sdk/action.py", line 228, in bound_callable
output_types, provenance)
File "/Users/ErikaGanda/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/qiime2/sdk/action.py", line 363, in callable_executor
output_views = self._callable(**view_args)
File "/Users/ErikaGanda/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 137, in denoise_single
" and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code -11), please inspect stdout and stderr to learn more.

Thank you so much!

I have the same problem, it seems there are different threads opened for the sample error for 2017.12 release ( *** caught segfault *** address 0x8, cause ‘memory not mapped’/(return code -11)).

I also followed the .Rprofile thread and it did not help.

Looking forward to a new solution.

Thank you!

Hi @ErikaGanda and @Dorothy!

Sorry that this has been such an issue, we’re trying to figure it out.

What version of OS X are you running?

Also, just to see, does activating your environment and then running this command:

CDPATH= R -e 'source("https://bioconductor.org/biocLite.R"); biocLite("dada2")'

fix the issue?

We still don’t really know what’s going on and we’re trying to narrow it down. Thanks for your patience!

Hi @ebolyen!

I am using OX X Yosemite Version 10.10.5
I ran the code above and am currently trying to run DADA2 again. Will let you know what happens.

Thanks for your prompt response, the qiime2 developer team rocks!!! =)

1 Like

Aaaand, I got an error again:

(qiime2-2017.12) dhcp-morris-2820:Rumen ErikaGanda$ qiime dada2 denoise-single \

--i-demultiplexed-seqs rumen-demux.qza
--p-trim-left 0
--p-trunc-len 270
--o-representative-sequences rumen-rep-seqs-dada2.qza
--o-table rumen-table-dada2.qza
--verbose

Plugin error from dada2:

An error was encountered while running DADA2 in R (return code -11), please inspect stdout and stderr to learn more.

Debug info has been saved to /var/folders/sn/92jxwrcd3qn6xp_y7c5rfdkc0000gn/T/qiime2-q2cli-err-2le6304u.log

hcp-morris-2820:T ErikaGanda$ cat qiime2-q2cli-err-2le6304u.log
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_single.R /var/folders/sn/92jxwrcd3qn6xp_y7c5rfdkc0000gn/T/qiime2-archive-ji9nnkhj/a6d39a9d-42e3-4f5d-a135-8c762b4c1ae7/data /var/folders/sn/92jxwrcd3qn6xp_y7c5rfdkc0000gn/T/tmpt1hoqo1n/output.tsv.biom /var/folders/sn/92jxwrcd3qn6xp_y7c5rfdkc0000gn/T/tmpt1hoqo1n 270 0 2.0 2 consensus 1.0 1 1000000

R version 3.4.1 (2017-06-30)
Loading required package: Rcpp
DADA2 R package version: 1.6.0

  1. Filtering ...............................
  2. Learning Error Rates
    Initializing error rates to maximum possible estimate.
    Sample 1 - 91807 reads in 56092 unique sequences.

*** caught segfault ***
address 0x8, cause 'memory not mapped'

Traceback:
1: .Call("_dada2_dada_uniques", PACKAGE = "dada2", seqs, abundances, err, quals, score, gap, use_kmers, kdist_cutoff, band_size, omegaA, max_clust, min_fold, min_hamming, min_abund, use_quals, final_consensus, vectorized_alignment, homo_gap, multithread, verbose, SSE)
2: dada_uniques(names(derep[[i]]$uniques), unname(derep[[i]]$uniques), err, qi, opts[["SCORE_MATRIX"]], opts[["GAP_PENALTY"]], opts[["USE_KMERS"]], opts[["KDIST_CUTOFF"]], opts[["BAND_SIZE"]], opts[["OMEGA_A"]], if (initializeErr) { 1 } else { opts[["MAX_CLUST"]] }, opts[["MIN_FOLD"]], opts[["MIN_HAMMING"]], opts[["MIN_ABUNDANCE"]], TRUE, FALSE, opts[["VECTORIZED_ALIGNMENT"]], opts[["HOMOPOLYMER_GAP_PENALTY"]], multithread, (verbose >= 2), opts[["SSE"]])
3: dada(drps[1:i], err = NULL, selfConsist = TRUE, multithread = multithread, VECTORIZED_ALIGNMENT = FALSE, SSE = 2)
An irrecoverable exception occurred. R is aborting now ...
Traceback (most recent call last):
File "/Users/ErikaGanda/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 126, in denoise_single
run_commands([cmd])
File "/Users/ErikaGanda/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 35, in run_commands
subprocess.run(cmd, check=True)
File "/Users/ErikaGanda/miniconda3/envs/qiime2-2017.12/lib/python3.5/subprocess.py", line 398, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['run_dada_single.R', '/var/folders/sn/92jxwrcd3qn6xp_y7c5rfdkc0000gn/T/qiime2-archive-ji9nnkhj/a6d39a9d-42e3-4f5d-a135-8c762b4c1ae7/data', '/var/folders/sn/92jxwrcd3qn6xp_y7c5rfdkc0000gn/T/tmpt1hoqo1n/output.tsv.biom', '/var/folders/sn/92jxwrcd3qn6xp_y7c5rfdkc0000gn/T/tmpt1hoqo1n', '270', '0', '2.0', '2', 'consensus', '1.0', '1', '1000000']' returned non-zero exit status -11

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/ErikaGanda/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2cli/commands.py", line 224, in call
results = action(**arguments)
File "", line 2, in denoise_single
File "/Users/ErikaGanda/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/qiime2/sdk/action.py", line 228, in bound_callable
output_types, provenance)
File "/Users/ErikaGanda/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/qiime2/sdk/action.py", line 363, in callable_executor
output_views = self._callable(**view_args)
File "/Users/ErikaGanda/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 137, in denoise_single
" and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code -11), please inspect stdout and stderr to learn more.

Hi ebolyen,

Thank you for your help. I ran the command but got same error.

I have macOS High Sierra Version 10.13.2.

Kind regards,
Dorothy

Thanks @ErikaGanda and @Dorothy!

Could you both run this command:

sysctl -a | grep machdep.cpu.leaf7_features

We’re hoping to see an entry for “AVX2”.

Thanks!

Hi @ebolyen!

This is what I got:

dhcp-morris-2820:~ ErikaGanda$ source activate qiime2-2017.12
(qiime2-2017.12) dhcp-morris-2820:~ ErikaGanda$ sysctl -a | grep machdep.cpu.leaf7_features
machdep.cpu.leaf7_features: SMEP ERMS RDWRFSGS TSC_THREAD_OFFSET BMI1 AVX2 BMI2 INVPCID
(qiime2-2017.12) dhcp-morris-2820:~ ErikaGanda$

Thanks @ErikaGanda!

Based on this and @Peris’s response, I think we can rule out missing-AVX2 as the problem. @ErikaGanda can you reproduce this problem with the moving pictures tutorial as well?

Actually I have an even better idea. Could you run this in your QIIME 2 environment:

conda install pytest
pytest --pyargs q2_dada2

It should produce a bunch of output, but you should see all the tests pass. If they don’t that gives us a really easy target while we figure out what is happening.

dhcp-morris-2820:~ ErikaGanda$ source activate qiime2-2017.12
(qiime2-2017.12) dhcp-morris-2820:~ ErikaGanda$ conda install pytest
Fetching package metadata …
Solving package specifications: .

Package plan for installation in environment /Users/ErikaGanda/miniconda3/envs/qiime2-2017.12:

The following NEW packages will be INSTALLED:

attrs:          17.3.0-py35hc219a11_0
coverage:       4.4.2-py35h36907ce_0 
hypothesis:     3.38.5-py35h843401b_0
pluggy:         0.6.0-py35hf57b818_0 
py:             1.5.2-py35h4d4d87d_0 
pympler:        0.5-py35hf41ee38_0   
pytest:         3.3.0-py35hfa6e833_0 
zope:           1.0-py35_0           
zope.interface: 4.4.3-py35hdbc5dd8_0 

Proceed ([y]/n)? pytest --pyargs q2_dada2y
Invalid choice: pytest --pyargs q2_dada2y
Proceed ([y]/n)? y

coverage-4.4.2 100% |################################| Time: 0:00:00 2.79 MB/s
pluggy-0.6.0-p 100% |################################| Time: 0:00:00 11.35 MB/s
py-1.5.2-py35h 100% |################################| Time: 0:00:00 4.35 MB/s
pympler-0.5-py 100% |################################| Time: 0:00:00 4.82 MB/s
zope-1.0-py35_ 100% |################################| Time: 0:00:00 1.44 MB/s
zope.interface 100% |################################| Time: 0:00:00 5.95 MB/s
attrs-17.3.0-p 100% |################################| Time: 0:00:00 9.21 MB/s
hypothesis-3.3 100% |################################| Time: 0:00:00 6.14 MB/s
pytest-3.3.0-p 100% |################################| Time: 0:00:00 7.20 MB/s
(qiime2-2017.12) dhcp-morris-2820:~ ErikaGanda$ pytest --pyargs q2_dada2
============================= test session starts ==============================
platform darwin – Python 3.5.4, pytest-3.3.0, py-1.5.2, pluggy-0.6.0
rootdir: /Users/ErikaGanda, inifile:
plugins: hypothesis-3.38.5
collected 11 items

miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2_dada2/tests/test_denoise.py . [ 9%]
… [100%]
===================================================================== 11 passed in 181.17 seconds ======================================================================
It doesn’t look like anything failed.
Any other thoughts @ebolyen?

Hi @ebolyen, I cannot reproduce the same problem with the moving pictures tutorial.
I started running both my data and the moving pictures in two different terminals. The output of the tutorial looks fine:

Last login: Wed Jan 3 14:13:16 on ttys000
dhcp-morris-2820:~ ErikaGanda$ cd /Users/ErikaGanda/Desktop/qiime2-moving-pictures-tutorial
dhcp-morris-2820:qiime2-moving-pictures-tutorial ErikaGanda$ ls
demux.qza emp-single-end-sequences
demux.qzv sample-metadata.tsv
dhcp-morris-2820:qiime2-moving-pictures-tutorial ErikaGanda$ source activate qiime2-2017.12
(qiime2-2017.12) dhcp-morris-2820:qiime2-moving-pictures-tutorial ErikaGanda$ qiime dada2 denoise-single \

--i-demultiplexed-seqs demux.qza
--p-trim-left 0
--p-trunc-len 120
--o-representative-sequences rep-seqs-dada2.qza
--o-table table-dada2.qza
Saved FeatureTable[Frequency] to: table-dada2.qza
Saved FeatureData[Sequence] to: rep-seqs-dada2.qza
(qiime2-2017.12) dhcp-morris-2820:qiime2-moving-pictures-tutorial ErikaGanda$

My data is still running. If I get another error I will post here. I went back and my demultiplexing files look ok:
rumen-demux.qzv (283.5 KB)

A while ago I had DADA2 give me errors and @jairideout helped me in this post: DADA2 error (return code -5) - #10 by jairideout
Maybe something similar is going on this time around?

Thanks again!!

Thanks for the info @ErikaGanda!

Would you be able to share your data? We’re starting to lean towards something about the data tripping things up. If we can reproduce locally we can start actually debugging the code itself to figure out what’s going on.

Thanks!

Sure thing. Which files do you need @ebolyen?

Thanks @ErikaGanda! rumen-demux.qza would let us run your dada2 denoise-single command from above!

There you go. Let me know if there is anything else I can do!
E

Actually, it looks like the file uploads to 100% in the browser but I don’t see it on the message. Is there any other way I can send you the file? The size is 448.3 MB

Do you have the ability to create a Dropbox or Google Drive link? 450MB is too large for the forum, unfortunately.

@ErikaGanda if you are able, would you also be able to try this out:

conda create -n DONOTUSE --file  https://data.qiime2.org/distro/core/qiime2-2017.12-conda-osx-64.txt
source activate DONOTUSE
wget -qO- https://forum-qiime2-org.s3-us-west-2.amazonaws.com/original/1X/ac621debfd04f561bf1075b4efa56c0e947a1928.txt > `which run_dada_single.R`
wget -qO- https://forum-qiime2-org.s3-us-west-2.amazonaws.com/original/1X/07fa5193f904022996a35a963b12b9ac79dbb1f4.txt > `which run_dada_paired.R`

This will create a dummy environment and patch two files to behave a little differently. If you could try it out on your dataset to see if it crashes the same way, that would be awesome!

Thanks so much for all your help so far!

1 Like

https://cornell.box.com/s/nm5gik81mdp0brwsojznazxqo7ag37p5

There you go!

1 Like

Qiime2 developers: THANK YOU SO MUCH for helping me through this!

I just got an error while running DADA2 on my dataset.
Output is below:

(qiime2-2017.12) dhcp-morris-2820:Rumen ErikaGanda$ ls
Rumen-sequences.qza RumenReads rumen-demux.qzv
RumenMap.tsv rumen-demux.qza
(qiime2-2017.12) dhcp-morris-2820:Rumen ErikaGanda$ qiime dada2 denoise-single \

--i-demultiplexed-seqs rumen-demux.qza
--p-trim-left 0
--p-trunc-len 270
--o-representative-sequences rumen-rep-seqs-dada2.qza
--o-table rumen-table-dada2.qza
--verbose
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_single.R /var/folders/sn/92jxwrcd3qn6xp_y7c5rfdkc0000gn/T/qiime2-archive-q3jvvitx/a6d39a9d-42e3-4f5d-a135-8c762b4c1ae7/data /var/folders/sn/92jxwrcd3qn6xp_y7c5rfdkc0000gn/T/tmp2p4lt4re/output.tsv.biom /var/folders/sn/92jxwrcd3qn6xp_y7c5rfdkc0000gn/T/tmp2p4lt4re 270 0 2.0 2 consensus 1.0 1 1000000

R version 3.4.1 (2017-06-30)
Loading required package: Rcpp
DADA2 R package version: 1.6.0

  1. Filtering ...............................
  2. Learning Error Rates
    Initializing error rates to maximum possible estimate.
    Sample 1 - 91807 reads in 56092 unique sequences.

*** caught segfault ***
address 0x8, cause 'memory not mapped'

Traceback:
1: .Call("_dada2_dada_uniques", PACKAGE = "dada2", seqs, abundances, err, quals, score, gap, use_kmers, kdist_cutoff, band_size, omegaA, max_clust, min_fold, min_hamming, min_abund, use_quals, final_consensus, vectorized_alignment, homo_gap, multithread, verbose, SSE)
2: dada_uniques(names(derep[[i]]$uniques), unname(derep[[i]]$uniques), err, qi, opts[["SCORE_MATRIX"]], opts[["GAP_PENALTY"]], opts[["USE_KMERS"]], opts[["KDIST_CUTOFF"]], opts[["BAND_SIZE"]], opts[["OMEGA_A"]], if (initializeErr) { 1 } else { opts[["MAX_CLUST"]] }, opts[["MIN_FOLD"]], opts[["MIN_HAMMING"]], opts[["MIN_ABUNDANCE"]], TRUE, FALSE, opts[["VECTORIZED_ALIGNMENT"]], opts[["HOMOPOLYMER_GAP_PENALTY"]], multithread, (verbose >= 2), opts[["SSE"]])
3: dada(drps[1:i], err = NULL, selfConsist = TRUE, multithread = multithread, VECTORIZED_ALIGNMENT = FALSE, SSE = 2)
An irrecoverable exception occurred. R is aborting now ...
Traceback (most recent call last):
File "/Users/ErikaGanda/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 126, in denoise_single
run_commands([cmd])
File "/Users/ErikaGanda/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 35, in run_commands
subprocess.run(cmd, check=True)
File "/Users/ErikaGanda/miniconda3/envs/qiime2-2017.12/lib/python3.5/subprocess.py", line 398, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['run_dada_single.R', '/var/folders/sn/92jxwrcd3qn6xp_y7c5rfdkc0000gn/T/qiime2-archive-q3jvvitx/a6d39a9d-42e3-4f5d-a135-8c762b4c1ae7/data', '/var/folders/sn/92jxwrcd3qn6xp_y7c5rfdkc0000gn/T/tmp2p4lt4re/output.tsv.biom', '/var/folders/sn/92jxwrcd3qn6xp_y7c5rfdkc0000gn/T/tmp2p4lt4re', '270', '0', '2.0', '2', 'consensus', '1.0', '1', '1000000']' returned non-zero exit status -11

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/ErikaGanda/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2cli/commands.py", line 224, in call
results = action(**arguments)
File "", line 2, in denoise_single
File "/Users/ErikaGanda/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/qiime2/sdk/action.py", line 228, in bound_callable
output_types, provenance)
File "/Users/ErikaGanda/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/qiime2/sdk/action.py", line 363, in callable_executor
output_views = self._callable(**view_args)
File "/Users/ErikaGanda/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 137, in denoise_single
" and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code -11), please inspect stdout and stderr to learn more.

Plugin error from dada2:

An error was encountered while running DADA2 in R (return code -11), please inspect stdout and stderr to learn more.

See above for debug info.
(qiime2-2017.12) dhcp-morris-2820:Rumen ErikaGanda$