DADA2 Exit Code -11

Harry · December 25, 2017, 5:59pm

Thanks! Finally, I successfully use R to run dada2 within 12 hours. That 's great. But I have another question regarding dada2.

Since qiime2-2017.12 version is released, I am trying the use qiime2-2017.12 to run dada2 1.6, but always get such error:

Plugin error from dada2:

An error was encountered while running DADA2 in R (return code -11), please inspect stdout and stderr to learn more.

Debug info has been saved to /var/folders/b0/k51zwjxj54g4z8mxsxlxgf6r0000gn/T/qiime2-q2cli-err-bftjvvjj.log

Very confused, I use terminal, why it runs R in terminal and get such error? How could I solve that?

Happy Holiday

thermokarst · December 27, 2017, 4:56pm

Hi @Harry, sorry to hear things aren't going well. Can you provide the following information:

The full command you ran (copy-and-paste please!)
If possible, your demux summarize visualization for the sequences you are trying to process through q2-dada2
The detailed output log, when run with --verbose, or, if the log listed above is still available, please attach that (/var/folders/b0/k51zwjxj54g4z8mxsxlxgf6r0000gn/T/qiime2-q2cli-err-bftjvvjj.log)

Thanks!

Martin · December 29, 2017, 12:48pm

Hi,

I encounter the same problem with release qiime2-2017.12 (it worked fine in qiime2-2017.11)

The full command:

qiime dada2 denoise-single
--p-trim-left 0
--p-trunc-len 0
--i-demultiplexed-seqs trimmed-seqs.qza
--p-n-threads 4
--o-representative-sequences rep-seqs.qza
--o-table table.qza
--verbose

detailed output log:

Command: run_dada_single.R /tmp/qiime2-archive-r2d1yed_/571a782a-373a-49f6-92b9-7fa50a71821c/data /tmp/tmp5ltx0l07/output.tsv.biom /tmp/tmp5ltx0l07 0 0 2.0 2 consensus 1.0 4 1000000

R version 3.4.1 (2017-06-30)
Loading required package: Rcpp
DADA2 R package version: 1.6.0

Filtering .......
Learning Error Rates
Not all sequences were the same length.
Not all sequences were the same length.
Not all sequences were the same length.
Not all sequences were the same length.
Not all sequences were the same length.
Not all sequences were the same length.
Not all sequences were the same length.
Initializing error rates to maximum possible estimate.
Sample 1 - 19785 reads in 10633 unique sequences.

*** caught segfault ***
address 0x8, cause 'memory not mapped'

Traceback:
1: .Call("_dada2_dada_uniques", PACKAGE = "dada2", seqs, abundances, err, quals, score, gap, use_kmers, kdist_cutoff, band_size, omegaA, max_clust, min_fold, min_hamming, min_abund, use_quals, final_consensus, vectorized_alignment, homo_gap, multithread, verbose, SSE)
2: dada_uniques(names(derep[[i]]$uniques), unname(derep[[i]]$uniques), err, qi, opts[["SCORE_MATRIX"]], opts[["GAP_PENALTY"]], opts[["USE_KMERS"]], opts[["KDIST_CUTOFF"]], opts[["BAND_SIZE"]], opts[["OMEGA_A"]], if (initializeErr) { 1 } else { opts[["MAX_CLUST"]] }, opts[["MIN_FOLD"]], opts[["MIN_HAMMING"]], opts[["MIN_ABUNDANCE"]], TRUE, FALSE, opts[["VECTORIZED_ALIGNMENT"]], opts[["HOMOPOLYMER_GAP_PENALTY"]], multithread, (verbose >= 2), opts[["SSE"]])
3: dada(drps[1:i], err = NULL, selfConsist = TRUE, multithread = multithread, VECTORIZED_ALIGNMENT = FALSE, SSE = 2)
An irrecoverable exception occurred. R is aborting now ...
Traceback (most recent call last):
File "/home/martin/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 126, in denoise_single
run_commands([cmd])
File "/home/martin/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2_dada2/denoise.py", line 35, in run_commands
subprocess.run(cmd, check=True)
File "/home/martin/miniconda3/envs/qiime2-2017.12/lib/python3.5/subprocess.py", line 398, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['run_dada_single.R', '/tmp/qiime2-archive-r2d1yed/571a782a-373a-49f6-92b9-7fa50a71821c/data', '/tmp/tmp5ltx0l07/output.tsv.biom', '/tmp/tmp5ltx0l07', '0', '0', '2.0', '2', 'consensus', '1.0', '4', '1000000']' returned non-zero exit status -11

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/martin/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2cli/commands.py", line 224, in call
results = action(**arguments)
File "", line 2, in denoise_single
File "/home/martin/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/qiime2/sdk/action.py", line 228, in bound_callable
output_types, provenance)
File "/home/martin/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/qiime2/sdk/action.py", line 363, in callable_executor
output_views = self._callable(**view_args)
File "/home/martin/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 137, in denoise_single
" and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code -11), please inspect stdout and stderr to learn more.

Plugin error from dada2:

An error was encountered while running DADA2 in R (return code -11), please inspect stdout and stderr to learn more.

See above for debug info.

demux summarize visulization:

ebolyen · December 29, 2017, 8:52pm

Hi @Martin!

Do you have an existing R/R Studio installation, if so what version was it? Something that changed between 2017.11 and 2017.12 is we are now using R 3.4.1.

I wonder if some people weren't using the RProfile trick and instead had a "compatible" R installed.

Thanks and Happy New Years!

Martin · December 30, 2017, 2:09pm

Hi,

I am a bit stucked. I am using a clean Linux Debian distribution with only qiime1&2 installed according to the install docs. There should be no existing R/R Studio installation.

Based on the description in the .RProfile trick I tested the command:

R -e ".libPaths()"

This command returned:

> .libPaths()
[1] "/home/martin/miniconda3/envs/qiime2-2017.12/lib/R/library"
> 
>

I guess this means that everything is okay and no conflicting R installations should be expected?

I even tried to create an ~/.Rprofile file, including the line .libPaths(.libPaths()[1]).

This didn't changed anything and I still encounter the return code -11 while running DADA2.

Cheers,
Martin

[EDIT]
I tried to remove old qiime2 environments (remove -n qiime2-2017.10, remove -n qiime2-2017.11, remove -n qiime2-2017.12, conda clean -y --all) and qiime2 re-installation, but without any success.

ebolyen · January 2, 2018, 7:11pm

Hey @Martin,

What does uname --all say for you? I think you are the first one to report this issue on Linux.

Also does running the following command in your QIIME 2 environment fix the issue?

CDPATH= R -e 'source("https://bioconductor.org/biocLite.R"); biocLite("dada2")'

Sorry, we're still trying to figure this out. Thanks for your patience!

Martin · January 2, 2018, 11:17pm

Hi @ebolyen,

thanks for looking into this! (and sorry for hijacking the thread @Harry)

uname --all returns:

Linux debian 4.9.0-4-amd64 #1 SMP Debian 4.9.65-3+deb9u1 (2017-12-23) x86_64 GNU/Linux

I further tried the command: CDPATH= R -e 'source("https://bioconductor.org/biocLite.R"); biocLite("dada2")' (in qiime2 environment).

This resulted in successful downloading and installing of:

Bioconductor version 3.6 (BiocInstaller 1.28.0)

and

dada2

Unfortunately, I still get the same -11 exit code when running dada2 with the following command:

qiime dada2 denoise-single
--p-trim-left 0
--p-trunc-len 0
--i-demultiplexed-seqs trimmed-seqs.qza
--p-n-threads 4
--o-representative-sequences rep-seqs.qza
--o-table table.qza
--verbose

Anything else I can do to assist with this problem? I am open for experiments

Best,
Martin

ebolyen · January 3, 2018, 6:19pm

Thanks @Martin!

Could you run the following:

grep avx2 /proc/cpuinfo | wc -l

You should see a number that matches the number of CPUs you have.

If that shows a number other than 0, then we need to look closer. Would you be able to share your data with us? We're not sure if it's a system problem or an input problem yet. Also if you are available to do further testing, we could try a very hand-crafted R installation to see if it's a linking problem in the binary.

Thanks!

ebolyen · January 3, 2018, 9:33pm

Here's another thing to try:

conda create -n DONOTUSE --file https://data.qiime2.org/distro/core/qiime2-2017.12-conda-linux-64.txt
source activate DONOTUSE
wget -qO- https:/uploads/qiime21/original/1X/ac621debfd04f561bf1075b4efa56c0e947a1928.txt > `which run_dada_single.R`
wget -qO- https:/uploads/qiime21/original/1X/07fa5193f904022996a35a963b12b9ac79dbb1f4.txt > `which run_dada_paired.R`

This'll create a dummy environment with a few tweaks. If you could test out your data inside this environment that would be really helpful!

Martin · January 3, 2018, 9:41pm

Hi @ebolyen,

the command grep avx2 /proc/cpuinfo | wc -l says that I have 8 CPUs.

I uploaded the qiime2 artifact which I wanted to use as input for dada2. If interested, I could give the full script I used to prepare these data.

And yes, I am available for further testing. Actually, I am very happy and positively surprised about the effort you put so far into my problem

Cheers,
Martin

[Edit]
hmmm... I uploaded the file trimmed-seqs.qza (twice), but I do not see how to link it. Did it worked?

ebolyen · January 3, 2018, 9:48pm

Our forum has a size limit and doesn't seem to be returning an error like it's supposed to. We fortunately have just been able to reproduce the issue with @Peris's data, so no worries there!

Thanks again for all the help, we're going to get started on the next steps now that we can reproduce!

Martin · January 3, 2018, 10:00pm

Well, I guess its a good thing to reproduce errors

The suggested creation of the conda DONOTUSE environment works!

Normally the dada2 stopped after giving this output:

Initializing error rates to maximum possible estimate.
Sample 1 - 19785 reads in 10633 unique sequences.

Now it proceeded with all of my 7 samples and is still running....

thermokarst · January 5, 2018, 8:08pm

We have just released a patched update to q2-dada2 that appears to fix this . Please follow the instructions here to get your hands on the patch, and please let us know how it goes! Thanks so much for all the support while we were debugging this!

Martin · January 6, 2018, 11:19pm

Well done

From my point of view, the issue is solved!

system · February 7, 2018, 5:20am

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.