DADA2 -11 exit code (paired-end reads)

That is super helpful! Thanks so much!

1 Like

@Peris if you have a moment, could you run the following in your QIIME 2 environment:

conda install pytest
pytest --pyargs q2_dada2

This will run our unit-tests, if they fail with the same issue, then we have a very small dataset we can use to reproduce this. I kind of hope they fail since it will make our debugging simpler.

Hi @ebolyen,

Please find my outputs:

(qiime2-2017.12) Periss-MacBook-Pro:~ perismunyaka$ conda install pytest

Fetching package metadata …
Solving package specifications: .

Package plan for installation in environment /Users/perismunyaka/miniconda3/envs/qiime2-2017.12:

The following NEW packages will be INSTALLED:

attrs:          17.3.0-py35hc219a11_0
coverage:       4.4.2-py35h36907ce_0 
hypothesis:     3.38.5-py35h843401b_0
pluggy:         0.6.0-py35hf57b818_0 
py:             1.5.2-py35h4d4d87d_0 
pympler:        0.5-py35hf41ee38_0   
pytest:         3.3.0-py35hfa6e833_0 
zope:           1.0-py35_0           
zope.interface: 4.4.3-py35hdbc5dd8_0 

Proceed ([y]/n)? y

coverage-4.4.2 100% |########################################################| Time: 0:00:00 1.82 MB/s
pluggy-0.6.0-p 100% |########################################################| Time: 0:00:00 5.42 MB/s
py-1.5.2-py35h 100% |########################################################| Time: 0:00:00 1.48 MB/s
pympler-0.5-py 100% |########################################################| Time: 0:00:00 1.87 MB/s
zope-1.0-py35_ 100% |########################################################| Time: 0:00:00 511.70 kB/s
zope.interface 100% |########################################################| Time: 0:00:00 1.99 MB/s
attrs-17.3.0-p 100% |########################################################| Time: 0:00:00 7.48 MB/s
hypothesis-3.3 100% |########################################################| Time: 0:00:00 1.55 MB/s
pytest-3.3.0-p 100% |########################################################| Time: 0:00:00 1.47 MB/s

(qiime2-2017.12) Periss-MacBook-Pro:~ perismunyaka$ pytest --pyargs q2_dada2

========================================= test session starts ==========================================
platform darwin – Python 3.5.4, pytest-3.3.0, py-1.5.2, pluggy-0.6.0
rootdir: /Users/perismunyaka, inifile:
plugins: hypothesis-3.38.5
collected 11 items

miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2_dada2/tests/test_denoise.py … [ 54%]
… [100%]

===================================== 11 passed in 185.51 seconds ======================================

1 Like

Thank you so much for all your help @Peris!

I’ve run a bunch of tests locally as well and I’m not seeing anything either. So hopefully this isn’t a stochastic issue (though it’s virtually impossible to prove that it isn’t…).

Does the DADA 2 step of the Moving Pictures tutorial consistently fail for you?

I think we’re looking at a mix of having a more realistic dataset (so not our tests) and the host. We’ve been able to run the Moving Pictures on our test hosts (macbook pro, mac mini, docker containers, and our own dev computers) and haven’t been able to see this ourselves yet :frowning:.

Hi @ebolyen,

I went through the Moving Pictures tutorial before I started running my own data, and that’s when I got the error for the first time (this was before I made my first post here).

Now I have repeated dada2 analysis for the Moving Tutorial data and this time round, it worked!

I got excited and decided to repeat dada2 for my data; however, I still got the error again!
I had 312 samples in my imported qiime2 artifatcs so, I decided to split the samples and tried running dada2 with fewer samples (around 59), but I still got the same error.

Attached please find part of my data that I tried dada2 on, and got the error!

Hi @Peris!

Did you mean to attach a file? If so, it doesn't look like it worked - can you try again? Thanks!

Was this before you made the Rprofile changes? Without that error log, we can't say for sure if you were experiencing the same error then or not, which might be a good thing (you have reported two separate errors here in this thread - I think the first one was resolved with the Rprofile fix).

If you are seeing stochastic errors with the Moving Pictures data, that type of bug is pretty hard to reproduce and diagnose, but if it is actually the case that the Moving Pictures error you experienced was an entirely different one, then maybe we can keep working the angle that this error is data-specific. To that end, as I mentioned above, getting a dataset that reproduces this error would be extremely helpful for us, since we have been unable to reproduce this specific error locally. Thanks! :t_rex:

Shot in the dark here, but would you be able to run the following:

conda create -n DONOTUSE --file  https://data.qiime2.org/distro/core/qiime2-2017.12-conda-osx-64.txt
source activate DONOTUSE
wget -qO- https://forum-qiime2-org.s3-us-west-2.amazonaws.com/original/1X/ac621debfd04f561bf1075b4efa56c0e947a1928.txt > `which run_dada_single.R`
wget -qO- https://forum-qiime2-org.s3-us-west-2.amazonaws.com/original/1X/07fa5193f904022996a35a963b12b9ac79dbb1f4.txt > `which run_dada_paired.R`

This is going to overwrite some stuff which might help. Then run dada2 like normal on your dataset.

Thanks again!

Hi @thermokarst,

Yes I did upload a file, but I can’t see it either! I used the upload button here and the upload was successful although I can’t see the uploaded file.

Is there a better way of sharing the data?

Thanks

It’s probably too big for our forum :frowning: Do you have a google-drive or dropbox link you could create/share?

Thanks!

Hi @thermokarst,

Yes that was before I made the Rprofile changes. Unfortunately I didn’t save that error log, too bad!

Now the moving pictures tutorial has worked for me so, may be the error was related to the Rprofile (but as you mentioned, I can’t prove this without the error log)

Thanks again

Hi @ebolyen and @thermokarst

Doest this work?

2 Likes

@thermokarst can reproduce the issue!!! Thanks so much! We can start actually debugging the code now to see what's going on under the hood. Stay tuned!

1 Like

@ebolyen

@thermokarst can reproduce the issue!!! Thanks so much! We can start actually debugging the code now to see what’s going on under the hood. Stay tuned!

Thanks for letting me know!

Shot in the dark here, but would you be able to run the following:

conda create -n DONOTUSE --file https://data.qiime2.org/distro/core/qiime2-2017.12-conda-osx-64.txt
source activate DONOTUSE
wget -qO- https://forum-qiime2-org.s3-us-west-2.amazonaws.com/original/1X/ac621debfd04f561bf1075b4efa56c0e947a1928.txt > which run_dada_single.R
wget -qO- https://forum-qiime2-org.s3-us-west-2.amazonaws.com/original/1X/07fa5193f904022996a35a963b12b9ac79dbb1f4.txt > which run_dada_paired.R

This is going to overwrite some stuff which might help. Then run dada2 like normal on your dataset.

I have finished running these scripts and am currently running dada2 on the data that I shared. I will let you know how it goes!

Your continued help is highly appreciated

Hi ebolyen,

This worked for my own data! Thank you! Are you going to fix it in the qiime2-2017.12 release? Do we need to reinstall?

I got the same problem on my Ubuntu 14.04, qiime2-2017.12

Plugin error from dada2:

An error was encountered while running DADA2 in R (return code -11), please inspect stdout and stderr to learn more.

Debug info has been saved to /tmp/qiime2-q2cli-err-lgevf14g.log

Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_paired.R /tmp/tmp8ct42kir/forward /tmp/tmp8ct42kir/reverse /tmp/tmp8ct42kir/output.tsv.biom /tmp/tmp8ct42kir/filt_f /tmp/tmp8ct42kir/filt_r 300 300 0 0 2.0 2 consensus 1.0 0 1000000

R version 3.4.1 (2017-06-30)
Loading required package: Rcpp
DADA2 R package version: 1.6.0

  1. Filtering …
  2. Learning Error Rates
    2a) Forward Reads
    Initializing error rates to maximum possible estimate.
    Sample 1 - 34346 reads in 32495 unique sequences.

*** caught segfault ***
address 0x8, cause ‘memory not mapped’

We have just released a patched update to q2-dada2 that appears to fix this :crossed_fingers: . Please follow the instructions here to get your hands on the patch, and please let us know how it goes! Thanks so much for all the support while we were debugging this! :t_rex:

Hi @thermokarst,

Thank you for this update and thank you again to the entire Qiime2 group working together to help all of us! I will follow the instructions on the above update and will let you know you.

FYI - I tried the (DONOTUSE) option that @ebolyen suggested earlier. It worked for a subset of my data although it really took long to finish, then I tried it on my entire data but it’s almost 48 hrs now and it’s still running! I will terminate it!

Kind regards

2 Likes

Hi @thermokarst @ebolyen

All worked for me!

Thank you so much

1 Like

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.