Error denoising reads (return code -11)

Hello,

I am running the following script on a publicly available data set, and am getting some error codes that I can’t decipher. I am able to import the demultiplexed single-end reads, but the denoising step is not running. Please see code and error message from the slurm out file:

qiime dada2 denoise-single
–i-demultiplexed-seqs single-end-demux1.qza
–p-trim-left 0
–p-trunc-len 147
–o-representative-sequences rep-seqs-dada2.qza
–o-table table-dada2.qza
–o-denoising-stats denoising-stats-dada2.qza
–verbose

R version 3.5.1 (2018-07-02)
Loading required package: Rcpp
DADA2: 1.10.0 / Rcpp: 1.0.1 / RcppParallel: 4.4.2

  1. Filtering …
  2. Learning Error Rates
    167082111 total bases in 1136613 reads from 1 samples will be used for learning the error rates.
  3. Denoise samples …
    *** caught segfault ***
    address (nil), cause ‘unknown’

Traceback:
1: dada(drp, err = err, multithread = multithread, HOMOPOLYMER_GAP_PENALTY = HOMOPOLYMER_GAP_PENALTY, BAND_SIZE = BAND_SIZE, verbose = FALSE)
An irrecoverable exception occurred. R is aborting now …
Traceback (most recent call last):
File “/nas/longleaf/apps/qiime2/2019.4/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 152, in _denoise_single
run_commands([cmd])
File “/nas/longleaf/apps/qiime2/2019.4/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 36, in run_commands
subprocess.run(cmd, check=True)
File “/nas/longleaf/apps/qiime2/2019.4/miniconda3/envs/qiime2-2019.4/lib/python3.6/subprocess.py”, line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command ‘[‘run_dada_single.R’, ‘/tmp/qiime2-archive-o7__8fik/f5effcc4-6d01-4d9b-9291-54d70d160723/data’, ‘/tmp/tmpnj78edhv/output.tsv.biom’, ‘/tmp/tmpnj78edhv/track.tsv’, ‘/tmp/tmpnj78edhv’, ‘147’, ‘0’, ‘2.0’, ‘2’, ‘Inf’, ‘consensus’, ‘1.0’, ‘1’, ‘1000000’, ‘NULL’, ‘16’]’ died with <Signals.SIGSEGV: 11>.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/nas/longleaf/apps/qiime2/2019.4/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/q2cli/commands.py”, line 311, in call
results = action(**arguments)
File “</nas/longleaf/apps/qiime2/2019.4/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/decorator.py:decorator-gen-449>”, line 2, in denoise_single
File “/nas/longleaf/apps/qiime2/2019.4/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 231, in bound_callable
output_types, provenance)
File “/nas/longleaf/apps/qiime2/2019.4/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 365, in callable_executor
output_views = self._callable(**view_args)
File “/nas/longleaf/apps/qiime2/2019.4/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 187, in denoise_single
band_size=‘16’)
File “/nas/longleaf/apps/qiime2/2019.4/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 163, in _denoise_single
" and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code -11), please inspect stdout and stderr to learn more.

Plugin error from dada2:

An error was encountered while running DADA2 in R (return code -11), please inspect stdout and stderr to learn more.

See above for debug info.
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_single.R /tmp/qiime2-archive-o7__8fik/f5effcc4-6d01-4d9b-9291-54d70d160723/data /tmp/tmpnj78edhv/output.tsv.biom /tmp/tmpnj78edhv/track.tsv /tmp/tmpnj78edhv 147 0 2.0 2 Inf consensus 1.0 1 1000000 NULL 16

Thank you very much for your help!

Hello Daria :wave:

That’s an error related to memory. I wonder if the computer does not have enough RAM / memory to run dada2 with these settings?

How much memory does this computer have? Do you get the same error when when running dada2 using --p-n-reads-learn 100000 or --p-n-reads-learn 10000, which would reduce the memory requirements?

Colin :whale:

Hi Colin,

Thank you so much for the suggestion! I am running the program on longleaf (computing cluster) so I’m not sure my computer memory would be the issue? I tried inserting a line into the code (–p-n-reads-learn 100000) as you suggested, and am now getting the following error message:

Traceback (most recent call last):
File “/nas/longleaf/apps/qiime2/2019.4/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 152, in _denoise_single
run_commands([cmd])
File “/nas/longleaf/apps/qiime2/2019.4/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 36, in run_commands
subprocess.run(cmd, check=True)
File “/nas/longleaf/apps/qiime2/2019.4/miniconda3/envs/qiime2-2019.4/lib/python3.6/subprocess.py”, line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command ‘[‘run_dada_single.R’, ‘/tmp/qiime2-archive-mzqwi17q/f5effcc4-6d01-4d9b-9291-54d70d160723/data’, ‘/tmp/tmpf7av969x/output.tsv.biom’, ‘/tmp/tmpf7av969x/track.tsv’, ‘/tmp/tmpf7av969x’, ‘147’, ‘0’, ‘2.0’, ‘2’, ‘Inf’, ‘consensus’, ‘1.0’, ‘1’, ‘100000’, ‘NULL’, ‘16’]’ died with <Signals.SIGABRT: 6>.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/nas/longleaf/apps/qiime2/2019.4/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/q2cli/commands.py”, line 311, in call
results = action(**arguments)
File “</nas/longleaf/apps/qiime2/2019.4/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/decorator.py:decorator-gen-449>”, line 2, in denoise_single
File “/nas/longleaf/apps/qiime2/2019.4/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 231, in bound_callable
output_types, provenance)
File “/nas/longleaf/apps/qiime2/2019.4/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 365, in callable_executor
output_views = self._callable(**view_args)
File “/nas/longleaf/apps/qiime2/2019.4/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 187, in denoise_single
band_size=‘16’)
File “/nas/longleaf/apps/qiime2/2019.4/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 163, in _denoise_single
" and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code -6), please inspect stdout and stderr to learn more.

Plugin error from dada2:

An error was encountered while running DADA2 in R (return code -6), please inspect stdout and stderr to learn more.

See above for debug info.
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_single.R /tmp/qiime2-archive-mzqwi17q/f5effcc4-6d01-4d9b-9291-54d70d160723/data /tmp/tmpf7av969x/output.tsv.biom /tmp/tmpf7av969x/track.tsv /tmp/tmpf7av969x 147 0 2.0 2 Inf consensus 1.0 1 100000 NULL 16

Might you have any other suggestions?

Thank you again! Daria

Hello again,

Looks like we got a new error this time:

… but I’m still not sure what’s causing this, and I’m not 100% certain that it’s from DADA2 and not your compute cluster ‘longleaf’ :leafy_green:

Have you shown this error to your HPC team and got their feedback on how you are submitting this job to the cluster? Maybe there’s a memory setting you need to pass to the cluster when you run the script. :thinking: