DADA2 error in R (Consistantly getting return codes 1, -6, -11)

Hello!

Recently I have been trying to run ~70 samples through dada2 and have had quite the time. I am running Qiime2 2020.8 and am doing so through the Ohio Supercomputer (OSC) (link for reference: https://www.osc.edu/resources)
I’ve been submitting most of my commands as batch jobs since the server will otherwise kick me off if it runs for ~20 minutes and have tried several different variations of job parameters with no luck. I believe it’s an environmental error based on reading issues other people have had on here, but I have no idea how to fix it! Here is my base script, though I’ve been changing things like the number of nodes, ppn, etc. in the job parameters and/or changing the number of threads or reads learned in the dada2 options. Though all this has accomplished is getting different errors… I’m very much a beginner at this and would appreciate any advice on how to fix this or how to set up the job parameters to make it run. Thank you in advance and sorry for the huge wall of copy and pasted errors!

#PBS -N Seqs_trimming
#PBS -l walltime=6:00:00
#PBS -l nodes=1:ppn=1
#PBS -j oe
#PBS -m abe
#PBS -A PES0823
cd $PBS_O_WORKDIR
cp a.out $TMPDIR
cd $TMPDIR
export OMP_NUM_THREADS=1
./a.out > my_results
cp my_results $PBS_O_WORKDIR

cd /users/PES0823/trutsclr/SerpSystems

source activate qiime2-2020.8

qiime dada2 denoise-single
–i-demultiplexed-seqs ./Serp_seqs.qza
–p-trunc-len 240
–p-trim-left 20
–p-n-threads 1
–p-chimera-method pooled
–p-n-reads-learn 10000
–o-table ./dada2_Serp_table.qza
–o-representative-sequences ./dada2_Serp_rep_set.qza
–o-denoising-stats ./dada2_Serp_stats.qza
–verbose

And here are some of the errors I’ve been receiving:


/var/spool/torque/mom_priv/jobs/11615015.owens-batch.ten.osc.edu.SC: line 23: /users/PES0823/trutsclr/SerpSystems: Is a directory
R version 3.5.1 (2018-07-02)
Loading required package: Rcpp
DADA2: 1.10.0 / Rcpp: 1.0.4.6 / RcppParallel: 5.0.0

  1. Filtering …
  2. Learning Error Rates
    10038380 total bases in 45629 reads from 1 samples will be used for learning the error rates.
  3. Denoise samples …Error in dada_uniques(names(derep[[i]]$uniques), unname(derep[[i]]$uniques), :
    Bad lambda.
    Calls: dada -> dada_uniques
    Execution halted
    Traceback (most recent call last):
    File “/users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 181, in _denoise_single
    run_commands([cmd])
    File “/users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 36, in run_commands
    subprocess.run(cmd, check=True)
    File “/users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/python3.6/subprocess.py”, line 438, in run
    output=stdout, stderr=stderr)
    subprocess.CalledProcessError: Command ‘[‘run_dada_single.R’, ‘/tmp/pbstmp.11615015/qiime2-archive-xp9uldrm/0f26a573-01db-4110-8d6f-6b36655439cf/data’, ‘/tmp/pbstmp.11615015/tmpqtsx8ui7/output.tsv.biom’, ‘/tmp/pbstmp.11615015/tmpqtsx8ui7/track.tsv’, ‘/tmp/pbstmp.11615015/tmpqtsx8ui7’, ‘240’, ‘20’, ‘2.0’, ‘2’, ‘Inf’, ‘independent’, ‘pooled’, ‘1.0’, ‘1’, ‘10000’, ‘NULL’, ‘16’]’ returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/q2cli/commands.py”, line 329, in call
results = action(**arguments)
File “”, line 2, in denoise_single
File “/users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 245, in bound_callable
output_types, provenance)
File “/users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 390, in callable_executor
output_views = self._callable(**view_args)
File “/users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 218, in denoise_single
band_size=‘16’)
File “/users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 192, in _denoise_single
" and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

Plugin error from dada2:

An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

See above for debug info.
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_single.R /tmp/pbstmp.11615015/qiime2-archive-xp9uldrm/0f26a573-01db-4110-8d6f-6b36655439cf/data /tmp/pbstmp.11615015/tmpqtsx8ui7/output.tsv.biom /tmp/pbstmp.11615015/tmpqtsx8ui7/track.tsv /tmp/pbstmp.11615015/tmpqtsx8ui7 240 20 2.0 2 Inf independent pooled 1.0 1 10000 NULL 16


Resources requested:
nodes=1:ppn=1
mem=4gb

Resources used:
cput=00:46:00
walltime=00:46:29
mem=1.795GB
vmem=2.527GB


Another one:

/var/spool/torque/mom_priv/jobs/11615398.owens-batch.ten.osc.edu.SC: line 23: /users/PES0823/trutsclr/SerpSystems: Is a directory
R version 3.5.1 (2018-07-02)
Loading required package: Rcpp
DADA2: 1.10.0 / Rcpp: 1.0.4.6 / RcppParallel: 5.0.0

  1. Filtering …
  2. Learning Error Rates
    43647780 total bases in 198399 reads from 2 samples will be used for learning the error rates.
  3. Denoise samples …*** Error in `/users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/R/bin/exec/R’: free(): invalid pointer: 0x00002aab41daad98 ***
    ======= Backtrace: =========
    /lib64/libc.so.6(+0x816b9)[0x2aab421896b9]
    /users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/R/bin/exec/…/…/lib/libR.so(+0x17e936)[0x2aab41bd7936]
    /users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/R/bin/exec/…/…/lib/libR.so(CONS_NR+0x93)[0x2aab41bd7e23]
    /users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/R/bin/exec/…/…/lib/libR.so(R_PreserveObject+0x11)[0x2aab41bd7fb1]
    /users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/R/library/dada2/libs/dada2.so(_Z9b_compareP1BjN4Rcpp6MatrixILi14ENS1_15PreserveStorageEEEiiiibdibibbb+0xf5)[0x2aab4ba888c5]
    /users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/R/library/dada2/libs/dada2.so(_Z8run_dadaPP3RawiN4Rcpp6MatrixILi14ENS2_15PreserveStorageEEEiiiibdiddidiibbbbbibb+0x4de)[0x2aab4ba7e54e]
    /users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/R/library/dada2/libs/dada2.so(_Z12dada_uniquesSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS5_EES_IiSaIiEES_IbSaIbEEN4Rcpp6MatrixILi14ENSC_15PreserveStorageEEESF_iiibdidddidiibbbibbibb+0x560)[0x2aab4ba7eec0]
    /users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/R/library/dada2/libs/dada2.so(_dada2_dada_uniques+0x72b)[0x2aab4ba760bb]
    /users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/R/bin/exec/…/…/lib/libR.so(+0xfae12)[0x2aab41b53e12]
    (This goes on for a long while so I didn’t copy all of it)
    Traceback (most recent call last):
    File “/users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 181, in _denoise_single
    run_commands([cmd])
    File “/users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 36, in run_commands
    subprocess.run(cmd, check=True)
    File “/users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/python3.6/subprocess.py”, line 438, in run
    output=stdout, stderr=stderr)
    subprocess.CalledProcessError: Command ‘[‘run_dada_single.R’, ‘/tmp/pbstmp.11615398/qiime2-archive-c4qbf8dc/0f26a573-01db-4110-8d6f-6b36655439cf/data’, ‘/tmp/pbstmp.11615398/tmpy9u2j47j/output.tsv.biom’, ‘/tmp/pbstmp.11615398/tmpy9u2j47j/track.tsv’, ‘/tmp/pbstmp.11615398/tmpy9u2j47j’, ‘240’, ‘20’, ‘2.0’, ‘2’, ‘Inf’, ‘independent’, ‘pooled’, ‘1.0’, ‘1’, ‘100000’, ‘NULL’, ‘16’]’ died with <Signals.SIGABRT: 6>.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/q2cli/commands.py”, line 329, in call
results = action(**arguments)
File “”, line 2, in denoise_single
File “/users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 245, in bound_callable
output_types, provenance)
File “/users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 390, in callable_executor
output_views = self._callable(**view_args)
File “/users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 218, in denoise_single
band_size=‘16’)
File “/users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 192, in _denoise_single
" and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code -6), please inspect stdout and stderr to learn more.

Plugin error from dada2:

An error was encountered while running DADA2 in R (return code -6), please inspect stdout and stderr to learn more.

See above for debug info.
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_single.R /tmp/pbstmp.11615398/qiime2-archive-c4qbf8dc/0f26a573-01db-4110-8d6f-6b36655439cf/data /tmp/pbstmp.11615398/tmpy9u2j47j/output.tsv.biom /tmp/pbstmp.11615398/tmpy9u2j47j/track.tsv /tmp/pbstmp.11615398/tmpy9u2j47j 240 20 2.0 2 Inf independent pooled 1.0 1 100000 NULL 16


Resources requested:
nodes=2:ppn=1
mem=4gb

Resources used:
cput=00:59:48
walltime=01:00:04
mem=1.878GB
vmem=6.954GB


And one other error:
/var/spool/torque/mom_priv/jobs/11616115.owens-batch.ten.osc.edu.SC: line 23: /users/PES0823/trutsclr/SerpSystems: Is a directory
R version 3.5.1 (2018-07-02)
Loading required package: Rcpp
DADA2: 1.10.0 / Rcpp: 1.0.4.6 / RcppParallel: 5.0.0

  1. Filtering …
  2. Learning Error Rates
    43647780 total bases in 198399 reads from 2 samples will be used for learning the error rates.
  3. Denoise samples …
    *** caught segfault ***
    address (nil), cause ‘unknown’
    Traceback (most recent call last):
    File “/users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 181, in _denoise_single
    run_commands([cmd])
    File “/users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 36, in run_commands
    subprocess.run(cmd, check=True)
    File “/users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/python3.6/subprocess.py”, line 438, in run
    output=stdout, stderr=stderr)
    subprocess.CalledProcessError: Command ‘[‘run_dada_single.R’, ‘/tmp/pbstmp.11616115/qiime2-archive-xpxc4piv/0f26a573-01db-4110-8d6f-6b36655439cf/data’, ‘/tmp/pbstmp.11616115/tmpy2es7ss1/output.tsv.biom’, ‘/tmp/pbstmp.11616115/tmpy2es7ss1/track.tsv’, ‘/tmp/pbstmp.11616115/tmpy2es7ss1’, ‘240’, ‘20’, ‘2.0’, ‘2’, ‘Inf’, ‘independent’, ‘pooled’, ‘1.0’, ‘1’, ‘100000’, ‘NULL’, ‘16’]’ died with <Signals.SIGSEGV: 11>.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/q2cli/commands.py”, line 329, in call
results = action(**arguments)
File “”, line 2, in denoise_single
File “/users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 245, in bound_callable
output_types, provenance)
File “/users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 390, in callable_executor
output_views = self._callable(**view_args)
File “/users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 218, in denoise_single
band_size=‘16’)
File “/users/PES0823/trutsclr/miniconda3/envs/qiime2-2020.8/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 192, in _denoise_single
" and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code -11), please inspect stdout and stderr to learn more.

Plugin error from dada2:

An error was encountered while running DADA2 in R (return code -11), please inspect stdout and stderr to learn more.

See above for debug info.
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_single.R /tmp/pbstmp.11616115/qiime2-archive-xpxc4piv/0f26a573-01db-4110-8d6f-6b36655439cf/data /tmp/pbstmp.11616115/tmpy2es7ss1/output.tsv.biom /tmp/pbstmp.11616115/tmpy2es7ss1/track.tsv /tmp/pbstmp.11616115/tmpy2es7ss1 240 20 2.0 2 Inf independent pooled 1.0 1 100000 NULL 16

Resources requested:
nodes=2:ppn=1
mem=4gb

Resources used:
cput=00:53:48
walltime=00:54:10
mem=1.813GB
vmem=6.905GB

Hi @Trutsclr!
These are some unfriendly errors, huh? :camel:

This appears to be a known issue (1, 2), with running DADA2 on certain system configurations.

These two ugly mugs :man_supervillain: :woman_supervillain: look kinda like errors you would get if your system’s scheduling software terminated your job. I’m not sure what’s going on here, but this often happens when your commands exceed some resource allocation (time, memory, etc.).

If you need a quick fix, your best bet might be to try running on a different system. Unless your samples are very large, this could be a strong laptop or a desktop computer. In the meantime, it might be worth talking with your sysadmin about these issues. This will let you confirm that your job allocations are reasonable, and might get you some insight into how to interpret or correct the errors you’re running into.

Let me know how everything goes!
Chris :turtle:

1 Like