Return code 1 running DADA2 with single-end PGM/Ion torrent data

Technical Support
1

Hello,
Ihave an error (return code 1) running DADA2 (qiime2 2019.1) with this:

qiime dada2 denoise-single \

--i-demultiplexed-seqs demux.qza
--p-n-threads 0
--p-trim-left 15
--p-trunc-len 291
--o-representative-sequences rep-seqs-dada2.qza
--o-table table-dada2.qza
--o-denoising-stats stats-dada2.qza
Plugin error from dada2:

An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

Debug info has been saved to /media/sf_qiime2/temp/qiime2-q2cli-err-5y3wmcox.log

As can you see I changed the tmp folder to other (/media/sf_qiime2/temp/) as is suggested in the discussion An error was encountered while running DADA2 in R (return code 1), but the error still persists.

The log file contains the following information:

Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_single.R /media/sf_qiime2/temp/qiime2-archive-hl8vluwv/9526611e-608a-4425-9e9f-99e212ab7669/data /media/sf_qiime2/temp/tmpjuhdtqix/output.tsv.biom /media/sf_qiime2/temp/tmpjuhdtqix/track.tsv /media/sf_qiime2/temp/tmpjuhdtqix 291 15 2.0 2 Inf consensus 1.0 0 1000000 NULL 16

R version 3.4.1 (2017-06-30)
Loading required package: Rcpp
DADA2 R package version: 1.6.0

  1. Filtering Error in filterAndTrim(unfilts, filts, truncLen = truncLen, trimLeft = trimLeft, :
    These are the errors (up to 5) encountered in individual cores...
    Error in add(bin) : internal: buf !=
    Error in add(raw()) : internal: buf !=
    Error in add(bin) : internal: buf !=
    Error in add(raw()) : internal: buf !=
    Error in add(bin) : internal: buf !=
    Execution halted
    Traceback (most recent call last):
    File "/home/qiime2/miniconda/envs/qiime2-2019.1/lib/python3.6/site-packages/q2_dada2/_denoise.py", line 152, in _denoise_single
    run_commands([cmd])
    File "/home/qiime2/miniconda/envs/qiime2-2019.1/lib/python3.6/site-packages/q2_dada2/_denoise.py", line 36, in run_commands
    subprocess.run(cmd, check=True)
    File "/home/qiime2/miniconda/envs/qiime2-2019.1/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
    subprocess.CalledProcessError: Command '['run_dada_single.R', '/media/sf_qiime2/temp/qiime2-archive-hl8vluwv/9526611e-608a-4425-9e9f-99e212ab7669/data', '/media/sf_qiime2/temp/tmpjuhdtqix/output.tsv.biom', '/media/sf_qiime2/temp/tmpjuhdtqix/track.tsv', '/media/sf_qiime2/temp/tmpjuhdtqix', '291', '15', '2.0', '2', 'Inf', 'consensus', '1.0', '0', '1000000', 'NULL', '16']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/qiime2/miniconda/envs/qiime2-2019.1/lib/python3.6/site-packages/q2cli/commands.py", line 274, in call
results = action(**arguments)
File "</home/qiime2/miniconda/envs/qiime2-2019.1/lib/python3.6/site-packages/decorator.py:decorator-gen-440>", line 2, in denoise_single
File "/home/qiime2/miniconda/envs/qiime2-2019.1/lib/python3.6/site-packages/qiime2/sdk/action.py", line 231, in bound_callable
output_types, provenance)
File "/home/qiime2/miniconda/envs/qiime2-2019.1/lib/python3.6/site-packages/qiime2/sdk/action.py", line 365, in callable_executor
output_views = self._callable(**view_args)
File "/home/qiime2/miniconda/envs/qiime2-2019.1/lib/python3.6/site-packages/q2_dada2/_denoise.py", line 187, in denoise_single
band_size='16')
File "/home/qiime2/miniconda/envs/qiime2-2019.1/lib/python3.6/site-packages/q2_dada2/_denoise.py", line 163, in _denoise_single
" and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

Aditional informaction: My data are single end PGM/Ion torrent and are imported with:

--type 'SampleData[SequencesWithQuality]'
--input-path se-33-manifest
--output-path demux.qza
--input-format SingleEndFastqManifestPhred33

The se-33-manifest was:

single-end PHRED 33 fastq manifest file for forward reads

sample-id,absolute-filepath,direction
F01,$PWD/se-33/F01.fastq,forward
F02,$PWD/se-33/F02.fastq,forward
F05,$PWD/se-33/F05.fastq,forward
F06,$PWD/se-33/F06.fastq,forward
F07,$PWD/se-33/F07.fastq,forward
F08,$PWD/se-33/F08.fastq,forward
F09,$PWD/se-33/F09.fastq,forward
F10,$PWD/se-33/F10.fastq,forward
F11,$PWD/se-33/F11.fastq,forward
F12,$PWD/se-33/F12.fastq,forward
F13,$PWD/se-33/F13.fastq,forward
F14,$PWD/se-33/F14.fastq,forward
F15,$PWD/se-33/F15.fastq,forward
F16,$PWD/se-33/F16.fastq,forward

I would appreciate any help,

Omar

2

Have you tried denoise-pyro? That command is pre-configured for pyrosequencing runs, and I think PGM/IT falls under that.

3

Hi thermokarst,
I tried and it gives the same error

qiime dada2 denoise-pyro \

--i-demultiplexed-seqs demux.qza
--p-trim-left 15
--p-trunc-len 250
--o-table table.qza
--o-representative-sequences rep-seqs.qza
--o-denoising-stats stats-dada2.qza

Plugin error from dada2:

An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

Debug info has been saved to /tmp/qiime2-q2cli-err-0_ub_96d.log

4

Can you please run qiime tools validate demux.qza?

5

I encountered exactly the same problem.

my code is as follows:

=========================================
source activate qiime2-2019.1

export TMPDIR=/mnt/lustre/user/xxx/tmpStool;
export HDF5_USE_FILE_LOCKING=FALSE;
export LANG=en_GB.utf8
echo -n "Start: ";date
qiime tools import
--type 'SampleData[SequencesWithQuality]'
--input-path manifest.csv
--output-path single-end-demux.qza
--input-format SingleEndFastqManifestPhred33

echo -n "start dada2 denoise-single: ";date
qiime dada2 denoise-single
--i-demultiplexed-seqs single-end-demux.qza
--p-trim-left 0
--p-trunc-len 350
--p-n-threads 60
--o-representative-sequences rep-seqs-dada2.qza
--o-table table-dada2.qza
--o-denoising-stats stats-dada2.qza

=========================================

and the error appeared:

Plugin error from dada2:

An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

Debug info has been saved to /tmp/qiime2-q2cli-err-7ljvldib.log

and in the file " /tmp/qiime2-q2cli-err-7ljvldib.log", there is :

R version 3.4.1 (2017-06-30)
Loading required package: Rcpp
DADA2 R package version: 1.6.0

  1. Filtering Error in filterAndTrim(unfilts, filts, truncLen = truncLen, trimLeft = trimLeft, :
    These are the errors (up to 5) encountered in individual cores...
    Error in writeFastq(fq, fout, "w", compress = compress) :
    failed to write record 125
    Error in writeFastq(fq, fout, "w", compress = compress) :
    failed to write record 125
    Error in writeFastq(fq, fout, "w", compress = compress) :
    failed to write record 125
    Error in writeFastq(fq, fout, "w", compress = compress) :
    failed to write record 114
    Error in writeFastq(fq, fout, "w", compress = compress) :
    failed to write record 114
    Execution halted
    Running external command line application(s). This may print messages to stdout and/or stderr.
    The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_single.R /tmp/qiime2-archive-a59pi7dd/62675334-10ce-4fb5-89f6-c0852967b566/data /tmp/tmpo5io0l32/output.tsv.biom /tmp/tmpo5io0l32/track.tsv /tmp/tmpo5io0l32 350 0 2.0 2 Inf consensus 1.0 60 1000000 NULL 16

Traceback (most recent call last):
File "/home/wubin/01.Program/02.software/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/q2_dada2/_denoise.py", line 152, in _denoise_single
run_commands([cmd])
File "/home/wubin/01.Program/02.software/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/q2_dada2/_denoise.py", line 36, in run_commands
subprocess.run(cmd, check=True)
File "/home/wubin/01.Program/02.software/miniconda3/envs/qiime2-2019.1/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['run_dada_single.R', '/tmp/qiime2-archive-a59pi7dd/62675334-10ce-4fb5-89f6-c0852967b566/data', '/tmp/tmpo5io0l32/output.tsv.biom', '/tmp/tmpo5io0l32/track.tsv', '/tmp/tmpo5io0l32', '350', '0', '2.0', '2', 'Inf', 'consensus', '1.0', '60', '1000000', 'NULL', '16']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/wubin/01.Program/02.software/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/q2cli/commands.py", line 274, in call
results = action(**arguments)
File "</home/wubin/01.Program/02.software/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/decorator.py:decorator-gen-138>", line 2, in denoise_single
File "/home/wubin/01.Program/02.software/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/qiime2/sdk/action.py", line 231, in bound_callable
output_types, provenance)
File "/home/wubin/01.Program/02.software/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/qiime2/sdk/action.py", line 365, in callable_executor
output_views = self._callable(**view_args)
File "/home/wubin/01.Program/02.software/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/q2_dada2/_denoise.py", line 187, in denoise_single
band_size='16')
File "/home/wubin/01.Program/02.software/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/q2_dada2/_denoise.py", line 163, in _denoise_single
" and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

and then, I check my input file: single-end-demux.qza

$ qiime tools validate single-end-demux.qza
Result single-end-demux.qza appears to be valid at level=max.

Then, what's the problem ?

6

Wow, that is a lot of threads!

Looks like you are running out of space, likely in your $TMPDIR. You will want to change that env var to another location with more disk space available (I suggest contacting your sys admin for guidance).

7

yes, maybe that’s the reason. when I set the TMPDIR = XXX, the XXX directory didn’t exists. So I need to mkdir XXX at first.

however, I don’t think -p-n-threads 60 caused the problem.

8

Yep!

Me either, I was just remarking on the magnitude of that hardware!

9

I had the same error running reads from IonTorrent. I solute this problem not using functions that filter qualities before entry dada2, because the quality scores from IonTorrent is special and can be a problem. For example, running TrimDNA function (DECIPHER R package) for remove short reads and above 20Qscore and after running this trimmed reads into dada2, was a problem. Try put your fastq directly on dada2 after cutadapt