An error occured while running DADA2 in R

[Fra (Francesco)]

Hello everyone,

I would like to ask your help on this error I get when running DADA2.

When I run the following command:
qiime dada2 denoise-paired
–i-demultiplexed-seqs 16S_trimmed.qza
–p-trunc-len-f 180 --p-trunc-len-r 120
–p-trim-left-f 0
–p-trim-left-r 5
–p-trunc-q 20
–p-n-threads 24
–p-chimera-method “pooled”
–output-dir dada2.1
–verbose

I get the following error: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_paired.R /tmp/tmpoqz0fanr/forward /tmp/tmpoqz0fanr/reverse /tmp/tmpoqz0fanr/output.tsv.biom /tmp/tmpoqz0fanr/track.tsv /tmp/tmpoqz0fanr/filt_f /tmp/tmpoqz0fanr/filt_r 180 120 0 5 2.0 2.0 20 independent pooled 1.0 24 1000000

R version 3.5.1 (2018-07-02)
Loading required package: Rcpp
DADA2: 1.10.0 / Rcpp: 1.0.4.6 / RcppParallel: 5.0.0

1. Filtering Error in sendMaster(try(lapply(X = S, FUN = FUN, …), silent = TRUE)) :
   write error, closing pipe to the master
   Error in names(answer) <- names1 :
   ‘names’ attribute [224] must be the same length as the vector [215]
   Execution halted
   Traceback (most recent call last):
   File “/home/ubuntu/francesco/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 264, in denoise_paired
   run_commands([cmd])
   File “/home/ubuntu/francesco/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 36, in run_commands
   subprocess.run(cmd, check=True)
   File “/home/ubuntu/francesco/miniconda3/envs/qiime2-2020.6/lib/python3.6/subprocess.py”, line 438, in run
   output=stdout, stderr=stderr)
   subprocess.CalledProcessError: Command ‘[‘run_dada_paired.R’, ‘/tmp/tmpoqz0fanr/forward’, ‘/tmp/tmpoqz0fanr/reverse’, ‘/tmp/tmpoqz0fanr/output.tsv.biom’, ‘/tmp/tmpoqz0fanr/track.tsv’, ‘/tmp/tmpoqz0fanr/filt_f’, ‘/tmp/tmpoqz0fanr/filt_r’, ‘180’, ‘120’, ‘0’, ‘5’, ‘2.0’, ‘2.0’, ‘20’, ‘independent’, ‘pooled’, ‘1.0’, ‘24’, ‘1000000’]’ returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/home/ubuntu/francesco/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/q2cli/commands.py”, line 329, in call
results = action(**arguments)
File “”, line 2, in denoise_paired
File “/home/ubuntu/francesco/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 245, in bound_callable
output_types, provenance)
File “/home/ubuntu/francesco/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 390, in callable_executor
output_views = self._callable(**view_args)
File “/home/ubuntu/francesco/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 279, in denoise_paired
" and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

[llenzi (Luca)]

Hi Francesco,

Which 16S region are you working on? And which full amplicon length would you expect for this?
What length are your sequences? (2x250bp, 2x300bp )
Which sequencing platform did you use?

Your settings seems quite stringent to me, I doubt you have any merging region left after the trimming. I would also remove the ‘-p-trunc-q’ option, because you probably are loosing lots of bases with this (I mean, the job of a denoiser is to work hard to improve base quality, isn’t it? )

Cheers,
Luca

[Fra (Francesco)]

Hi Luca,

I’ve tried with less stringent parameters, but still nothing. SPecifically --p-trunc-q 0 and --p-trim-len-f 200 and --p-trim-len-r 140.

The sequencing platform is MiSeq 2x300bp.

Cheers

[llenzi (Luca)]

Hi Francesco,

Please keep in mind that you should allow for at least 12 bases of overlap between forward and reverse reads, after the primer trimming and considering the --p-trim-len option.

I think your reverse trimming length is still too stringent.
How long is the expected amplicon?

Could you share your demultiplexed.qzv file (after cutadapt step)? If you prefer, feel free to PM me to keep it private.

Cheers
Luca

[Fra (Francesco)]

Yes I am aware of the overlapping. My 16S region is 270bp and there is plenty of overlapping between F and R reads.

Cheers
Francesco

[Fra (Francesco)]

Btw I haven’t figured out the problem yet, it would be great if someone else from the qiime2 team could help me out with this.

Thanks

[Fra (Francesco)]

[Fra (Francesco)]

This is the graph of the trimmed reads.

[Fra (Francesco)]

This is the command I am running at the moment:

qiime dada2 denoise-paired --i-demultiplexed-seqs 16S_trimmed_run1_v2.qza --p-trunc-len-f 190 --p-trunc-len-r 160 --p-trim-left-f 0 --p-trim-left-r 5 --p-trunc-q 0 --p-n-threads 10 --p-chimera-method “pooled” --output-dir dada2.1 --verbose

and this is the error:

Command: run_dada_paired.R /tmp/tmp9lzgg7m9/forward /tmp/tmp9lzgg7m9/reverse /tmp/tmp9lzgg7m9/output.tsv.biom /tmp/tmp9lzgg7m9/track.tsv /tmp/tmp9lzgg7m9/filt_f /tmp/tmp9lzgg7m9/filt_r 190 160 0 5 2.0 2.0 0 independent pooled 1.0 10 1000000

R version 3.5.1 (2018-07-02)
Loading required package: Rcpp
DADA2: 1.10.0 / Rcpp: 1.0.4.6 / RcppParallel: 5.0.0

1. Filtering Error in filterAndTrim(unfiltsF, filtsF, unfiltsR, filtsR, truncLen = c(truncLenF, :
   These are the errors (up to 5) encountered in individual cores…
   Error in writeFastq(fqF, fout[[1]], “w”, compress = compress) :
   failed to write record 526
   Error in writeFastq(fqF, fout[[1]], “w”, compress = compress) :
   failed to write record 381
   Error in writeFastq(fqF, fout[[1]], “w”, compress = compress) :
   failed to write record 399
   Error in writeFastq(fqF, fout[[1]], “w”, compress = compress) :
   failed to write record 526
   Error in writeFastq(fqR, fout[[2]], “w”, compress = compress) :
   failed to write record 68757
   Execution halted
   Traceback (most recent call last):
   File “/home/ubuntu/francesco/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 264, in denoise_paired
   run_commands([cmd])
   File “/home/ubuntu/francesco/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 36, in run_commands
   subprocess.run(cmd, check=True)
   File “/home/ubuntu/francesco/miniconda3/envs/qiime2-2020.6/lib/python3.6/subprocess.py”, line 438, in run
   output=stdout, stderr=stderr)
   subprocess.CalledProcessError: Command ‘[‘run_dada_paired.R’, ‘/tmp/tmp9lzgg7m9/forward’, ‘/tmp/tmp9lzgg7m9/reverse’, ‘/tmp/tmp9lzgg7m9/output.tsv.biom’, ‘/tmp/tmp9lzgg7m9/track.tsv’, ‘/tmp/tmp9lzgg7m9/filt_f’, ‘/tmp/tmp9lzgg7m9/filt_r’, ‘190’, ‘160’, ‘0’, ‘5’, ‘2.0’, ‘2.0’, ‘0’, ‘independent’, ‘pooled’, ‘1.0’, ‘10’, ‘1000000’]’ returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/home/ubuntu/francesco/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/q2cli/commands.py”, line 329, in call
results = action(**arguments)
File “”, line 2, in denoise_paired
File “/home/ubuntu/francesco/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 245, in bound_callable
output_types, provenance)
File “/home/ubuntu/francesco/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 390, in callable_executor
output_views = self._callable(**view_args)
File “/home/ubuntu/francesco/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 279, in denoise_paired
" and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

[llenzi (Luca)]

Hi @Fra,

I agree, your sequences should have plenty of overlap you may eve try with R1 alone in your case!

The last error looks like to be different from the initial one!

These are the errors (up to 5) encountered in individual cores…
Error in writeFastq(fqF, fout[[1]], “w”, compress = compress) :
failed to write record 526
Error in writeFastq(fqF, fout[[1]], “w”, compress = compress) :
failed to write record 381
Error in writeFastq(fqF, fout[[1]], “w”, compress = compress) :
failed to write record 399
Error in writeFastq(fqF, fout[[1]], “w”, compress = compress) :
failed to write record 526
Error in writeFastq(fqR, fout[[2]], “w”, compress = compress) :
failed to write record 68757
Execution halted

Do you run the code in a virtual machine? Are you running out of space ?

Cheers
Luca

[Fra (Francesco)]

Does anybody in the forum has a clue about what’s going on with this error? Maybe @Nicholas_Bokulich?

Thanks
Francesco

[Fra (Francesco)]

Hi ran the code on an older qiime2 version and it worked fine. Perhaps there’s a bug of some sort in dada2 in the last released qiime2 version?

[llenzi (Luca)]

Hi @Francesco,

It seems I was not quite on the right track and thanks to @Nicholas_Bokulich for the support!

The

Blockquote ‘names’ attribute [224] must be the same length as the vector [215]

In the initial error may be also the effect of a memory error, as discussed in the following thread

Denoising error, names attribute must be the same length as vector

Hi @Roger_Huerlimann, Congratulations for running into one of the more cryptic error messages that dada2 can throw at you. This is effectively a memory error even though it does not look like one — or more specifically a multithreading issue that can be fixed by reducing the number of threads used. See here for some discussion: I think the dada2 version was upgraded somewhere in there, so it could be a version issue, but it could also be coincidence. Reduce the number of threads you are …

Could you test the command using less core processors?
Hope will do the trick!

Luca

[llenzi (Luca)]

Hi @Fra,

A quick follow up on the error in the command command, did you check the available space on your machine?

Your error seems the same as in:

Return code 1 running DADA2 with single-end PGM/Ion torrent data

yes, maybe that’s the reason. when I set the TMPDIR = XXX, the XXX directory didn’t exists. So I need to mkdir XXX at first. however, I don’t think -p-n-threads 60 caused the problem.

(The fact that this thread is related to ion-torrent data should not be a problem, the error is not linked to the type of data in this case!)

Luca