Qiime2 dada2 error

Hi guys,

While I was running dada2 for paired-end data I got this error:
/////////////////////////////////////////////////////////////////////////////////////////
(qiime2-2018.8) qiime2@qiime2core2018-8:~$ qiime dada2 denoise-paired --i-demultiplexed-seqs /media/sf_gvhd/demux-paired-end.qza --o-table /media/sf_gvhd/table.qza --o-representative-sequences /media/sf_gvhd/rep-seqs.qza --p-trim-left-f 0 --p-trim-left-r 1 --p-trunc-len-f 230 --p-trunc-len-r 200 --o-denoising-stats /media/sf_gvhd/denoising-stats.qza --p-n-threads 40 --p-n-reads-learn 200000 --verbose
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_paired.R /tmp/tmp788pga9m/forward /tmp/tmp788pga9m/reverse /tmp/tmp788pga9m/output.tsv.biom /tmp/tmp788pga9m/track.tsv /tmp/tmp788pga9m/filt_f /tmp/tmp788pga9m/filt_r 230 200 0 1 2.0 2 consensus 1.0 40 200000

R version 3.4.1 (2017-06-30)
Loading required package: Rcpp
DADA2 R package version: 1.6.0

  1. Filtering ^[[AError in names(answer) <- names1 :
    ‘names’ attribute [115] must be the same length as the vector [98]
    Execution halted
    Traceback (most recent call last):
    File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/q2_dada2/_denoise.py”, line 229, in denoise_paired
    run_commands([cmd])
    File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/q2_dada2/_denoise.py”, line 36, in run_commands
    subprocess.run(cmd, check=True)
    File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/subprocess.py”, line 398, in run
    output=stdout, stderr=stderr)
    subprocess.CalledProcessError: Command ‘[‘run_dada_paired.R’, ‘/tmp/tmp788pga9m/forward’, ‘/tmp/tmp788pga9m/reverse’, ‘/tmp/tmp788pga9m/output.tsv.biom’, ‘/tmp/tmp788pga9m/track.tsv’, ‘/tmp/tmp788pga9m/filt_f’, ‘/tmp/tmp788pga9m/filt_r’, ‘230’, ‘200’, ‘0’, ‘1’, ‘2.0’, ‘2’, ‘consensus’, ‘1.0’, ‘40’, ‘200000’]’ returned non-zero exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/q2cli/commands.py”, line 274, in call
results = action(**arguments)
File “”, line 2, in denoise_paired
File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/qiime2/sdk/action.py”, line 231, in bound_callable
output_types, provenance)
File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/qiime2/sdk/action.py”, line 362, in callable_executor
output_views = self._callable(**view_args)
File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/q2_dada2/_denoise.py”, line 244, in denoise_paired
" and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

Plugin error from dada2:

An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

See above for debug info.
/////////////////////////////////////////////////////////////////////////////////////////

Is there anyone who knows what went wrong here?

Hi guys,

I have no idea what the problem is here…
I tried the same command after reinstalling qiime2 but I got the same error as below…
If anyone has any idea with this error… reply me please.
Thank you in advance for your help!

(qiime2-2018.8) qiime2@qiime2core2018-8:~$ qiime dada2 denoise-paired --i-demultiplexed-seqs /media/sf_gvhd/demux-paired-end.qza --o-table /media/sf_gvhd/table.qza --o-representative-sequences /media/sf_gvhd/rep-seqs.qza --p-trim-left-f 0 --p-trim-left-r 1 --p-trunc-len-f 230 --p-trunc-len-r 200 --o-denoising-stats /media/sf_gvhd/denoising-stats.qza --p-n-threads 40 --p-n-reads-learn 200000 --verbose
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_paired.R /tmp/tmpkfp2tjad/forward /tmp/tmpkfp2tjad/reverse /tmp/tmpkfp2tjad/output.tsv.biom /tmp/tmpkfp2tjad/track.tsv /tmp/tmpkfp2tjad/filt_f /tmp/tmpkfp2tjad/filt_r 230 200 0 1 2.0 2 consensus 1.0 40 200000

R version 3.4.1 (2017-06-30)
Loading required package: Rcpp
DADA2 R package version: 1.6.0

  1. Filtering Error in names(answer) <- names1 :
    ‘names’ attribute [115] must be the same length as the vector [97]
    Execution halted
    Traceback (most recent call last):
    File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/q2_dada2/_denoise.py”, line 229, in denoise_paired
    run_commands([cmd])
    File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/q2_dada2/_denoise.py”, line 36, in run_commands
    subprocess.run(cmd, check=True)
    File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/subprocess.py”, line 398, in run
    output=stdout, stderr=stderr)
    subprocess.CalledProcessError: Command ‘[‘run_dada_paired.R’, ‘/tmp/tmpkfp2tjad/forward’, ‘/tmp/tmpkfp2tjad/reverse’, ‘/tmp/tmpkfp2tjad/output.tsv.biom’, ‘/tmp/tmpkfp2tjad/track.tsv’, ‘/tmp/tmpkfp2tjad/filt_f’, ‘/tmp/tmpkfp2tjad/filt_r’, ‘230’, ‘200’, ‘0’, ‘1’, ‘2.0’, ‘2’, ‘consensus’, ‘1.0’, ‘40’, ‘200000’]’ returned non-zero exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/q2cli/commands.py”, line 274, in call
results = action(**arguments)
File “”, line 2, in denoise_paired
File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/qiime2/sdk/action.py”, line 231, in bound_callable
output_types, provenance)
File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/qiime2/sdk/action.py”, line 362, in callable_executor
output_views = self._callable(**view_args)
File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/q2_dada2/_denoise.py”, line 244, in denoise_paired
" and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

Plugin error from dada2:

An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

See above for debug info.

Hey @Jeongsu_Kim sorry for the wait,

I have not seen this error before :frowning:

It seems as though you might have incomplete fastq records in your demux-paired-end.qza?

Does qiime tools validate /media/sf_gvhd/demux-paired-end.qza tell you anything?

qiime tools validate says: Artifact demux-paired-end.qza appears to be valid at level=max.

Does this mean the qza file is okay?

And when you say “incomplete fastq records”, what does it mean?

Thanks for your help!

Shoot :frowning: I don't think that tool has ever given us a true negative result. We should figure out what's not working.

I still doubt that the qza file is correct, so we'll push a little further.

I mean something like this:

@someheader
ACGTATAGCT
+
<oops this is the end of the file, we're missing quality information,
 so the record is incomplete>

Let's have you run qiime tools export on demux.qza, then could you cd into the resulting directory and run this command:

find . -name '*.fastq.gz' -exec sh -c "echo -n '{}: ' && zcat {} | wc -l" \; | sort

That should list every file and give us a line count of the uncompressed fastq file.

This is the list of files.

(qiime2-2018.8) qiime2@qiime2core2018-8:/media/sf_gvhd$ qiime tools export --input-path demux-paired-end2.qza --output-path export
Exported demux-paired-end2.qza as SingleLanePerSamplePairedEndFastqDirFmt to directory export

(qiime2-2018.8) qiime2@qiime2core2018-8:/media/sf_gvhd/export$ find . -name ‘*.fastq.gz’ -exec sh -c “echo -n '{}: ’ && zcat {} | wc -l” ; | sort
./A100_S60_L001_R1_001.fastq.gz: 622496
./A100_S60_L001_R2_001.fastq.gz: 622496
./A105_S7_L001_R1_001.fastq.gz: 717148
./A105_S7_L001_R2_001.fastq.gz: 717148
./A107_S57_L001_R1_001.fastq.gz: 644996
./A107_S57_L001_R2_001.fastq.gz: 644996
./A10_S3_L001_R1_001.fastq.gz: 322296
./A10_S3_L001_R2_001.fastq.gz: 322296
./A114_S58_L001_R1_001.fastq.gz: 431584
./A114_S58_L001_R2_001.fastq.gz: 431584
./A116_S9_L001_R1_001.fastq.gz: 846796
./A116_S9_L001_R2_001.fastq.gz: 846796
./A119_S59_L001_R1_001.fastq.gz: 645496
./A119_S59_L001_R2_001.fastq.gz: 645496
./A11_S1_L001_R1_001.fastq.gz: 369732
./A11_S1_L001_R2_001.fastq.gz: 369732
./A123_S12_L001_R1_001.fastq.gz: 842688
./A123_S12_L001_R2_001.fastq.gz: 842688
./A12_S72_L001_R1_001.fastq.gz: 414124
./A12_S72_L001_R2_001.fastq.gz: 414124
./A14_S75_L001_R1_001.fastq.gz: 167108
./A14_S75_L001_R2_001.fastq.gz: 167108
./A15_S61_L001_R1_001.fastq.gz: 187756
./A15_S61_L001_R2_001.fastq.gz: 187756
./A18_S7_L001_R1_001.fastq.gz: 342444
./A18_S7_L001_R2_001.fastq.gz: 342444
./A1_S68_L001_R1_001.fastq.gz: 796308
./A1_S68_L001_R2_001.fastq.gz: 796308
./A204_S15_L001_R1_001.fastq.gz: 637952
./A204_S15_L001_R2_001.fastq.gz: 637952
./A206_S11_L001_R1_001.fastq.gz: 735276
./A206_S11_L001_R2_001.fastq.gz: 735276
./A22_S63_L001_R1_001.fastq.gz: 260236
./A22_S63_L001_R2_001.fastq.gz: 260236
./A23_S78_L001_R1_001.fastq.gz: 271652
./A23_S78_L001_R2_001.fastq.gz: 271652
./A24_S59_L001_R1_001.fastq.gz: 270380
./A24_S59_L001_R2_001.fastq.gz: 270380
./A25_S8_L001_R1_001.fastq.gz: 303044
./A25_S8_L001_R2_001.fastq.gz: 303044
./A26_S40_L001_R1_001.fastq.gz: 216712
./A26_S40_L001_R2_001.fastq.gz: 216712
./A27_S73_L001_R1_001.fastq.gz: 407996
./A27_S73_L001_R2_001.fastq.gz: 407996
./A28_S60_L001_R1_001.fastq.gz: 214144
./A28_S60_L001_R2_001.fastq.gz: 214144
./A29_S11_L001_R1_001.fastq.gz: 235948
./A29_S11_L001_R2_001.fastq.gz: 235948
./A30_S77_L001_R1_001.fastq.gz: 223168
./A30_S77_L001_R2_001.fastq.gz: 223168
./A33_S13_L001_R1_001.fastq.gz: 651760
./A33_S13_L001_R2_001.fastq.gz: 651760
./A34_S39_L001_R1_001.fastq.gz: 212360
./A34_S39_L001_R2_001.fastq.gz: 212360
./A35_S10_L001_R1_001.fastq.gz: 252732
./A35_S10_L001_R2_001.fastq.gz: 252732
./A36_S6_L001_R1_001.fastq.gz: 427596
./A36_S6_L001_R2_001.fastq.gz: 427596
./A37_S12_L001_R1_001.fastq.gz: 343440
./A37_S12_L001_R2_001.fastq.gz: 343440
./A3_S70_L001_R1_001.fastq.gz: 650160
./A3_S70_L001_R2_001.fastq.gz: 650160
./A40_S65_L001_R1_001.fastq.gz: 178680
./A40_S65_L001_R2_001.fastq.gz: 178680
./A42_S79_L001_R1_001.fastq.gz: 435260
./A42_S79_L001_R2_001.fastq.gz: 435260
./A45_S2_L001_R1_001.fastq.gz: 352288
./A45_S2_L001_R2_001.fastq.gz: 352288
./A48_S76_L001_R1_001.fastq.gz: 175836
./A48_S76_L001_R2_001.fastq.gz: 175836
./A49_S69_L001_R1_001.fastq.gz: 534712
./A49_S69_L001_R2_001.fastq.gz: 534712
./A50_S67_L001_R1_001.fastq.gz: 210640
./A50_S67_L001_R2_001.fastq.gz: 210640
./A51_S74_L001_R1_001.fastq.gz: 356732
./A51_S74_L001_R2_001.fastq.gz: 356732
./A54_S81_L001_R1_001.fastq.gz: 220748
./A54_S81_L001_R2_001.fastq.gz: 220748
./A55_S71_L001_R1_001.fastq.gz: 308804
./A55_S71_L001_R2_001.fastq.gz: 308804
./A56f_S17_L001_R1_001.fastq.gz: 234804
./A56f_S17_L001_R2_001.fastq.gz: 234804
./A57f_S18_L001_R1_001.fastq.gz: 226552
./A57f_S18_L001_R2_001.fastq.gz: 226552
./A58f_S19_L001_R1_001.fastq.gz: 200152
./A58f_S19_L001_R2_001.fastq.gz: 200152
./A71_S13_L001_R1_001.fastq.gz: 874712
./A71_S13_L001_R2_001.fastq.gz: 874712
./A7_S5_L001_R1_001.fastq.gz: 393420
./A7_S5_L001_R2_001.fastq.gz: 393420
./A81_S8_L001_R1_001.fastq.gz: 438004
./A81_S8_L001_R2_001.fastq.gz: 438004
./A90_S10_L001_R1_001.fastq.gz: 684408
./A90_S10_L001_R2_001.fastq.gz: 684408
./A93_S14_L001_R1_001.fastq.gz: 607884
./A93_S14_L001_R2_001.fastq.gz: 607884
./A99_S58_L001_R1_001.fastq.gz: 559436
./A99_S58_L001_R2_001.fastq.gz: 559436
./A9_S62_L001_R1_001.fastq.gz: 189636
./A9_S62_L001_R2_001.fastq.gz: 189636
./B126_S2_L001_R1_001.fastq.gz: 661548
./B126_S2_L001_R2_001.fastq.gz: 661548
./B129_S22_L001_R1_001.fastq.gz: 722748
./B129_S22_L001_R2_001.fastq.gz: 722748
./B130_S29_L001_R1_001.fastq.gz: 503004
./B130_S29_L001_R2_001.fastq.gz: 503004
./B132_S24_L001_R1_001.fastq.gz: 722524
./B132_S24_L001_R2_001.fastq.gz: 722524
./B133_S32_L001_R1_001.fastq.gz: 617228
./B133_S32_L001_R2_001.fastq.gz: 617228
./B137_S5_L001_R1_001.fastq.gz: 776024
./B137_S5_L001_R2_001.fastq.gz: 776024
./B138_S33_L001_R1_001.fastq.gz: 666756
./B138_S33_L001_R2_001.fastq.gz: 666756
./B140_S19_L001_R1_001.fastq.gz: 615408
./B140_S19_L001_R2_001.fastq.gz: 615408
./B141_S59_L001_R1_001.fastq.gz: 1223128
./B141_S59_L001_R2_001.fastq.gz: 1223128
./B142_S44_L001_R1_001.fastq.gz: 368980
./B142_S44_L001_R2_001.fastq.gz: 368980
./B145_S46_L001_R1_001.fastq.gz: 448096
./B145_S46_L001_R2_001.fastq.gz: 448096
./B147_S60_L001_R1_001.fastq.gz: 1462980
./B147_S60_L001_R2_001.fastq.gz: 1462980
./B148_S31_L001_R1_001.fastq.gz: 530756
./B148_S31_L001_R2_001.fastq.gz: 530756
./B14_S30_L001_R1_001.fastq.gz: 493484
./B14_S30_L001_R2_001.fastq.gz: 493484
./B152_S51_L001_R1_001.fastq.gz: 923812
./B152_S51_L001_R2_001.fastq.gz: 923812
./B155_S4_L001_R1_001.fastq.gz: 762156
./B155_S4_L001_R2_001.fastq.gz: 762156
./B157_S21_L001_R1_001.fastq.gz: 855040
./B157_S21_L001_R2_001.fastq.gz: 855040
./B158_S25_L001_R1_001.fastq.gz: 591568
./B158_S25_L001_R2_001.fastq.gz: 591568
./B159_S3_L001_R1_001.fastq.gz: 641988
./B159_S3_L001_R2_001.fastq.gz: 641988
./B15_S80_L001_R1_001.fastq.gz: 198600
./B15_S80_L001_R2_001.fastq.gz: 198600
./B160_S20_L001_R1_001.fastq.gz: 754004
./B160_S20_L001_R2_001.fastq.gz: 754004
./B162_S23_L001_R1_001.fastq.gz: 773968
./B162_S23_L001_R2_001.fastq.gz: 773968
./B163_S26_L001_R1_001.fastq.gz: 587652
./B163_S26_L001_R2_001.fastq.gz: 587652
./B164_S1_L001_R1_001.fastq.gz: 691272
./B164_S1_L001_R2_001.fastq.gz: 691272
./B165_S30_L001_R1_001.fastq.gz: 589956
./B165_S30_L001_R2_001.fastq.gz: 589956
./B167_S27_L001_R1_001.fastq.gz: 504820
./B167_S27_L001_R2_001.fastq.gz: 504820
./B168_S6_L001_R1_001.fastq.gz: 755340
./B168_S6_L001_R2_001.fastq.gz: 755340
./B171_S34_L001_R1_001.fastq.gz: 750548
./B171_S34_L001_R2_001.fastq.gz: 750548
./B172_S28_L001_R1_001.fastq.gz: 480460
./B172_S28_L001_R2_001.fastq.gz: 480460
./B17_S32_L001_R1_001.fastq.gz: 569868
./B17_S32_L001_R2_001.fastq.gz: 569868
./B183_S17_L001_R1_001.fastq.gz: 628360
./B183_S17_L001_R2_001.fastq.gz: 628360
./B184_S45_L001_R1_001.fastq.gz: 334912
./B184_S45_L001_R2_001.fastq.gz: 334912
./B185_S48_L001_R1_001.fastq.gz: 272608
./B185_S48_L001_R2_001.fastq.gz: 272608
./B186_S49_L001_R1_001.fastq.gz: 396356
./B186_S49_L001_R2_001.fastq.gz: 396356
./B187_S50_L001_R1_001.fastq.gz: 986208
./B187_S50_L001_R2_001.fastq.gz: 986208
./B188_S36_L001_R1_001.fastq.gz: 356196
./B188_S36_L001_R2_001.fastq.gz: 356196
./B18_S17_L001_R1_001.fastq.gz: 278716
./B18_S17_L001_R2_001.fastq.gz: 278716
./B190_S35_L001_R1_001.fastq.gz: 385000
./B190_S35_L001_R2_001.fastq.gz: 385000
./B196_S38_L001_R1_001.fastq.gz: 433164
./B196_S38_L001_R2_001.fastq.gz: 433164
./B199_S39_L001_R1_001.fastq.gz: 342456
./B199_S39_L001_R2_001.fastq.gz: 342456
./B19_S34_L001_R1_001.fastq.gz: 247060
./B19_S34_L001_R2_001.fastq.gz: 247060
./B203_S42_L001_R1_001.fastq.gz: 335940
./B203_S42_L001_R2_001.fastq.gz: 335940
./B22_S38_L001_R1_001.fastq.gz: 289800
./B22_S38_L001_R2_001.fastq.gz: 289800
./B23_S27_L001_R1_001.fastq.gz: 526940
./B23_S27_L001_R2_001.fastq.gz: 526940
./B24_S28_L001_R1_001.fastq.gz: 445220
./B24_S28_L001_R2_001.fastq.gz: 445220
./B25_S18_L001_R1_001.fastq.gz: 283272
./B25_S18_L001_R2_001.fastq.gz: 283272
./B26_S37_L001_R1_001.fastq.gz: 234188
./B26_S37_L001_R2_001.fastq.gz: 234188
./B27_S26_L001_R1_001.fastq.gz: 848124
./B27_S26_L001_R2_001.fastq.gz: 848124
./B28_S14_L001_R1_001.fastq.gz: 589492
./B28_S14_L001_R2_001.fastq.gz: 589492
./B29_S21_L001_R1_001.fastq.gz: 414264
./B29_S21_L001_R2_001.fastq.gz: 414264
./B30_S31_L001_R1_001.fastq.gz: 521160
./B30_S31_L001_R2_001.fastq.gz: 521160
./B33_S36_L001_R1_001.fastq.gz: 255764
./B33_S36_L001_R2_001.fastq.gz: 255764
./B34_S64_L001_R1_001.fastq.gz: 170816
./B34_S64_L001_R2_001.fastq.gz: 170816
./B35_S35_L001_R1_001.fastq.gz: 232500
./B35_S35_L001_R2_001.fastq.gz: 232500
./B37_S33_L001_R1_001.fastq.gz: 258188
./B37_S33_L001_R2_001.fastq.gz: 258188
./B40_S15_L001_R1_001.fastq.gz: 665424
./B40_S15_L001_R2_001.fastq.gz: 665424
./B42_S24_L001_R1_001.fastq.gz: 236820
./B42_S24_L001_R2_001.fastq.gz: 236820
./B48_S16_L001_R1_001.fastq.gz: 732828
./B48_S16_L001_R2_001.fastq.gz: 732828
./B51_S20_L001_R1_001.fastq.gz: 203552
./B51_S20_L001_R2_001.fastq.gz: 203552
./B64_S47_L001_R1_001.fastq.gz: 211728
./B64_S47_L001_R2_001.fastq.gz: 211728
./B65_S40_L001_R1_001.fastq.gz: 354264
./B65_S40_L001_R2_001.fastq.gz: 354264
./B75_S37_L001_R1_001.fastq.gz: 221180
./B75_S37_L001_R2_001.fastq.gz: 221180
./B88_S41_L001_R1_001.fastq.gz: 394744
./B88_S41_L001_R2_001.fastq.gz: 394744
./B93_S18_L001_R1_001.fastq.gz: 644948
./B93_S18_L001_R2_001.fastq.gz: 644948
./B96_S16_L001_R1_001.fastq.gz: 643896
./B96_S16_L001_R2_001.fastq.gz: 643896

Hi @Jeongsu_Kim,

I think I found the same issue elsewhere:

It seems to be a case of an out-of-memory error happening while trimming in a multi-threaded environment.

I would change this parameter:

To something smaller OR increase the amount of memory available to you (if that's possible).

More specifically, there's not a lot of use in setting --p-n-threads higher than the number of cores available within a node, so that number should probably be something more like 4 or 8 since it looks like you are on a virtual-machine (make sure it has enough CPU's allocated from the host machine).

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.