An error was encountered while running DADA2 in R

Hi
I have imported some paired-end reads(generated by a simulator) to qiime (version qiime2-2019.1). The simulator produced one fastq file (interleaved) for each sample. To import I have used extract_reads_from_interleaved_file.py. Then Imported to qiime using following command
qiime tools import
–type ‘SampleData[PairedEndSequencesWithQuality]’
–input-path fastq_manifest
–output-path paired-end-demux.qza
–input-format PairedEndFastqManifestPhred33
After visualize the demux.qzv I want to use following command to denoise

qiime dada2 denoise-paired --i-demultiplexed-seqs paired-end-demux.qza --output-dir dada2/ --o-representative-sequences rep-seqs --p-trim-left-f 20 --p-trim-left-r 20 --p-trunc-len-f 150 --p-trunc-len-r 140 --p-n-threads 4

Then I am getting this error
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_paired.R /tmp/tmpd2xagkli/forward /tmp/tmpd2xagkli/reverse /tmp/tmpd2xagkli/output.tsv.biom /tmp/tmpd2xagkli/track.tsv /tmp/tmpd2xagkli/filt_f /tmp/tmpd2xagkli/filt_r 150 140 20 20 2.0 2 consensus 1.0 4 1000000

R version 3.4.1 (2017-06-30)
Loading required package: Rcpp
DADA2 R package version: 1.6.0
1) Filtering ..........
2) Learning Error Rates
2a) Forward Reads
Initializing error rates to maximum possible estimate.
Sample 1 - 159 reads in 159 unique sequences.
Sample 2 - 161 reads in 161 unique sequences.
Sample 3 - 165 reads in 165 unique sequences.
Sample 4 - 168 reads in 168 unique sequences.
Sample 5 - 157 reads in 157 unique sequences.
Sample 6 - 158 reads in 158 unique sequences.
Sample 7 - 160 reads in 160 unique sequences.
Sample 8 - 178 reads in 178 unique sequences.
Sample 9 - 163 reads in 163 unique sequences.
Sample 10 - 145 reads in 145 unique sequences.
   selfConsist step 2
Convergence after  2  rounds.
2b) Reverse Reads
Initializing error rates to maximum possible estimate.
Sample 1 - 159 reads in 159 unique sequences.
Sample 2 - 161 reads in 161 unique sequences.
Sample 3 - 165 reads in 165 unique sequences.
Sample 4 - 168 reads in 168 unique sequences.
Sample 5 - 157 reads in 157 unique sequences.
Sample 6 - 158 reads in 158 unique sequences.
Sample 7 - 160 reads in 160 unique sequences.
Sample 8 - 178 reads in 178 unique sequences.
Sample 9 - 163 reads in 163 unique sequences.
Sample 10 - 145 reads in 145 unique sequences.
   selfConsist step 2
Convergence after  2  rounds.

3) Denoise remaining samples
4) Remove chimeras (method = consensus)
Error in isBimeraDenovoTable(unqs[[i]], ..., verbose = verbose) :
  Input must be a valid sequence table.
Calls: removeBimeraDenovo -> isBimeraDenovoTable
In addition: Warning message:
In is.na(colnames(unqs[[i]])) :
  is.na() applied to non-(list or vector) of type 'NULL'
Execution halted
Traceback (most recent call last):
  File "/home/ashutosh/anaconda2/envs/qiime2-2019.1/lib/python3.6/site-packages/q2_dada2/_denoise.py", line 231, in denoise_paired
    run_commands([cmd])
  File "/home/ashutosh/anaconda2/envs/qiime2-2019.1/lib/python3.6/site-packages/q2_dada2/_denoise.py", line 36, in run_commands
    subprocess.run(cmd, check=True)
  File "/home/ashutosh/anaconda2/envs/qiime2-2019.1/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['run_dada_paired.R', '/tmp/tmpd2xagkli/forward', '/tmp/tmpd2xagkli/reverse', '/tmp/tmpd2xagkli/output.tsv.biom', '/tmp/tmpd2xagkli/track.tsv', '/tmp/tmpd2xagkli/filt_f', '/tmp/tmpd2xagkli/filt_r', '150', '140', '20', '20', '2.0', '2', 'consensus', '1.0', '4', '1000000']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ashutosh/anaconda2/envs/qiime2-2019.1/lib/python3.6/site-packages/q2cli/commands.py", line 274, in __call__
    results = action(**arguments)
  File "</home/ashutosh/anaconda2/envs/qiime2-2019.1/lib/python3.6/site-packages/decorator.py:decorator-gen-442>", line 2, in denoise_paired
  File "/home/ashutosh/anaconda2/envs/qiime2-2019.1/lib/python3.6/site-packages/qiime2/sdk/action.py", line 231, in bound_callable
    output_types, provenance)
  File "/home/ashutosh/anaconda2/envs/qiime2-2019.1/lib/python3.6/site-packages/qiime2/sdk/action.py", line 365, in _callable_executor_
    output_views = self._callable(**view_args)
  File "/home/ashutosh/anaconda2/envs/qiime2-2019.1/lib/python3.6/site-packages/q2_dada2/_denoise.py", line 246, in denoise_paired
    " and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

Can you suggest what wrong with my files or command?

Thanks in advance

Hi @ashutosh!

Yep, the error above:

indicates that your forward and reverse reads are not joining. Options include adjusting the trim/trunc params, or proceeding with just the forward or reverse reads on their own.

1 Like

I’m having the same error. I guess this page will answer my question.

Hi @thermokarst
I need a suggestion on DADA2 run using qiime2-2019.4
I have demultiplexed 889 SingleEnd Read file downloaded from SRA database (se_demux file is attached).
Then I used DADA2 to denoise the reads using following commands
qiime dada2 denoise-single --i-demultiplexed-seqs se_demux.qza --output-dir dada2/ --p-trunc-len 540 --p-trim-left 20 --p-n-threads 16
After that, I have checked the se_denoising_stats.qzv file (attached here) and it showed that I didn't get any sequence denoised except for 4 samples.
Can you suggest, what is wrong with my data

Thanks in advance se_demux.qzv (306.7 KB)
se_denoising_stats.qzv (1.2 MB)

Ashutosh

An off-topic reply has been split into a new topic: dada2 error: filtered all reads

Please keep replies on-topic in the future.

Hi @ashutosh,
What exact is the nature of these samples? What is the sequencing system used? Are these samples all from the same run? have they been quality controlled or pre-merged prior?
DADA2 expects the reads to come from the same run, see FMT tutorial for an example of merging separate runs, which is what you’ll probably have to do. These should also not be merged and note that the denoise-single is optimized for Illumina runs, see denoise-pyro if these are 454.
There’s probably a lot of other moving parts as to why you are getting zero filtered reads but lets start with these before digging any deeper.

Hi @Mehrbod_Estaki
Thanks for your reply,
I ran dada2 separately for different runs/experiment, most of the dada2 runs worked except one with 289 samples from a single bioproject data, It seems demux file is ok
I checked the demux file using
qiime tools validate data5_demux.qza
Result data5_demux.qza appears to be valid at level=max.

(qiime2-2019.4) ashutosh@fisher:~/data/analysis/data_5$ qiime dada2 denoise-single --i-demultiplexed-seqs data5_demux.qza  --p-trunc-len 0 --p-trim-left 0 --output-dir dada2/ --p-n-threads 16
Plugin error from dada2:

  An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

Debug info has been saved to /tmp/qiime2-q2cli-err-u0jnlsow.log

The check my fastq files , they don’t have any thins in quality lines, as follows

@SRX2802320.1.2 1 length=335
            CCGGATTAGATACCCTGGTAGTCCACGCTGTAAACGATGTCGATTTGGAGTTTGTTGCCTAGAGTGATGGGCTCCGAAGCTAACGCGTGAAATTGTCCGCCTGGGGAGTACGGTCGCAAGATTAAAACTCAAAGGAATTGACGGGGGCCCGCACAAGCGGTGGAGCATGTGGTTTAATTCGAAGCAACGCGCAGAACCTTACCAGGTCTTGACATGTAGACGCTGTATTCAGAGATGAATGCTTCCAGCAATGGACGTCTAACACAGGTGCTGCATGGCTGTCGTCAGCTCGTGTCGTGAGATGTTGGGTTAAGTCCCGCAACGAGCGCAACCCT
            +SRX2802320.1.2 1 length=335
            ???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
            @SRX2802320.2.2 2 length=333
            CCGGATTAGATACCCTGGTAGTCCACGCCGTAAACGATGTCTACTTGGAGGCTGTGGTCATGAACCGTGGCTTCCGGAGCTAACGCGTTAAGTAGACCGCCTGGGGAGTACGGCCGCAAGGTTAAAACTCAAATGAATTGACGGGGGCCCGCACAAGCGGTGGAGCATGTGGTTTAATTCGATGCAACGCGAAGAACCTTACCTGGTCTTGACATCCACAGAATTCTCCGGAGACGGAGAAGTGCCTTCGGGAACTGTGAGACAGGTGCTGCATGGCTGTCGTCAGCTCGTGTTGTGAAATGTTGGGTTAAGTCCCGCAACGAGCGCAACCCT
            +SRX2802320.2.2 2 length=333
            ?????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
            @SRX2802320.3.2 3 length=333
            CAGGATTAGATACCCTGGTAGTCCACGCCGTAAACGATGTCTACTTGGAGGCTGTGGTCATGAACCGTGGCTTCCGGAGCTAACGCGTTAAGTAGACCGCCTGGGGAGTACGGCCGCAAGGTTAAAACTCAAATGAATTGACGGGGGCCCGCACAAGCGGTGGAGCATGTGGTTTAATTCGATGCAACGCGAAGAACCTTACCTGGTCTTGACATCCACAGAATTCTCCGGAGACGGAGAAGTGCCTTCGGGAACTGTGAGACAGGTGCTGCATGGCTGTCGTCAGCTCGTGTTGTGAAATGTTGGGTTAAGTCCCGCAACGAGCGCAACCCT
            +SRX2802320.3.2 3 length=333
            ?????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
            @SRX2802320.4.2 4 length=333
            CCGGATTAGATACCCTGGTAGTCCACGCCCTAAACGATGACAATTAGCTGTTGGGACACTAGATGTTTTAGTAGCGAAGCTAACGCGTGAAATTGTCCGCCTGGGGAGTACGGTCGCAAGATTAAAACTCAAAGGAATTGACGGGGACCCGCACAAGCGGTGGATGATGTGGTTTAATTCGAAGCAACGCGAAGAACCTTACCAGGTCTTGACATCTAGTGAGAGTCCTAGAGATAGGATGTTTCCTTCGGGAACCGTAACACAGGTGCTGCATGGCTGTCGTCAGCTCGTGTCGTGAGATGTTGGGTTAAGTCCCGCAACGAGCGCAACCCT
            +SRX2802320.4.2 4 length=333
            ?????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????

Can you give me any clue for that

Thanks in advanced

Ashutosh

Hi @ashutosh,
It would be a lot easier to troubleshoot this if we had some basic information about these reads. For example, as per my previous inquiry: What system was used, any prior quality control, trim, filter etc...
For example, your original quality plots show 700+ long reads which is well beyond the usual Illumina data. Even 2x300 reads which is Illuminas max would not be as long as your plots show. This makes me think dada2 denoise single is already not the right tool for job.So can you give us more informationa bout these reads please.

Can you elaborate on what you made you think these ones worked vs the other projects? Are each of these runs/experiments using the same sequencing system?

Can you also re-run your dada2 command with the --verbose tag added.
Lastly, the .fastq example you are showing does not look like normal fastq file. Notice that all the quality scores are all the same ?. While this is a valid symbol (thus it passed the validation test) it is not normal to have all identicial values, at least not with common files we see here. Again we'll need to know more about these files and runs to make sense of it all.

The interactive quality plot shows the Quality for all reads is 30 (data5_demux.qzv is attached )
data5_demux.qzv (289.8 KB)

Hi @ashutosh,
The quality plot shows these all 30s because all the quality scores are ? which is ASCII symbol for 30. We’ll still need the information asked in my previous 2 posts if we’re to trouble-shoot this any further. Thanks.

1 Like

Actually I have downloaded these files from the NCBI SRA database the accession list is attached here. SRR_Acc_List.txt (3.1 KB)
In SRA database it shows Illumina Miseq paired-end data but when I have checked the reads it shows only one read for each identifier. ( maybe someone messed up these reads during curation

The log file is here with the rerun
qiime2-q2cli-err-0_6c66n1.txt (3.3 KB)

Can you suggest anything to proceed with these data

Thanks

Hi @ashutosh,
It would certainly be worth looking into contacting either the curators or the original authors. There seems to be something wrong with the files you downloaded. I looked up one of those files and though they say it is MiSeq data, they seem to have artificial quality scores. Those quality scores are very important for any type of denoising or quality control so I would start there. If they are already quality controlled prior to uploading (though they really shouldn’t be) for some reason then you may have to use an OTU picking method. But let’s see what info you can get before we move there.

Thanks @Mehrbod_Estaki
My goal is to generate the feature table, taxonomy table with count data for each sample and the tree file.
As you suggested can I go with this tutorial https://docs.qiime2.org/2018.4/tutorials/otu-clustering/
I have tried with this command
qiime tools import --input-path ../../single-end/fasta/*.fasta.gz --output-path test.qza --type 'SampleData[Sequences]'

I got this error

There was a problem with the command:
     (1/1) Got unexpected extra arguments
      (./fasta/SRX2802059.fasta.gz
      ./fasta/SRX2802060.fasta.gz
      ./fasta/SRX2802061.fasta.gz
      ./fasta/SRX2802062.fasta.gz)

I am wondering whether I can import multiple fasta files into qiime2 as I need count data per sample?

Ashutosh

Hi @ashutosh,
Sound like you are skipping a few crucial steps here we discussed. As I mentioned, you really should inquire about those quality scores before you go any further. Either way you’ll need to either access the proper fastq files or find out what has been done to them exactly. Before that happens I wouldn’t carry on any further with your data. Even with OTU picking methods you should have quality-controlled reads, so we need to know that info.
As for your importing question, could you please read the importing tutorial and if you don’t have your question answered still, open a new topic since that is deviating quite away from the original topic here.

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.