DADA2 Merge problems

[Shlomit (Shlomit Medina)]

Hi
I have a problem with DADA2 (version 2019.4.0) in QIIME2 (qiime2-v2019.04) in conda: the merge results are poor. The region is V3+V4, the primers are 341F-806R. The primers are attached to the sequences. I took 3 samples for a tests. The original demux.qzv file is attached.
I used the following script in QIIME2:

qiime dada2 denoise-paired
–i-demultiplexed-seqs demux-paired-end.qza
–p-trim-left-f 20
–p-trim-left-r 20
–p-trunc-len-f 286
–p-trunc-len-r 223
–o-table table.qza
–o-representative-sequences rep-seqs.qza
–o-denoising-stats denoising-stats.qza

The denoising-stats results:
F+R input filtered denoised merged non-chimeric
#q2:types numeric numeric numeric numeric numeric %
Shlomit001-Sh-Cn-0T-S1 40,340 33,272 26,315 14,905 13,547 34
Shlomit002-Sh-Cn-0T-S2 51,994 42,342 34,953 21,235 19,402 37
Shlomit003-Sh-Cn-0T-S3 45,586 37,210 31,013 18,938 17,263 38

Sequence length 400-425 bp.

Working with longer sequences --p-trunc-len-f 300 --p-trunc-len-r 289 , gave worse results.
I tried to use DADA2 in R for further possibilities, but it did not help.
I did DADA2 for forward files only, --p-trunc-len 286, and it doubled the recovery, but the sequences were shorter: 266 bp.

Are these results acceptable?
Is the sequencing is OK?
What can I do to improve the results?

demux.qzv (293.2 KB)

[yanxianl (Yanxian Li)]

Hi Shlomit,

Welcome to the QIIME2 community.

The denoising stats look fine except for the chimeras removing, which resulted in the lost of most reads. What’s the length of your forward and reverse primer? Are they all 20 bases? The length of commonly used v3-4 forward and reverse primer is 17 and 21.

Cheers, Yanxian

[Shlomit (Shlomit Medina)]

31,013 18,938 17,263

Hi
In the stat table, the numbers of the last sample are: input 45,586, filtered 37,210, denoised 31,013, merged 18,938, non-chimeric 17,263. the recovery percentage as I calculated is 38%. So it looks like the problem is “merge” and not “chimeras”. The same pattern is in the other 2 samples.

Primers length are probably as you said, but I was told it is not very important. Is it?

[yanxianl (Yanxian Li)]

Hi Shlomit,

Sorry about the confusion. I thought the number in bold was the sequence count after chimera removing. You’re right that you lost quite a bit of reads during the merging. As mismatches between the forward and reverse reads after denoising is not allowed, using longer forward/reverse truncate lengths may result in worse results. Your reads are of high quality. Based on the quality plot, maybe you can try –p-trunc-len-f 290, –p-trunc-len-r 261/287(median quality score 30/20)? You’ll still have a merged sequence length of ~510/~536 bps.

In my experience, most of samples retain more than 50% of reads after denoising by the DADA2 pipeline. But if samples have really bad read quality towards the 3’ end, you may see a huge loss of reads during merging because of mismatches or insufficient overlapping. Below are dada2 stats from one of our study for your reference, but they may differ greatly among different studies.

For the primer length, I assume it’s best to trim off the exact lengths as the presence of primer sequence may affect the chimera dectition. The new qiime2 taxonomic assignment method uses machine learning to train the model on the amplicon without primer sequences, so I guess it’s best to just trim off the exact primer sequence. But I may be wrong. QIIME2 developers may have better answers.

P.S.
I’m not sure if 3 samples are enough for the DADA2 to learn the error rate. Maybe also try a higher number of testing samples, say 6?

Cheers, Yanxian

[ben (benjamin w.)]

Hi Shlomit, I was just troubleshooting samples for another group and these were roughly the recovery after DADA2. I think it’s acceptable, but here’s what I would do:

1. Check the quality of forward and reverse
2. Are these degenerate primers?

I found that the quality of overlap between the forward and reverse are likely the reason for many of these issues.

Please see this link where I was trouble shooting DADA2 V3V4 region:

Nextera paired-ends using a linker

Hi @ben, Definitely not a one-stop shop. There are many steps, each with its own troubleshooting peculiarities. We should really compile a single forum post to help navigate the process! Oh that’s a tough one — I am not sure we have had to troubleshoot that step before, since usually filtering or merging is where the big issues happen. Maybe you could upload the qzv of your dada2 stats? Filtering has nothing to do with overlap — at that stage the reads are still being filtered independently…

I think after reviewing I had significant loss as well. Ben

[Shlomit (Shlomit Medina)]

Hi

I took 6 samples for a new test, as suggested. The results were similar.

I tried a range of trunc-len values in F and R. The best results were f260 r220, after trimming the primers.

Is it like ca f280 r240 if I would do the analysis without trimming the primers with cutadapt?

The best results were 32-37% recovery. Not very good.

The quality of forward and reverse are good, as you can see in the demux.qzv attached file. The primers were new.

I got similar merge results in another project, from another working group. It looks like DADA2 problem.

Do you suggest to accept the results, (32-37% recovery), or do the analysis on Foreword sequences only, or do something else?

[yanxianl (Yanxian Li)]

I tried a range of trunc-len values in F and R. The best results were f260 r220, after trimming the primers.

Is it like ca f280 r240 if I would do the analysis without trimming the primers with cutadapt?

Hi, assuming that cutadapt trimmed off the first 20 bases of your froward and reverse reads, then you’re right. Note that primer sequences in the reads can interfere with both merging and chimera removal. Make sure that you correctly removed the primer sequences.

The best results were 32-37% recovery. Not very good.

Losing a large fraction of your reads may not necessarily be a bad thing. Check this comment by Dr. Benjamin Callahan. You can run a rarefaction analysis to find out if the resulting sequences are enough for the downstream analysis for most of your samples. If that’s the case, then you’re fine.

I got similar merge results in another project, from another working group. It looks like DADA2 problem.

The lost of large fraction of reads is more likely caused by the data than the DADA2 pipeline. To verify that, you can download some similar data with mock, denoise the sequence with the QIIME2 version you’re using and see what you get. If the mock looks as it should be, then it’s not the DADA2’s problem.

Do you suggest to accept the results, (32-37% recovery), or do the analysis on Foreword sequences only, or do something else?

As mentioned above, a positive control helps you to find out if there’s something wrong with the bioinformatics. You can test your ideas with the mock sample if you happened to include one in your sequencing run. If not, you may want to go ahead with both approaches and judge the results with your knowledge about your samples. Use the one that makes sense to you.

-Yanxian

[Shlomit (Shlomit Medina)]

I did the rarefaction as suggested and the result is:

(I am not sure the picture is loaded properly).
How do I know if the resulting sequences after dada2 are enough for the downstream analysise?

[Shlomit (Shlomit Medina)]

I compared the best DADA2 results of paired end F+R, and DADA2 results of single end.
The results of BARPLOT were very different. F+R is much more detailed than the F. The distribution of taxa is different.
I do not know what to choose.
taxa-bar-plots260.220.qzv (410.8 KB) taxa-bar-plots.F.qzv (331.7 KB)

[yanxianl (Yanxian Li)]

As the number of sequences used for computing alpha-diversity increases, the observed alpha-diversity increases as well. If a rarefaction curve reaches a plateau at a certain sequencing depth, then it’s an indication that the sequencing depth is sufficient to uncover the taxa composition in that sample. Based on your rarefaction plot, the observed OTUs seem to level out starting from 8000 reads. Thus, 12000 sequences seem to be sufficient for your downstream analyses.

[yanxianl (Yanxian Li)]

For the taxa barplot, you got very different taxonomic composition. It’s expected that the pair-ended reads gave you a higher taxonomic resolution as the amplicon lengths used for the sequence classification are longer than those of the forward reads only.

However, it’s alarming that the number of taxa and their relative abundances are so different for the pair-ended reads and forward reads only. You may want to check your workflow to identify if there’s anything that has not been done correctly.

The observed OTUs in your samples are quite high. Another thing you can do to help make the decision is to go through the papers describing the microbiota profile of samples similar to yours: how diverse the biota is and what the dominant taxa are. I believe the known findings can assist you to judge the quality of your results.

[Shlomit (Shlomit Medina)]

I compared my BarPlot results to other very similar soils, and it is similar. So I am happy with the results of F+R. I will use them.
Thank you very much.