Using DADA2, losing 50% of reads after filtering and 80% of reads by the end

[Nicholas_Bokulich]

@amorris28 and @FrancineMarques,
Thanks for the detailed post and providing the demux summary QZV — just what I needed to diagnose.

The issue does appear to be your trimming parameters. The forward reads are pretty good quality but 290 nt is a but too long to truncate, given the quality dropoff around base 280-290.

The reverse reads are pretty much rubbish. You could try trimming a lot and pray it joins, but given the length of the V4 I wonder if it may be better to just use the forward reads (since that covers most of the V4 and the most informative part of the V4 anyway, presuming the reads are in the correct orientation). But I recommend trying it both ways: trim the forward a little bit more and try both single-end and paired-end and look at the yields.

That should improve read yield at the filtering stage, since you will save more forward reads that have some low-quality bases between base 280-290.

@amorris28

I’ve tried not trimming at all, but I get fewer reads in the end (which makes sense because more errors).

Don't do that! That will lead to more filtering. You are losing most of your reads at the filtering stage, not the overlap stage.

@amorris28

• Increasing the --p-max-ee should increase the number of reads. I’m hesitant to relax this parameter, but I could still try that.

Don't! It can help but I think what I have recommended above is a better way.

@amorris28

1. In some circumstances, tossing the reverse reads (if they’re low quality) and just keeping the forward reads can increase coverage because you lose fewer from merging. Haven’t tried this yet.

Try it! Your reverse reads do not look good, but the forward looks great. Don't sully the forward by pairing it with grubby reverse reads.

@amorris28

• Is losing 80% of reads high or is that typical? What about 50% during filtering?

This is more than usual

@amorris28

• Would my data be worth running just the forward reads and not trying to merge forward and reverse?

YES!

Now to you @FrancineMarques
(Thanks by the way for posting on this topic, @FrancineMarques, instead of opening a new one, since you both have what sounds like quite similar problems)

@FrancineMarques to diagnose your issue I would really need to see the demux summary QZV to view the quality profiles. But your problem is pretty similar to @amorris28's — you are probably not trimming/truncating adequately and low-quality bases are being left in the reads, leading to too much filtering at the pre-filtering step. Try trimming and truncating more to get out more reads.

HOWEVER, your issue is slightly different because you have a longer amplicon (V4-V5) and so need to leave enough read length to permit overlap. So try truncating more and pay very close attention to the read yield at the "merged" step... you want to adjust the parameters so that "filtered" yield increases but the % merged does not go down.

Good luck both.