Hi! After reading through the ITSxpress tutorial, the Fungal ITS tutorial, and Rob's discussion around the current recommended fungal workflow, I've come to the conclusion that using ITSxpress is the blatantly superior option over truncating with dada2. My "confidence" in this choice suggests to me that I understand even less than I thought. So, as someone completely new to processing sequence data, I'm wondering...
When processing fungal sequences, why would someone opt to trim/truncate with dada2 instead of extract the ITS region using ITSxpress?
What you typically see is that at the end (3') of the reads, the quality drops quite a lot with Illumina sequencing data (especially in the reverse read). In general it is better to truncate the reads with dada2 to remove the bad quality regions. This is common practice with e.g., 16S reads.
However, when you use ITSxpress to - for example - extract the ITS2 region, the basepairs at the ends of the reads are already removed, thereby removing the most likely low quality regions. Additionally, apparently ITSxpress could recognize the ITS region, so I somewhat assume that the quality is not extremely bad. But low quality at the 3' ends can still occur so be careful with that.
I analysed various runs with this workflow (first ITSxpress and then dada2 without trunc) and I always ran several DADA2 runs with increasing truncation levels (starting at no truncation). During these test I always found that using DADA2 without truncation led to the best merging and denoising dada2 stats, even if there is a drop in quality at the 3' end. DADA2 also merges and there is overlap, especially at the lower quality 3' regions, so because you have this overlap, the low quality is also a compensated to some extent.
So my advice would be to check the quality after ITSxpress, and based on that run several DADA2 commands with 0 truncation and increasing truncation (e.g., 5 steps).
See also a remark by Benjjneb in a ITS dada2 tutorial: 'The length variation of the ITS region has significant consequences for the filtering and trimming steps of the standard DADA2 workflow. First, truncation to a fixed length is no longer appropriate, as that approach remove real ITS variants with lengths shorter than the truncation length.'
After sequencing reads you need some way to group amplicons from the same organism together and count them up. Because sequencing introduces errors, just counting all the unique sequences does not work, that would make it seem like there are more organisms than there are. Initially this error issue was handled by greedy clustering of sequences at 97% identity, this was the Operational Taxonomic unit (OUT) approach. But we have per position information about the error rate for each read. Callahan et al. (2016) realized that we could more accurately separate error from true diversity by developing a statistical model of errors to create Amplicon Sequence Variants (ASVs). So in short, Dada2 identifies and counts how many real ASVs you have.
ITSxpress does not define and count ASVs, it solves a problem that occurs because of the difference in the structure between 16S rRNA regions and ITS regions. The 16S rRNA gene is made up of a mix of highly variable and highly conserved regions. 16S primers are designed to position themselves at the very edge of conserved regions, which is good for making primers “universal”. Another advantage of positioning them at the edge is they cleanly extract a region of higher, more uniform evolutionary rate and are fairly uniform in size (because big length changes would break the ribosome). Fairly uniform evolutionary rates are helpful for making good phylogenetic estimates. All of this means that 16S can be fed right into Dada2 to count the number of different unique ASVs in a sample.
The ITS regions are different. Typically, they are arranged as SSU-ITS1-5.8S-ITS2-LSU. They can resolve more closely related organisms. ITS regions do not encode structural RNAs so they are under less selective pressure and their margins vary more. To create good primers you need to anchor the primers in the more conserved rRNA regions. ITS sequences also very a lot in length. This leaves a fuzzy margin, and portions of a sequence that evolve very slowly (the structural RNA portions) and parts the evolve fast (the ITS). ITSxpress cuts the ITS out from the LSU, SSU, or 5.8S, this provides clean ITS sequences for Dada2 to call and count ASVs from.
ITSxpress is a tool to preprocess ITS sequences for more accurate ASV calling and better phylogenetics downstream.
Thank you for this reply @Adam_Rivers, you've helped several things "click" in my brain. I understand that the pipeline when using ITSxpress would be Demultiplexing->ITSxpress->DADA2->Assign taxonomy. The reason for this post is that I'm referencing several scripts, and some folks opt not to use ITSxpress at all prior to using DADA2. Instead, they trim and truncate solely with DADA2. I was trying to understand why folks working with ITS regions skip using ITSxpress. At first, I thought it was because ITSxpress didn't work for current versions (but all you have to do is install it). So, I assumed folks skipped that step because they believed another route to be better.
However, since making this post, I am unable to get ITSxpress to work because QIIME caches the deployment (which takes hours in itself) and then fails due to an Out of Range IndexError. I spent all yesterday trying to diagnose the issue to no avail, so now I'm one of the folks skipping extraction with ITSxpress...
Thank you for sharing that tutorial @Rob_DNA! I appreciate you recommending a quality check and further truncation with DADA2 after ITSxpress, as I was under the impression no more trimming/truncation was necessary afterwards and that DADA2 was just for de-noising/ASVs.
I appreciate knowing I shouldn't truncate to a fixed length. The script I'm referencing does exactly that at 249bp (regardless of the interactive quality plot results). However, my quality doesn't drop for forward reads, so I didn't want to do that. (My reverse are <30 so I'm only using forward)
Hi @kranderson ,
just to chime in as someone who analyzes ITS data and has used both options but usually just truncates with dada2 instead — to add to @Adam_Rivers 's point, some ITS primers are sitting fairly far into the SSU and trimming to the ITS domain is much more advantageous, but other primers sit directly flanking the ITS so primer trimming can already remove most of these conserved regions without the added step of running ITSx. So as with any marker I think the choice of workflow depends a bit on a combination of your methods as well as experimental goals, there is not really "one right way" to do things.
Oof yeah I would definitely just toss the reverse reads then. For ITS data I often use only the forward read myself to avoid issues with merge failures (because ITS is variable in length, many paired-end reads may fail to merge, leading to data loss... even when the reverse reads are longer than 30 bp!)
I think a lot of people just stick with dada2 truncation because it's the path of least resistance — it's the standard workflow, it's what everyone learns, and if your data quality is decent it works fine. ITSxpress is an extra dependency and another thing to troubleshoot, so unless you've got a specific reason to use it (bad quality at the ends, primers far into the SSU like that last reply mentioned), the simpler approach wins out even if it's not theoretically ideal.
How did you obtain this run? It looks quite strange and atypical. What kind of platform was used? Based on the number of reads I assume a Illumina NextSeq run ?
I of course do not know what is exactly in your library, but some samples have a read number of 10's-100's, while some have a couple of millions reads. There thus seems to be an equimolar pooling issue or something else.
Then your quality plot looks quite strange in my opinion. In my experience, Illumina forward reads stay very high uniform quality till at least 200-250 bp. See a screenshot from an illumina miseq from 2017 below. I think this still had an issue with the v3 (or v2?) kit that their reverse dropped significantly after a certain position. Your median qualities look OK, but overall there seems to be quite some low quality bases early on in the sequence. Could you give us some more info on your run? Did you do the lib prep or you send your DNA extracts to a company?
with Merge failures do you mean that there is no overlap with the FW and RV read? I think this is rarely/not an issue if you analyse a single ITS region (ITS1 or ITS2) with Ilumina sequencing that have 300 bp PE sequencing.
So, as someone completely new to processing sequence data, I'm wondering...
