Sir, there's no fly in my digital soup. what gives?

[Nicholas_Bokulich]

@devonorourke

The thing about that 1 nt difference pushing the similarity threshold to include several other hits… it seems crazy to think that it would result in shifting the taxonomic profile of that ASV from something that includes everything up to and including species to maybe only now including a order or class level

The thing I like about working with the alignment+consensus classifiers is that they are fairly "transparent". I.e., that 1nt would make such a difference is surprising, but the cause is obvious. That 1nt just pushes it over the threshold, probably supporting matching to the one additional sequence needed to reduce the consensus threshold.

@devonorourke

And yet, you were crazy correct!

So you have the right idea — there are a few ways to address this (you mentioned a few, I will list + add others) but by fixing it for this specific sequence you don't know how you will impact other sequence classifications, potentially leading to poorer classifications for those.

1. Adjust maxaccepts. In this case even maxaccepts 5 would do the trick. But yes this would impact other classifications, potentially in bad ways.
2. Adjust the percent identity — but then some other sequences may become unclassified.
3. Database curation: you mention eliminating replicates, but as you say this will open up a big old can of worms!!! Better yet would be to add additional sequences for sparsely represented taxa (if you knew the sample size for each taxon you could set very rational maxaccepts but the uneven representation makes this tricky). But good luck with that. A third option is to remove all incompletely classified taxa. E.g., the sequence labeled only as k:Animalia,p:Arthropoda,c:Insecta,o:Megaloptera is causing problems here, and how useful a sequence like that is is questionable. I have done this for other databases with success... it may be benchmarked in tax-credit if you want to take a look.
4. Develop a new consensus classifier. Obviously this is not a quick fix! But classifiers that do some sort of dynamic matching, e.g., only perform consensus matching on matches that tie for percent identity. This would actually be really easy to do using the existing vsearch classifier as a template... you would just create a new consensus assignment algorithm and peg it on the back.

@devonorourke

I can see why all of this becomes a frustratingly long series choices - and a lot of testing. It feels like the 16S/ITS worlds have been doing this for years, and I don’t know of anyone who has touched on these questions in COI (at least for arthropods). Perhaps there are simple steps to take to manage my risk other than what you’ve already suggested which mirror what some of the earlier 16S/ITS development followed.

Indeed! Benchmark what works best for arthropod COI. We have already discussed this — I made tax-credit expressly for this purpose, so that others can re-use for this same type of benchmarking.

@devonorourke

I’m just not sure of the history of that testing/development.

That's the past few years of my life .