Confidence values - taxonomic assignment

[uschuette (ursel)]

I am new to Qiime and I want to make sure I interpret the output correctly. We have been using Qiime 2, and. I have a question about the confidence value that is given for the taxonomic level that each ASV is assigned to.

For the assignment of some ASVs we get low confidence value such as 0.9, and I think it indicates that we should not assign the ASV to that particular genus. But if I understand it correctly, I cannot just infer from the confidence value which taxonomic level the ASV can be assigned to. Is that correct? For example, if we have an ASV that is assigned to the genus Polynucelobacter with a value of 0.94, I cannot just assign the ASV to the corresponding family (Burkholderiaceae), correct?

Can we set the cutoff for the confidence level to e.g. 0.95 prior to analysis so that ASVs are only assigned down to a taxonomic level where I have a confidence value of >0.95, or am I missing something about what the confidence value actually means?

I really appreciate your help,

Ursel

[Nicholas_Bokulich]

Welcome to the forum @uschuette!

This topic describes how the confidence score is calculated:

How is the "Confidence" calculated with taxa assignments?

Thanks @John_Chase, the Confidence column isn’t documented yet, sorry. It is not yet a stable feature. For the current release, I have tried to mimic the way that RDP classifier calculates confidence values. Also, confidence is only calculated and used if the confidence parameter is set to a non-negative value when calling the classify method. The basic classification method is to decompose the read into a bag of overlapping 8-mers, then feed that as input to the machine learning (Naive Bayes …

I cannot just infer from the confidence value which taxonomic level the ASV can be assigned to. Is that correct? For example, if we have an ASV that is assigned to the genus Polynucelobacter with a value of 0.94, I cannot just assign the ASV to the corresponding family (Burkholderiaceae), correct?

The confidence values reported are for the level that is reported… so e.g., the 0.94 confidence is for the genus that is reported and you cannot infer from that what confidence you will have for family-level classification from that score.

Can we set the cutoff for the confidence level to e.g. 0.95 prior to analysis so that ASVs are only assigned down to a taxonomic level where I have a confidence value of >0.95, or am I missing something about what the confidence value actually means?

Yes, that’s what the confidence parameter in this method will do for you. In practice, you should just stick with the default (c=0.7), as it strikes the correct balance between precision/recall — c=0.95 is likely to be way too strict and result in quite poor classification. See the benchmarks here for the evidence, and how the default setting was chosen:

Optimizing taxonomic classification of marker-gene amplicon sequences with...

[uschuette (ursel)]

Thank you, Nicholas for the fast and kind reply. I will look at the paper, and let you know if I have any further questions.
Best wishes,
Ursel