Interpreting confidence scores for taxonomic assignment (classify-sklearn)

[Nicholas_Bokulich]

Hi @Matilda_H-D,

@Matilda_H-D

I’m looking for some guidance on how to interpret the ‘Confidence’ column for taxonomic assignments obtained using classify-sklearn

See this post for some more details: Confidence values - taxonomic assignment - #2 by Nicholas_Bokulich

@Matilda_H-D

but since the answer then indicated that the notion of confidence was under devlopment

There have been no changes since then. We were testing out different ways to calculate confidence but wound up sticking with the original.

@Matilda_H-D

I also noticed the default value for --p-confidence parameter in the current version of the classify-sklearn plugin differs from what is described in the answer to that question, I was hoping to get an updated response.

we wound up adjusting the default, based on benchmarking results, but not how confidence was calculated.

@Matilda_H-D

Can we interpret the Confidence value as % sequence identity to the reference sequence for the taxon an ASV is assigned to (i.e. 1 is 100% sequence identity to the reference sequence, 0.95 is 95% sequence identity)

No, this is totally unrelated to % identity. Confidence values here are the raw probability estimates output by the naive Bayes classifier, i.e., the predicted probability that the predicted taxon is correct and not another taxon. Naive Bayes classifiers are good at classifying but poor at estimating probabilities, so the “confidence” scores should not be taken too seriously.... just a rough estimate of how confident the classifier feels about its own prediction!