I see that q2-feature-classifier classify-sklearn has a default 0.7 confidence score. Where does this value come from? I found this reply by @Nicholas_Bokulich saying that this value was adjusted based on benchmarking results, but there is no reference associated. Was this published or was it an unpublished analysis by the QIIME 2 team? Are there any confidence scores benchmarking papers out there?
yes, you can see some confidence parameter comparisons here:
In a much more extensive benchmark, we did a full grid search of confidence and other parameter settings with Naive Bayes and other classifiers in this paper, though you need to check out the linked testing repo for the full test results, the paper might not show the full grid search results (but see also Table 2):