Hi @Kris_P,
Is this an issue with other project data, or just this project? Often this approach works well. I usually observe such patterns when the data set itself is awry, or the orientation of the reads is out of sync with the naïve bayes classifier. That is, the sequence data should be in the same orientation as the reference database. To aid in this situation you can use the rescript orient-reads action to correct your paired-end reads prior to denoising and then run naïve bayes classifier. Alternatively, you can use rescript orient-seqs, to re-orient your ASVs, prior to running the naïve bayes classifier.
You can also try constructing your reference database with extract-seq-segments approach to potentially improve the quality your16S rRNA gene NCBI classifier. Basically, just follow this tutorial after the extract reads step.
Finally, is there a reason why you are not using SILVA, GTDB, or Greengenes2 for this? Note: the upcoming QIIME 2 release will have the latest GTDB reference within RESCRIPt, but if you would like to use it now, you should be able to pip install the latest version of RESCRIPt from the GitHub repo.
Finally, I have a working draft of some tools that have been removed from RESCRIPt, and we are in the process of "adding them back" via a separate plugin called RESCRIPT-evaluate. This will enable you to evaluate the characteristics of the classifier you generate. That is, the equivalent RESCRIPt evaluation commands within this tutorial should be replaced with the same-named commands of this new plugin (which requires RESCRIPt).