Clarification of a bug/or not a bug? qiime2 demux with little reads (2019.4 version to 2019.7 version)

Hi @ben!

I think the parameter you are referring to is --p-no-golay-error-correction :wink:

Nope, not a fluke, just poorly described. Let me try and summarize!

Without Golay error correction, it doesn't matter the orientation your barcodes sequences are in, so long as they are the same orientation as the barcode mapping in your metadata file. If they aren't, you can use --p-rev-comp-barcodes to reverse complement the barcode sequences to match the barcode map OR use the --p-rev-comp-mapping-barcodes to reverse complement the barcode map to match the barcode sequence. Either option is fine, because we aren't doing anything to the barcode sequence, except for a direct match ("is ACGT equal to ACGC?"). So, that was the status quo for a long time with this plugin.

Now, enter into a world where Golay barcode correction is enabled by default (because this is technically how the EMP spec is defined). Now the matching problem is preceded by a step where the barcode sequence is first error corrected. The thing about Golay error correction though, is that they only work in one orientation. You can't reverse complement a Golay barcode and expect it to be error corrected. So now is when really understanding the difference between --p-rev-comp-barcodes and --p-rev-comp-mapping-barcodes matters: first, you must ensure that your barcode sequences are in the correct orientation in order to be error corrected (using --p-rev-comp-barcodes, if your barcode sequences are reverse complemented). Then, you need to ensure that the barcode mapping is in the same orientation as the barcode sequence (using --p-rev-comp-mapping-barcodes if necessary).

So, to tie it all back to people using the --p-no-golay-error-correction as a "solution" to the problem --- I will suggest that that solution only works because it completely neglects to perform Golay error correction. If you are seeing a significant number of reads lost when demuxing with Golay error correction on, please take a closer look at the orientation of your reads, and determine if you need to perform reverse complementing as described above.

:qiime2: :qiime2: :qiime2:

EDIT:

please see this much shorter summary here: Demuxing golay barcodes? - #8 by thermokarst

3 Likes