Clarification of a bug/or not a bug? qiime2 demux with little reads (2019.4 version to 2019.7 version)

Hi all,

I wanted to follow up with a quick question, but there was an issue with some folks who had problems with the "qiime demux" command, where demux would only assign very little reads to the correct barcode:

I had a very similar problem running QIIME2-2019.4 as these users:

And it was solved by running the --p-these-are-not-golay-barcodes, but the thing is, as far as I know, we used Golay barcodes. I think this problem has been solved in 2019.7, but could the developers/coders explain what the issue was?

I only ask because I was running the April 2019 build, as the HPC we run it on hasn't been updated and I had to trouble shoot the issue this weekend with some older code. I reviewed the changelog (at least what was posted on the forums), I could only assume it was a fluke? The problem fixed itself on 2019.7? Thank you.

Sincerely, Ben

Hi @ben!

I think the parameter you are referring to is --p-no-golay-error-correction :wink:

Nope, not a fluke, just poorly described. Let me try and summarize!

Without Golay error correction, it doesn't matter the orientation your barcodes sequences are in, so long as they are the same orientation as the barcode mapping in your metadata file. If they aren't, you can use --p-rev-comp-barcodes to reverse complement the barcode sequences to match the barcode map OR use the --p-rev-comp-mapping-barcodes to reverse complement the barcode map to match the barcode sequence. Either option is fine, because we aren't doing anything to the barcode sequence, except for a direct match ("is ACGT equal to ACGC?"). So, that was the status quo for a long time with this plugin.

Now, enter into a world where Golay barcode correction is enabled by default (because this is technically how the EMP spec is defined). Now the matching problem is preceded by a step where the barcode sequence is first error corrected. The thing about Golay error correction though, is that they only work in one orientation. You can't reverse complement a Golay barcode and expect it to be error corrected. So now is when really understanding the difference between --p-rev-comp-barcodes and --p-rev-comp-mapping-barcodes matters: first, you must ensure that your barcode sequences are in the correct orientation in order to be error corrected (using --p-rev-comp-barcodes, if your barcode sequences are reverse complemented). Then, you need to ensure that the barcode mapping is in the same orientation as the barcode sequence (using --p-rev-comp-mapping-barcodes if necessary).

So, to tie it all back to people using the --p-no-golay-error-correction as a "solution" to the problem --- I will suggest that that solution only works because it completely neglects to perform Golay error correction. If you are seeing a significant number of reads lost when demuxing with Golay error correction on, please take a closer look at the orientation of your reads, and determine if you need to perform reverse complementing as described above.

:qiime2: :qiime2: :qiime2:

EDIT:

please see this much shorter summary here: Demuxing golay barcodes? - #8 by thermokarst

3 Likes

Ok, this actually makes sense and I thought I was going crazy.

I think we used to have our barcodes on the reverse primer, but then switched to the forward primer some time a couple of runs ago.

Our sequences used to demux really well, but then suddenly the code stopped working. I then demultiplexed with some of the error correction turned off and then it worked.

I tried to run both --p-rev-comp-barcodes and --p-rev-comp-mapping-barcodes. I can’t remember which one worked, but suddenly it started to demultiplex again.

This weekend while working on old sequence runs for a collaborator, I tried do it the new way trying to understand if this was a run where the barcode primer was on the forward or the reverse and I could only assume that it was on the reverse primer. Thus, I think turning off the --p-no-golay-error-correction fixed the demultiplex issue.

Ben

1 Like

Thanks for following up, @ben!

To anyone who might stumble across this thread: don’t immediately disable Golay barcode error correction if you have Golay barcodes and are suddenly seeing a huge drop in recovered reads. Instead, take some time to understand the orientation of your reads, barcode sequences, and mapping barcodes. If you don’t know, ask your sequencing center.

2 Likes

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.