Problem with demux using qiime2 2020 version vs 2019 version

[lttnguye (Tu)]

I update my qiime 2 by installing the most recent version using conda. Right now I have the most recent qiime2-2020.11 but have the same problem with qiime2-2020.8. My problem is for running demux emp-paired command and here is the error:
Plugin error from demux:

No sequences were mapped to samples. Check that your barcodes are in the correct orientation (see the rev_comp_barcodes and/or rev_comp_mapping_barcodes options). If barcodes are NOT Golay format set golay_error_correction to False.
Please note that:

• I’ve checked metadata file; created test file using barcode from from barcode file/ have manually checked that barcode in metadata file have matches in sequences files.
• Run the same command/ test file using an older qiime version from 2019 and it works, no problem, no error at all.
• Here is my command line: qiime demux emp-paired --i-seqs test.qza --m-barcodes-file test_metadata.txt --m-barcodes-column Golay_Barcode --output-dir test

Has anyone had the same problem and can point me into the right direction to troubleshoot this? Since I’ve already created a test files and re-install qiime2, I’m not sure what to do at this point. Thanks.

[ChrisKeefe]

Hi @lttnguye,
Are you sure you have golay barcodes? And are you sure reverse-complementing isn’t required? Have you tried the suggestions from the error message?

If you’re still having issues after trying that, sharing some of your data and metadata might help us troubleshoot. If you’re not able to share complete files for data privacy reasons, screenshots or copy-paste of the barcodes and some sequences with barcodes visible would be fine, so long as it’s possible to match barcodes from one to the other.

Best,
Chris

[lttnguye (Tu)]

Hi Chris,

Yes, sure that I have Golay barcode and I actually have files & command lines that I successfully ran & demuxed with previous Qiime2 version (2019.1) that when I use them to test the newly installed Qimme2-2020.11 still gave me the exact same error.
I made these test files to make the command run faster but it's basically the first few sequences of my files, and the barcodes in the metadata file are perfect matches from barcode sequence file; they all however gave me the same error message.

Do you have an idea what's causing this error? Thanks for your help.

test_metadata.txt (424 Bytes) reverse.fastq.gz (3.1 KB) forward.fastq.gz (3.0 KB) barcodes.fastq.gz (546 Bytes)

[ChrisKeefe]

@lttnguye

Do you have an idea what’s causing this error?

Golay error correction was introduced in 2019.4. The suggestions in your error message encourage you to work with reverse-complementing parameters, because orientation matters when you're using error correction. This has been discussed a number of times on this forum - here's one example.

A colleague of mine was successful in demultiplexing your test data with error correction using the following:

qiime demux emp-paired \
  --i-seqs mux.qza \
  --m-barcodes-file test_metadata.txt \
  --m-barcodes-column Golay_Barcode \
  --p-rev-comp-barcodes \
  --p-rev-comp-mapping-barcodes \
  --o-per-sample-sequences demux.qza \
  --o-error-correction-details ecc.qza \
  --verbose

Try something similar with your complete data set, and let us know how it goes for you. As a side note, it's also possible to turn golay error correction off using --p-no-golay-error-correction. This will likely lose you some reads, save you some time, and give you similar results to what you got in 2019.1.

Best,
CK

[lttnguye (Tu)]

Thanks Chris. I remember trying to set --p-golay-error-correction to False before posting on the forum, but that gave me the same error… not sure any more why.

One follow-up question: I tried both the command in your post and the option of using only --p-no-golay-error-correction and they gave me quite different results of demux; which was very surprising to me considering the very simple data test set. Can you point me to the right documentation where the reverse complement mapping or golay correction parameters are further explained? Thank you so much for your help!

[ChrisKeefe]

Not sure exactly which documentation you’re looking for, but here are some resources:

• demux-paired documentation (describes what each parameter does)
• moving pictures tutorial (explains how to get an error-correction report)
• paper on error-correcting barcodes
• scikit-bio reverse_complement docs (we use this for finding the reverse complement)
• emp_paired source code
• golay decoder source code

[thermokarst]

@lttnguye

I tried both the command in your post and the option of using only --p-no-golay-error-correction and they gave me quite different results of demux; which was very surprising to me considering the very simple data test set

Take a look at the error correction stats output - you'll see that quite a few reads are recovered after applying the barcode correction. Without that barcode correct, those reads would just be discarded (or worse yet - assigned to the wrong sample ) .

Five reads had barcode errors that were successfully fixed, which means that those reads were recovered. Without the error correction, those 5 reads would've been discarded (granted, not all of those reads are associated with a sample, but if they were, via metadata, they would've been assigned). On the flip side, disabling error correction might wind up in a situation where reads are erroneously assigned to samples, because they have errors in the NTs