I am processing 16S Illumina MiSeq fastq files using QIIME2 2019.10 and am not getting the expected reads per sample after running demux emp-single. Here is my script:
I first tried running demux emp-single without the reverse complement flags and received this error.
ValueError: No sequences were mapped to samples. Check that your barcodes are in the correct orientation (see the rev_comp_barcodes and/or rev_comp_mapping_barcodes options)
After reading through the forum, I saw that golay error correction had been added in version 2019.4 so I added the reverse complement flags. However, with both flags I get 1-3 samples with 60K-70K reads and the rest of the 150+ samples have 1000 or less. I get the same result when I only reverse complement the mapping barcodes (–p-rev-comp-mapping-barcodes). If I only complement the sequence barcodes (–p-rev-comp-barcodes), I get the error that no sequences were mapped to the samples. The same data was processed using version 2018.11 and all of the samples have 20K+ reads after demultiplexing.
I am using barcoded forward primers made from IDT based on the EMP 16S Parada Apprill barcodes.
Do you need to RC anything in the first place? If you don’t know the orientation of these things then this is a good question for your sequencing center. Have you run without reverse complementing both the barcode map and the barcode sequences?
Yes, I mentioned that I had first tried without the reverse complement flags and received the error that no sequences mapped to the samples. Since the barcodes are on the forward primer and I use the barcode list downloaded from the EMP website, I didn’t think that anything had to be RC’d.
Yep! My point is that there are some questions you need to ask your sequencing center in order to get a handle on the data. The first thing to confirm is whether or not your barcodes are Golay barcodes (you mentioned in your post you used EMP barcodes, which should be Golay barcodes, but worth double-checking).
Please see these posts for discussion of the things that should get sorted out in the discussion with the sequencing center:
Thank you for the feedback. I do my own library prep and ordered the primers so I know that they are the EMP Golay barcodes from IDT. We only use the sequencing core to load the MiSeq and I hand them the pooled library.
Although I know I have golay barcodes, I am going to add the --p-no-golay-error-correction flag to another attempt. I am also running the exact files in version 2019.1 and will compare those results.
Have you verified this? If you have Golay barcodes, then Golay error correction should work as expected. Again, I encourage you to make sure you have a handle on the nature of the barcodes (Golay or not), their orientation in the sequencing product, and their orientation in your "barcode" column of your metadata file --- these are three parts to the equation that need to be provided by you. Keep us posted!
Again, I do our library prep. I handle every step of the process: normalization, barcoding PCR, pooling amplified DNA. I also ordered the primers. I downloaded the forward barcode spreadsheet from the EMP website in July 2018, checked that the primer sequence that IDT sent me matched that spreadsheet and use that spreadsheet to create our barcode text file. According to everything I can check, I have Golay barcodes and the demux doesn’t work. I have tried adding both reverse complement flags, neither flag, and one or the other flag.
Did it work with previous versions of QIIME2? I am only wondering because you mentioned that this version is what you’re having problems with. (reading it seems to have worked on November 2018 version of QIIME2)?
Could you post your barcode/mapping file or a screen shot? Ben
Taking one step back from just randomly trying things --- have you verified the orientation of the barcodes and the barcode sequences? Attempting to apply these flags may or may not work, but I think it would be really helpful for you moving forward if we had a clear explanation for why the various flags may or may not be necessary for your data (and these explanations will come from understanding the orientation and nature of the barcodes). If you want to send some data our way to peek at, please send me a download link in a DM.
Yep! That means you should be well situated to answer the questions above!
Just to double-check, did you get a chance to review the links I provided above? They go into a lot of detail about the orientation requirements for Golay error correction, and why you might need to reverse complement the barcodes or the barcode sequences. I am happy to answer any questions that might come up specific to those posts!
I do my own library prep and ordered the primers so I know that they are the EMP Golay barcodes from IDT. We only use the sequencing core to load the MiSeq and I hand them the pooled library.
