Creating fastq manifest formats from a single multiplexed fastq file

Hello,

I am new to Qiime2 and is facing some difficulties importing my fastq file. I have a single fastq file containing multiple paired end sequence reads. Based on what I understood, this means that the fastq file is multiplexed. According to online platforms, I should import them using the fastq manifest formats and treat them as single end reads.

I followed the steps as stated in the Qiime2 docs on importing data, however it keeps telling me that an error has occured. It says that my file is not a SingleEndFastqManifestPhred33 file. I was wondering if I am missing a step? How should I convert my fastq file to a manifest format?

Here’s a printout of the fastq file I am trying to import:
(qiime2-2019.7) [email protected]:~/QIIME2/16S$ head 16S.fastq
@A8.S501.N705.R7_000000001 M04529:115:000000000-BN78B:1:1101:17721:1125 1:N:0:33
TACGGAGGGGGCAAGCGTTGTTCGGAATTACTGGGCGTAAAGGGCGCGTAGGCGGTCTGTTGCGTCAGGTGTGAAAGCCCCGGGCTCAACCTGGGAGGTGCACTTGATACGGGCAGGCTAGAATCCGGGAGAGGATGGTGGAATTCCCAGTGTAGAGGTGAAATTCGTAGATATTGGGAAGAACACCGATGGCGAAGGCAGCCATCTGGACCGGTATTGACGCTGAGGCGCGAAAGCGTGGGGAGCAAACA
+
[email protected]@GGGGGGGGGGG?GGGGGGGGGGGGGGFGGGGGDGGGGGGGGGGGGGFGGFGGGGFGFGFGFGFFFGGGGGGFFGGGFFFGGGGFGGGFFFGGGGFGGFFFGGGGGGGGGCEGGGGFGGGGGGFE;GGGGGGGGGGGFGGGGGGGDGGGGGGGGGGGGGGGGGGGFGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG
@A8.S501.N705.R7_000000002 M04529:115:000000000-BN78B:1:1101:8843:1148 1:N:0:33
TACGGAGGGGGCAAGCGTTGTTCGGAATTACTGGGCGTAAAGGGCGCGTAGGCGGCCGGTCGCGTCAGATGTGAAAGCCCCGGGCTCAACCTGGGAACTGCATTTGATACGGGCTGGCTTGAGAACGGAAGAGGAGTGTGGAATTCCCAGTGTAGAGGTGAAATTCGTAGATATTGGGAAGAACACCGGTGGCGAAGGCGGCACTCTGGTCCGTTTCTGACGCTGAGGCGCGAAAGCGTGGGGAGCAAACA
+
[email protected]GGEGGFGFGFGFGFFFFFD>>DFGGGFFGFFDGGGGGGGGGGGGFFEGFGGFGFCGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGFGGGGGGGGGGGGGGGGGGGGFGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGEGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG
@A8.S501.N705.R7_000000003 M04529:115:000000000-BN78B:1:1101:10038:1152 1:N:0:33
TACGGAGGGGGCAAGCGTTGTTCGGATTTACTGGGCGTAAAGGGCGCGTAGGCGGGGTATCAAGTTAGGGGTGAAAGCCCGGGGCTCAACCTCGGAACTGCCTTTAAAACTGATACTCTAGAGTCCGGAAGAGGGTCGCGGAATTCCCAGTGTAGAGGTGAAATTCGTAGATATTGGGAAGAACACCGGTGGCGAAGGCGGCGACCTGGTCCGGTACTGACGCTGAGGCGCGAAAGCGTGGGGAGCAAACA

Here is the error that occurred when I try to import the data:
(qiime2-2019.7) [email protected]:~/QIIME2/16S$ qiime tools import
–type ‘SampleData[SequencesWithQuality]’
–input-path 16S.fastq
–output-path 16S-single-end-demux.qza
–input-format SingleEndFastqManifestPhred33

There was a problem importing 16S.fastq:

16S.fastq is not a(n) SingleEndFastqManifestPhred33 file:

Found header on line 1 with the following labels: [’@A8.S501.N705.R7_000000001 M04529:115:000000000-BN78B:1:1101:17721:1125 1:N:0:33’], expected: [‘sample-id’, ‘absolute-filepath’, ‘direction’]

It may be a simple issue but I will really appreciate your help!!
Xin

Hi @xin,

Welcome to the :qiime2: forum!

Before we work through the manifest format, I want to ask if you have multiple samples in that single fastq file, or if it's a single sample?

Best,
Justine

Hi @jwdebelius,

Thank you for replying so swiftly!

There are multiple samples in the single fastq file. Also, I am not sure if this information will help, but I have just checked again and it seems like all the unnecessary parts (e.g. indexes, primers, etc) are already being removed.

Cheers,
Xin

Hi @xin,

This is not a file that should be brought in via the manifest format. However, Im not sure how you can convert it to what you need if you don’t have information about barcodes somewhere. Do you have a second barcode file?

Best,
Justine

Hi @jwdebelius,

I have an excel spreatsheet that details the complete forward and reverse sequences of the 16S primers that were being used. Will that help?

Xin

Hi @xin,

You need somethingt that will map the sequences back to. your excel sheet: a barcode file, barcodes in the sequence headers (which I don’t see), or barcodes in the sequence.

Best,
Justine

Hi @jwdebelius,

I think the barcodes are already being removed from the sequences. Is is not possible for me to import the data if the barcodes have already been removed? What do I have to do to map the sequences? Is it a file which I can create on my own?

Sorry for the many questions. I am really new to this and so I am a little lost. Thank you so much for helping me thus far.

Cheers,
Xin

Hi @xin,

This makes it really difficult because you somehow need barcodes to demultiplex. Can you contact your sequencing provider about how to get the barcodes or if they can give you demultiplexed files?

Best,
Justine

Hi @jwdebelius,

Can I clarify, when you say get the barcodes, do you mean get multiplexed fastq files with barcodes still in the sequences, or just a separate barcode file (of a specific file type)?

Cheers,
Xin

Hi @xin,

You either need a separate file which contains the barcodes or you need the barcodes in the sequences. But, you need a way to associate each sequence with their barcode.

Best,
Justine

Hi @jwdebelius,

Will a file like this with the barcode information work? Or does it have to be a different kind of file that has lines of barcodes corresponding to each line of sequences?

Cheers,
Xin

1 Like

Hi @xin,

This is half of what you need. You also either need the barcode in the sequence header (which I don’t see), the barcode in the sequence itself, or you need a fastq file which contains the barcodes.

Best,
Justine

Hi @jwdebelius,

I see… I will try to obtain this files. Thank you for letting me know!

Meanwhile, is it okay if I ask, does it mean that once I have these files, I should be able to upload them using the manifest format? Because you previously mentioned that my file should not be brought in via the manifest format, but I am not sure what other file types am I suppose to use?

Cheers,
Xin

1 Like

Hi @xin,

Once you have the files, you need to import them as a multiplexed format. Check out the import tutorial for the EMP format, which is multiplexed.

Best,
Justine

Hi @jwdebelius,

I think I am starting to understand a little better now. Thank you so much for helping me with it. I will attempt to get the necessary files and try to import the fastq into qiime2 again.

Thank you so much once again!

Cheers,
Xin

1 Like

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.