Error importing fastq files using “Fastq manifest” formats

[EGA (Edisa)]

Hi QIIME2 team,
I have a persistent error when I try to import single-end using SingleEndFastqManifestPhred3.

command:

qiime tools import
--type 'SampleData[SequencesWithQuality]'
--input-path manifest.txt
--output-path single-end-demux.qza
--source-format SingleEndFastqManifestPhred33

Error:

/software/software/QIIME2/2018.2/lib/python3.5/site-packages/matplotlib/init.py:962: UserWarning: Duplicate key in file "/home/p277936/.config/matplotlib/matplotlibrc", line #17
(fname, cnt))
There was a problem importing manifest.txt:

/local/1498484/q2-SingleLanePerSampleSingleEndFastqDirFmt-_1v7m77v/CH3E_50_L001_R1_001.fastq.gz is not a(n) FastqGzFormat file:

Missing sequence for record beginning on line 5

Manifest file: NOTE: they dont even are named as in the error appears

how fastqfile looks like

Could someone help explain what is happening?
If you requiired mor details please et me know
Thanks in advance!!

[thermokarst]

Hey there @EGA!

@EGA

Could someone help explain what is happening?

I will try!

@EGA

/local/1498484/q2-SingleLanePerSampleSingleEndFastqDirFmt-_1v7m77v/CH3E_50_L001_R1_001.fastq.gz is not a(n) FastqGzFormat file:

Missing sequence for record beginning on line 5

Let's break down the error message: in the file for the forwards reads of sample CH3E, the fastq record beginning on line 5 is missing sequence data.

Is there screenshot you provided for sample CH3E? If not, can you send along a screenshot for that sample?

@EGA

Manifest file: NOTE: they dont even are named as in the error appears

They look right to me - you specified the sample ID is CH3E, and that was is what is reflected in the filename during import (CH3E_50_L001_R1_001.fastq.gz). The first part of the filename is the sample ID. The rest is described here: Importing data — QIIME 2 2018.6.0 documentation

The underscore-separated fields in this file name are the sample identifier, the barcode sequence or a barcode identifier, the lane number, the read number, and the set number.

Basically, everything besides the sample ID and the read direction don't really serve a purpose at this stage (and on) in an analysis.

Hope that helps, looking forward to your reply!

[EGA (Edisa)]

Hi thermokarst,
Thanks for your reply!
Here the screenshot.

I see some empty sequences. For trimming the barcodes and demultiplexing I used the -cutadapt command maybe it is causing the problem?

Looking forward to your reply

[thermokarst]

Ah ha!

@EGA

I used the -cutadapt command maybe it is causing the problem?

Yep!

Check out this post for a workaround!

Cutadapt thinks my fastq.gz files aren't fastq.gz

Woohoo!!! !!! With that little bit of data you provided, I was able to reproduce the error. Not only that, but I was able to see () exactly what the issue is. Running the same command you provided above, but instead of using q2-cutadapt we go through cutadapt directly, I get some reads out of cutadapt that look like this: [00%20PM] Yowzah! So, hopefully that QIIME 2 error makes a bit more sense now --- it was complaining about records that had no sequence as…

Let us know if you need help with any of that!