Importing Fastq manifest format

[jmb]

Hi,
I am trying to import demultiplexed Fastq.gz files using a manifest but keep getting this error:

There was a problem importing Manifest.csv:
Manifest.csv is not a(n) PairedEndFastqManifestPhred33 file

Here is what I am entering:

qiime tools import \
> --type 'SampleData[PairedEndSequencesWithQuality]' \
> --input-path Manifest.csv \
> --output-path import_results.qza \
> --source-format PairedEndFastqManifestPhred33

Although I am not sure which format my files are in, I get the same error whether I use Phred33 or Phred64. It appears I cannot upload a .csv file (created in excel) here but this is what it looks like:

sample-id,absolute-filepath,direction
A1C,/Users/xx/Documents/Aberts/16sData/sequences/A1C_1.fq.gz,forward
A1C,/Users/xx/Documents/Aberts/16sData/sequences/A1C_2.fq.gz,reverse
A1FS,/Users/xx/Documents/Aberts/16sData/sequences/A1FS_1.fq.gz,forward
A1FS,/Users/xx/Documents/Aberts/16sData/sequences/A1FS_2.fq.gz,reverse
A1LI1,/Users/xx/Documents/Aberts/16sData/sequences/A1LI1_1.fq.gz,forward
A1LI1,/Users/xx/Documents/Aberts/16sData/sequences/A1LI1_2.fq.gz,reverse
A1LI2,/Users/xx/Documents/Aberts/16sData/sequences/A1LI2_1.fq.gz,forward

Also, in case it helps, here are the first few lines of one of the Fastq files:

@FCHVLF2BCXX:2:1101:2953:2250#GTAATGAC_TTCGGATT/1
TACGTAGGTGGCAAGCGTTATCCGGATTTATTGGGCGTAAAGCGAGCGCAGGCGGTTGCTTAGGTCTGATGTGAAAGCCTTCGGCTTAACCGAAGAAGGGCATCGGAAACCGGGCGACTTGAGTGCAGAAGAGGACAGTGGAACTCCATGTGTAGCGGTGGAATGCGTAGATATATGGAAGAACACCAGTGGCGAAGGCGGCTGTCTGGTCTGCAACTGACGCTGAGGCTCGAAAGCATGGGTAGCGAAC
+
cccccefgghhh_gghhhhhhhhhhhgfgghhhhghhhhhhhhhhhghgghgghhghhhghhhhghhhhhhhhgghghghghhhhhhggghhhhfhhhhhhgghhhhhhggghhhhhhgghgfghhhhhhhhhhhhhhhhhhhhhghfgggggeghhhfgggdghhghggcfghghhhhgccfgghhhhhhhhhhhhhhgghhgegbgghghhhgghhgggghhhghghhhhgggghgghhhg_Xeggbg
@FCHVLF2BCXX:2:1101:17968:2459#GTAATGAC_TTCGGATT/1
TACGTAGGTGGCAAGCGTTATCCGGATTTATTGGGCGTAAAGGGAACGCAGGCGGTCTTTTAAGTCTGATGTGAAAGCCTTCGGCTTAACCGGAGTAGTGCATTGGAAACTGGGAGACTTGAGTGCAGAAGAGGAGAGTGGAACTCCATGTGTAGCGGTGAAATGCGTAGATATATGGAAGAACACCAGTGGCGAAAGCGGCTCTCTGGTCTGTAACTGACGCTGAGGTTCGAAAGCGTGGGTAGCAAAC
+
cccccegghhghhhgghhhhhfhhhghhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhghhhhhhghhhhhhhhhhhhhhhghhhhhhhhhghhhhheghhhgghhhhhhghhhhhhgghhhhhhhghhhhhhhhhhghghhhhhhgghghhhgdeceghghhhge\cfghhhhhhhghhhfhhhhhhhhhb[eghhhhghhhhhhhhhhhhhdg_fgghhffgghhfbgM^fghghhcfddgggf

Thank you, I appreciate any input I receive!

[thermokarst]

Thanks for all the supporting documentation, @jmb, it was really helpful!

My guess right now is that there is more to your CSV manifest file (as in, more samples listed below, you just didn’t include them here) — we have seen this situation in the past where Excel adds extra blank rows or columns in strange places (just the first thought that comes to mind).

Would it be able to try to re-attach that CSV file here? I updated the forum settings to permit posting of CSV files. Thanks!

[jmb]

Thank you so much for your prompt response--I am attaching the file. Just a little bit more background of how I generated the file, in case it helps diagnose the problem. I originally entered the information for the three fields (sample-id, absolute-file path,direction) in different excel cells and then used the concatenate command to put them together into one cell. I used paste special to paste the values of each cell into a new sheet, which I then saved as a .csv. Manifest.csv (9.1 KB)

[thermokarst]

Ah ha! Check this out:

Each of your lines in this file is quoted (as in, wrapped in "). That is causing the CSV parsing to go crazy --- the parser doesn't look inside the quotes, because that is typically used to denote values within a CSV "cell". If you drop those quotes you should be good to go:

Good luck!!

[jmb]

Thank you so much. I’m not sure why excel was adding the quotes, but deleting them fixed the problem. I appreciate all of your time and effort on this forum.