problem with importing sequence data

Hello there, i have issue importing my fastq files using a manifest file i created, after i run the the code i encountered the following error:
(qiime2-2022.2) [email protected]:/media/sf_Qiime2$ qiime tools import \

--input-path manifest.csv
--type 'SampleData[SequencesWithQuality]'
--input-format SingleEndFastqManifestPhred33V2
--output-path se-demux.qza
Traceback (most recent call last):
File "/home/qiime2/miniconda/envs/qiime2-2022.2/lib/python3.8/site-packages/q2cli/builtin/tools.py", line 157, in import_data
artifact = qiime2.sdk.Artifact.import_data(type, input_path,
File "/home/qiime2/miniconda/envs/qiime2-2022.2/lib/python3.8/site-packages/qiime2/sdk/result.py", line 277, in import_data
return cls.from_view(type, view, view_type, provenance_capture,
File "/home/qiime2/miniconda/envs/qiime2-2022.2/lib/python3.8/site-packages/qiime2/sdk/result.py", line 305, in _from_view
result = transformation(view, validate_level)
File "/home/qiime2/miniconda/envs/qiime2-2022.2/lib/python3.8/site-packages/qiime2/core/transform.py", line 68, in transformation
self.validate(view, level=validate_level)
File "/home/qiime2/miniconda/envs/qiime2-2022.2/lib/python3.8/site-packages/qiime2/core/transform.py", line 143, in validate
view.validate(level)
File "/home/qiime2/miniconda/envs/qiime2-2022.2/lib/python3.8/site-packages/qiime2/plugin/model/file_format.py", line 26, in validate
self.validate(level)
File "/home/qiime2/miniconda/envs/qiime2-2022.2/lib/python3.8/site-packages/q2_types/per_sample_sequences/_format.py", line 40, in validate
md = qiime2.Metadata.load(str(self))
File "/home/qiime2/miniconda/envs/qiime2-2022.2/lib/python3.8/site-packages/qiime2/metadata/metadata.py", line 396, in load
return MetadataReader(filepath).read(
File "/home/qiime2/miniconda/envs/qiime2-2022.2/lib/python3.8/site-packages/qiime2/metadata/io.py", line 73, in read
header = self._read_header()
File "/home/qiime2/miniconda/envs/qiime2-2022.2/lib/python3.8/site-packages/qiime2/metadata/io.py", line 144, in _read_header
for row in self._reader:
File "/home/qiime2/miniconda/envs/qiime2-2022.2/lib/python3.8/site-packages/qiime2/metadata/io.py", line 71, in
self._reader = (self._strip_cell_whitespace(row)
_csv.Error: ' ' expected after '"'

An unexpected error has occurred:

' ' expected after '"'

See above for debug info.
please need your help

Hello @Zango,

Here's the important part of the error:

Looks like there is an issue with the manifest.csv file. Would you be willing to post the file here so we can take a look?

thank you for the reply, attached is the manifest file i used
manifest.csv (2.6 KB)

Thanks! I found the issue!

image
Looks like your columns are messed up, with your three column headers all bunched into column 1, for some reason...

Why? Well, if we open up the file:
image

Those three column headers are interpredted as being in a single column because they are "wrapped with quotes"

This works because it becomes three columns:
column1,column2,column3

This does not because it becomes 1 column:
"column1,column2,column3"

Take out those quotes and see if you can get the correct number of columns you want.

Also, check the [format examples[(Importing data — QIIME 2 2022.8.3 documentation). I'm not sure you need a direction column when you have single-end data with only one direction. :arrow_forward:

Thank you very much it was really helpful, however i encountred another problem after importing the data when i tried to visualize the sequence reads as follows
qiime2-2022.2) [email protected]:/media/sf_Qiime2$ qiime tools import \

--type 'SampleData[SequencesWithQuality]'
--input-path manifest2.txt
--output-path single-end-demux.qza
--input-format SingleEndFastqManifestPhred33V2
Imported manifest2.txt as SingleEndFastqManifestPhred33V2 to single-end-demux.qza
(qiime2-2022.2) [email protected]:/media/sf_Qiime2$ qiime demux summarize
--i-data single-end-demux.qza
--o-visualization se-demux.qzv
Plugin error from demux:

'utf-8' codec can't decode byte 0xc6 in position 4227: invalid continuation byte

Debug info has been saved to /tmp/qiime2-q2cli-err-jlzu50sn.log

I'm glad you got the data imported!

I searched for this error on the forums, and found this old thread between the two of us. It's the same issue:

How did you solve this last time?

I am still having the same issue after copying my data from UTD8 encoder multiple times


1 Like

Let's see if we can see the byte that's causing an issue:

dd if=manifest2.txt ibs=1 skip=4225 count=10

Try running that on your input metadata file.

Or you can post the manifest2.txt file here so we can take a look. :paperclip:

Thank you Sir, here is the manifest file i used
manifest2.txt (1.9 KB)

after running the above command (dd if=manifest2.txt ibs=1 skip=4225 count=10), this is what i got

1 Like

Ah, OK!

I bet the invalid byte is inside one of the .fastq files, and not the manifest file itself.

Can you rerun with the --verbose flag to get more information and hopefully see what file has that strange byte?

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.