problem with importing sequence data

[Zango (Sauban Musa Jibril)]

Hello there, i have issue importing my fastq files using a manifest file i created, after i run the the code i encountered the following error:
(qiime2-2022.2) qiime2@qiime2core2022-2:/media/sf_Qiime2$ qiime tools import \

--input-path manifest.csv
--type 'SampleData[SequencesWithQuality]'
--input-format SingleEndFastqManifestPhred33V2
--output-path se-demux.qza
Traceback (most recent call last):
File "/home/qiime2/miniconda/envs/qiime2-2022.2/lib/python3.8/site-packages/q2cli/builtin/tools.py", line 157, in import_data
artifact = qiime2.sdk.Artifact.import_data(type, input_path,
File "/home/qiime2/miniconda/envs/qiime2-2022.2/lib/python3.8/site-packages/qiime2/sdk/result.py", line 277, in import_data
return cls.from_view(type, view, view_type, provenance_capture,
File "/home/qiime2/miniconda/envs/qiime2-2022.2/lib/python3.8/site-packages/qiime2/sdk/result.py", line 305, in _from_view
result = transformation(view, validate_level)
File "/home/qiime2/miniconda/envs/qiime2-2022.2/lib/python3.8/site-packages/qiime2/core/transform.py", line 68, in transformation
self.validate(view, level=validate_level)
File "/home/qiime2/miniconda/envs/qiime2-2022.2/lib/python3.8/site-packages/qiime2/core/transform.py", line 143, in validate
view.validate(level)
File "/home/qiime2/miniconda/envs/qiime2-2022.2/lib/python3.8/site-packages/qiime2/plugin/model/file_format.py", line 26, in validate
self.validate(level)
File "/home/qiime2/miniconda/envs/qiime2-2022.2/lib/python3.8/site-packages/q2_types/per_sample_sequences/_format.py", line 40, in validate
md = qiime2.Metadata.load(str(self))
File "/home/qiime2/miniconda/envs/qiime2-2022.2/lib/python3.8/site-packages/qiime2/metadata/metadata.py", line 396, in load
return MetadataReader(filepath).read(
File "/home/qiime2/miniconda/envs/qiime2-2022.2/lib/python3.8/site-packages/qiime2/metadata/io.py", line 73, in read
header = self._read_header()
File "/home/qiime2/miniconda/envs/qiime2-2022.2/lib/python3.8/site-packages/qiime2/metadata/io.py", line 144, in _read_header
for row in self._reader:
File "/home/qiime2/miniconda/envs/qiime2-2022.2/lib/python3.8/site-packages/qiime2/metadata/io.py", line 71, in
self._reader = (self._strip_cell_whitespace(row)
_csv.Error: ' ' expected after '"'

An unexpected error has occurred:

' ' expected after '"'

See above for debug info.
please need your help

[colinbrislawn (Colin J Brislawn)]

Hello @Zango,

Here's the important part of the error:

@Zango

_csv.Error: ' ' expected after '"'

Looks like there is an issue with the manifest.csv file. Would you be willing to post the file here so we can take a look?

[Zango (Sauban Musa Jibril)]

thank you for the reply, attached is the manifest file i used
manifest.csv (2.6 KB)

[colinbrislawn (Colin J Brislawn)]

Thanks! I found the issue!

Looks like your columns are messed up, with your three column headers all bunched into column 1, for some reason...

Why? Well, if we open up the file:

Those three column headers are interpredted as being in a single column because they are "wrapped with quotes"

This works because it becomes three columns:
column1,column2,column3

This does not because it becomes 1 column:
"column1,column2,column3"

Take out those quotes and see if you can get the correct number of columns you want.

Also, check the [format examples[(Importing data — QIIME 2 2022.8.3 documentation). I'm not sure you need a direction column when you have single-end data with only one direction.

[Zango (Sauban Musa Jibril)]

Thank you very much it was really helpful, however i encountred another problem after importing the data when i tried to visualize the sequence reads as follows
qiime2-2022.2) qiime2@qiime2core2022-2:/media/sf_Qiime2$ qiime tools import \

--type 'SampleData[SequencesWithQuality]'
--input-path manifest2.txt
--output-path single-end-demux.qza
--input-format SingleEndFastqManifestPhred33V2
Imported manifest2.txt as SingleEndFastqManifestPhred33V2 to single-end-demux.qza
(qiime2-2022.2) qiime2@qiime2core2022-2:/media/sf_Qiime2$ qiime demux summarize
--i-data single-end-demux.qza
--o-visualization se-demux.qzv
Plugin error from demux:

'utf-8' codec can't decode byte 0xc6 in position 4227: invalid continuation byte

Debug info has been saved to /tmp/qiime2-q2cli-err-jlzu50sn.log

[colinbrislawn (Colin J Brislawn)]

I'm glad you got the data imported!

@Zango

'utf-8' codec can't decode byte 0xc6 in position 4227: invalid continuation byte

I searched for this error on the forums, and found this old thread between the two of us. It's the same issue:

qiime demux summarize error

Hello Sir, I imported my single sequence fastq files using manifest file with the following command qiime tools import \ --type 'SampleData[SequencesWithQuality]' --input-path manifest.csv --output-path single-end-demux.qza --input-format SingleEndFastqManifestPhred33 But when i tried to summarize the .qza file i got the following error; qiime demux summarize --i-data single-end-demux.qza --o-visualization single-end-demux.qzv Plugin error from demux: 'utf-8' codec can't decode byt…

How did you solve this last time?

[Zango (Sauban Musa Jibril)]

I am still having the same issue after copying my data from UTD8 encoder multiple times

[colinbrislawn (Colin J Brislawn)]

Let's see if we can see the byte that's causing an issue:

dd if=manifest2.txt ibs=1 skip=4225 count=10

Try running that on your input metadata file.

Or you can post the manifest2.txt file here so we can take a look.

[Zango (Sauban Musa Jibril)]

Thank you Sir, here is the manifest file i used
manifest2.txt (1.9 KB)

[Zango (Sauban Musa Jibril)]

@colinbrislawn

dd if=manifest2.txt ibs=1 skip=4225 count=10

after running the above command (dd if=manifest2.txt ibs=1 skip=4225 count=10), this is what i got

[colinbrislawn (Colin J Brislawn)]

Ah, OK!

I bet the invalid byte is inside one of the .fastq files, and not the manifest file itself.

Can you rerun with the --verbose flag to get more information and hopefully see what file has that strange byte?