error in demux summarize

Hi, everyone, I am Mavis.

Today I met an error when I want to visualize my demultiplexed data imported by manifest. And I don’t know how to deal with it, so I come here to ask for help.

The story happened when I use these command:

qiime tools import 
--type 'SampleData[PairedEndSequencesWithQuality]' 
--input-path  '/home/qiime2/16s lab/manifest.txt'  
--output-path lab_row_data.qza 
--input-format PairedEndFastqManifestPhred33V2

Imported /home/qiime2/16s lab/manifest.txt as PairedEndFastqManifestPhred33V2 to lab_row_data.qza

There is not error, the qza file was successfully generated. But when I want to visualize the qza file, something wrong happened.

My command is:

qiime demux summarize \
> --i-data  '/home/qiime2/16s lab/lab_row_data.qza'  \
> --o-visualization demux.qzv

And I got bug info like this:

Plugin error from demux:

  'utf-8' codec can't decode byte 0x8b in position 6764: invalid start byte

Debug info has been saved to /tmp/qiime2-q2cli-err-40wmy7vz.log

Then I less it and got:

Traceback (most recent call last):
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/q2cli/commands.py", line 327, in __call__
    results = action(**arguments)
  File "</home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/decorator.py:decorator-gen-440>", line 2, in summarize
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/sdk/action.py", line 240, in bound_callable
    output_types, provenance)
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/sdk/action.py", line 445, in _callable_executor_
    ret_val = self._callable(output_dir=temp_dir, **view_args)
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/q2_demux/_summarize/_visualizer.py", line 131, in summarize
    for seq in _read_fastq_seqs(file):
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/q2_demux/_demux.py", line 36, in _read_fastq_seqs
    for seq_header, seq, qual_header, qual in itertools.zip_longest(*[fh] * 4):
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 6764: invalid start byte

(END)

Before I come here to create topic, I saw there are others met the similar error, so I tried some methods, I will show you the command I used and the info I got:

echo $LC_ALL
C.UTF-8
echo $LANG
C.UTF-8
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8

It still didn’t work.

So I go on and tried:

qiime tools validate '/home/qiime2/16s lab/lab_row_data.qza'

What I got was:

Traceback (most recent call last):
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/q2cli/builtin/tools.py", line 411, in validate
    result.validate(level)
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/sdk/result.py", line 321, in validate
    self.format.validate(self.view(self.format), level)
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/plugin/model/directory_format.py", line 171, in validate
    getattr(self, field)._validate_members(collected_paths, level)
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/plugin/model/directory_format.py", line 101, in _validate_members
    self.format(path, mode='r').validate(level)
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/plugin/model/file_format.py", line 24, in validate
    self._validate_(level)
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/q2_types/per_sample_sequences/_format.py", line 279, in _validate_
    self._check_n_records(record_count_map[level])
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/q2_types/per_sample_sequences/_format.py", line 239, in _check_n_records
    for i, record in file_:
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 6764: ordinal not in range(128)

An unexpected error has occurred while attempting to validate result /home/qiime2/16s lab/lab_row_data.qza:

  'ascii' codec can't decode byte 0x8b in position 6764: ordinal not in range(128)

See above for debug info.

And at last, I file one of my fastq.gz in the qza file exported by the manifest:

file '/home/qiime2/16s lab/111A_0_L001_R1_001.fastq.gz' 

/home/qiime2/16s lab/111A_0_L001_R1_001.fastq.gz: gzip compressed data, was "111A_0_L001_R1_001.fastq", last modified: Fri Sep 13 07:03:55 2019, max compression

gunzip '/home/qiime2/16s lab/111A_0_L001_R1_001.fastq.gz' 

file '/home/qiime2/16s lab/111A_0_L001_R1_001.fastq' 

/home/qiime2/16s lab/111A_0_L001_R1_001.fastq: ASCII text

Now I am really confused and don’t know what to do next.

Could anyone help me, please?

Thank you!

Mavis

Hi! Could you try to remove a space and quotes or replace a space in:

and run it like:
home/qiime2/16s_lab/lab_row_data.qza

Hi,Timanix,Sorry for the late reply.

And I have run it followed your words, but sadly the same error was gotten.

But still thanks for your suggestions which made me feel I am not alone :wink::100:

By the way, my classmate told me to check my raw data, I followed her suggestion and cat the data from different lanes again and have solved the error already! So I thought the error happened because of some error in my data. Although I still feel a litter confused on the reason why it can be import if there is error with raw data.

Thank you again sincerely!

Mavis

2 Likes

Hi, I am glade that you already solved your problem!
I am surprised to hear about the mistakes in row data, that never happened to me!
Good luck in your analysis

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.