error in demux summarize

User Support
,
1

Hi, everyone, I am Mavis.

Today I met an error when I want to visualize my demultiplexed data imported by manifest. And I don’t know how to deal with it, so I come here to ask for help.

The story happened when I use these command:

qiime tools import 
--type 'SampleData[PairedEndSequencesWithQuality]' 
--input-path  '/home/qiime2/16s lab/manifest.txt'  
--output-path lab_row_data.qza 
--input-format PairedEndFastqManifestPhred33V2

Imported /home/qiime2/16s lab/manifest.txt as PairedEndFastqManifestPhred33V2 to lab_row_data.qza

There is not error, the qza file was successfully generated. But when I want to visualize the qza file, something wrong happened.

My command is:

qiime demux summarize \
> --i-data  '/home/qiime2/16s lab/lab_row_data.qza'  \
> --o-visualization demux.qzv

And I got bug info like this:

Plugin error from demux:

  'utf-8' codec can't decode byte 0x8b in position 6764: invalid start byte

Debug info has been saved to /tmp/qiime2-q2cli-err-40wmy7vz.log

Then I less it and got:

Traceback (most recent call last):
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/q2cli/commands.py", line 327, in __call__
    results = action(**arguments)
  File "</home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/decorator.py:decorator-gen-440>", line 2, in summarize
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/sdk/action.py", line 240, in bound_callable
    output_types, provenance)
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/sdk/action.py", line 445, in _callable_executor_
    ret_val = self._callable(output_dir=temp_dir, **view_args)
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/q2_demux/_summarize/_visualizer.py", line 131, in summarize
    for seq in _read_fastq_seqs(file):
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/q2_demux/_demux.py", line 36, in _read_fastq_seqs
    for seq_header, seq, qual_header, qual in itertools.zip_longest(*[fh] * 4):
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 6764: invalid start byte

(END)

Before I come here to create topic, I saw there are others met the similar error, so I tried some methods, I will show you the command I used and the info I got:

echo $LC_ALL
C.UTF-8
echo $LANG
C.UTF-8
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8

It still didn’t work.

So I go on and tried:

qiime tools validate '/home/qiime2/16s lab/lab_row_data.qza'

What I got was:

Traceback (most recent call last):
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/q2cli/builtin/tools.py", line 411, in validate
    result.validate(level)
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/sdk/result.py", line 321, in validate
    self.format.validate(self.view(self.format), level)
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/plugin/model/directory_format.py", line 171, in validate
    getattr(self, field)._validate_members(collected_paths, level)
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/plugin/model/directory_format.py", line 101, in _validate_members
    self.format(path, mode='r').validate(level)
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/plugin/model/file_format.py", line 24, in validate
    self._validate_(level)
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/q2_types/per_sample_sequences/_format.py", line 279, in _validate_
    self._check_n_records(record_count_map[level])
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/q2_types/per_sample_sequences/_format.py", line 239, in _check_n_records
    for i, record in file_:
  File "/home/qiime2/miniconda/envs/qiime2-2019.7/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 6764: ordinal not in range(128)

An unexpected error has occurred while attempting to validate result /home/qiime2/16s lab/lab_row_data.qza:

  'ascii' codec can't decode byte 0x8b in position 6764: ordinal not in range(128)

See above for debug info.

And at last, I file one of my fastq.gz in the qza file exported by the manifest:

file '/home/qiime2/16s lab/111A_0_L001_R1_001.fastq.gz' 

/home/qiime2/16s lab/111A_0_L001_R1_001.fastq.gz: gzip compressed data, was "111A_0_L001_R1_001.fastq", last modified: Fri Sep 13 07:03:55 2019, max compression

gunzip '/home/qiime2/16s lab/111A_0_L001_R1_001.fastq.gz' 

file '/home/qiime2/16s lab/111A_0_L001_R1_001.fastq' 

/home/qiime2/16s lab/111A_0_L001_R1_001.fastq: ASCII text

Now I am really confused and don’t know what to do next.

Could anyone help me, please?

Thank you!

Mavis

2

Hi! Could you try to remove a space and quotes or replace a space in:

and run it like:
home/qiime2/16s_lab/lab_row_data.qza

3

Hi,Timanix,Sorry for the late reply.

And I have run it followed your words, but sadly the same error was gotten.

But still thanks for your suggestions which made me feel I am not alone :wink::100:

By the way, my classmate told me to check my raw data, I followed her suggestion and cat the data from different lanes again and have solved the error already! So I thought the error happened because of some error in my data. Although I still feel a litter confused on the reason why it can be import if there is error with raw data.

Thank you again sincerely!

Mavis

2 Likes
4

Hi, I am glade that you already solved your problem!
I am surprised to hear about the mistakes in row data, that never happened to me!
Good luck in your analysis

5

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.