EOFError: Compressed file ended before the end-of-stream marker was reached

User Support
1

Thanks, my manifest file is working now by creating a csv file (manifest) in window and save it as csv (MS DOS) or creating the manifest file using R code.
With the following syntax:
qiime tools import
–type ‘SampleData[PairedEndSequencesWithQuality]’
–input-path csv2
–output-path paired30.qza
–source-format PairedEndFastqManifestPhred33

I have imported my raw paired sequencing data with the output file “paired30.qza”
Then, with the following syntax:
qiime demux summarize
–i-data paired30.qza
–o-visualization demux30.qzv

I have the following error:
Plugin error from demux:
Compressed file ended before the end-of-stream marker was reached
Debug info has been saved to /tmp/qiime2-q2cli-err-qper0vdr.log

The following is the texts from the log file:

Traceback (most recent call last):
File “/home/213002h/miniconda3/envs/qiime2-2018.4/lib/python3.5/site-packages/q2cli/commands.py”, line 274, in call
results = action(**arguments)
File “”, line 2, in summarize
File “/home/213002h/miniconda3/envs/qiime2-2018.4/lib/python3.5/site-packages/qiime2/sdk/action.py”, line 231, in bound_callable
output_types, provenance)
File “/home/213002h/miniconda3/envs/qiime2-2018.4/lib/python3.5/site-packages/qiime2/sdk/action.py”, line 428, in callable_executor
ret_val = self._callable(output_dir=temp_dir, **view_args)
File “/home/213002h/miniconda3/envs/qiime2-2018.4/lib/python3.5/site-packages/q2_demux/_summarize/_visualizer.py”, line 117, in summarize
for seq in _read_fastq_seqs(file):
File “/home/213002h/miniconda3/envs/qiime2-2018.4/lib/python3.5/site-packages/q2_demux/_demux.py”, line 34, in _read_fastq_seqs
for seq_header, seq, qual_header, qual in itertools.zip_longest(*[fh] * 4):
File “/home/213002h/miniconda3/envs/qiime2-2018.4/lib/python3.5/gzip.py”, line 287, in read1
return self._buffer.read1(size)
File “/home/213002h/miniconda3/envs/qiime2-2018.4/lib/python3.5/_compression.py”, line 68, in readinto
data = self.read(len(byte_view))
File “/home/213002h/miniconda3/envs/qiime2-2018.4/lib/python3.5/gzip.py”, line 480, in read
raise EOFError("Compressed file ended before the "
EOFError: Compressed file ended before the end-of-stream marker was reached

I thought that the error may come from my raw data. So I imported raw sequencing data divided by groups (such as first 10 sample, then 11 to 20 …). Re-run “qiime demux summarize” for each group. The error message appeared for some groups. However after further dividing the raw data group to identify the raw sequencing data with problem I cannot find any raw sequencing data with the problem. I am confused. Thanks, Brad

3

This non-QIIME error usually occurs if the file became corrupted, e.g., during gzipping your fastq sequences.

Does that mean that you tried dividing and re-importing the sequences and the error no longer occurred? That would make sense if the initial gzip file became corrupted.

You could also try qiime tools validate paired30.qza to check that file before demultiplexing.

Please let us know if you are still having trouble. If so, please post the output of validate.

5

Thanks, Nicholas_Bokulich. I have identified which file is corrupted. After excluding these files I am doing well with qiime2.

Brad

2 Likes
6

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.