Compressed file ended before the end-of-stream marker was reached

[chacha]

I’m new to QIIME2 and I am trying to do the “demux summarize” after importing my fastq files but I have this error:

"Compressed file ended before the end-of-stream marker was reached"

My command was:
qiime demux summarize \

--i-data demux_bathymetrie.qza
--o-visualization demux_bathymetrie.qzv

Any suggestions?

[thermokarst]

Hi @chacha! It sounds like maybe one or more of your fastq.gz files is broken. Can you run the following and send us the output?

qiime tools validate demux_bathymetrie.qza

Thanks!

[chacha]

Hi, thanks for your answer,

This is was I get :

[thermokarst]

Ah, that is a bummer, @chacha. Let’s check the files outside of QIIME 2:

qiime tools export demux_bathymetrie.qza --output-dir debug
for f in debug/*.fastq.gz; do gunzip -t -v $f; done

Can you run the commands above and copy-and-paste the results here? This will export your demux reads, then it will use a tool called gunzip to test the validity of all of the fastq.gz files (it won’t test the validity of the data, just of the file compression, which seems to be where the issue is).

Did you import demultiplexed sequences into QIIME 2? Or, did you use QIIME 2 to demux you reads?

Thanks!

[chacha]

Hi @thermokarst,

Ok I see,
How can I fix the problem?

I imported my demultiplexed sequences …

eduroam-10-33-32-250:test1 cha$ for f in debug/*.fastq.gz; do gunzip -t -v $f; done

debug/Moorea16_001_0_L001_R1_001.fastq.gz: OK
debug/Moorea16_001_1_L001_R2_001.fastq.gz: OK
debug/Moorea16_002_2_L001_R1_001.fastq.gz: OK
debug/Moorea16_002_3_L001_R2_001.fastq.gz: OK
debug/Moorea16_003_4_L001_R1_001.fastq.gz: OK
debug/Moorea16_003_5_L001_R2_001.fastq.gz: OK
debug/Moorea16_004_6_L001_R1_001.fastq.gz: OK
debug/Moorea16_004_7_L001_R2_001.fastq.gz: OK
debug/Moorea16_005_8_L001_R1_001.fastq.gz: OK
debug/Moorea16_005_9_L001_R2_001.fastq.gz: OK
debug/Moorea16_006_10_L001_R1_001.fastq.gz: OK
debug/Moorea16_006_11_L001_R2_001.fastq.gz: OK
debug/Moorea16_007_12_L001_R1_001.fastq.gz: OK
debug/Moorea16_007_13_L001_R2_001.fastq.gz: OK
debug/Moorea16_008_14_L001_R1_001.fastq.gz: OK
debug/Moorea16_008_15_L001_R2_001.fastq.gz: OK
debug/Moorea16_009_16_L001_R1_001.fastq.gz: OK
debug/Moorea16_009_17_L001_R2_001.fastq.gz: OK
debug/Moorea16_010_18_L001_R1_001.fastq.gz: OK
debug/Moorea16_010_19_L001_R2_001.fastq.gz: OK
debug/Moorea16_011_20_L001_R1_001.fastq.gz: OK
debug/Moorea16_011_21_L001_R2_001.fastq.gz: OK
debug/Moorea16_012_22_L001_R1_001.fastq.gz: OK
debug/Moorea16_012_23_L001_R2_001.fastq.gz: OK
debug/Moorea16_013_24_L001_R1_001.fastq.gz: OK
gunzip: debug/Moorea16_013_25_L001_R2_001.fastq.gz: unexpected end of file
gunzip: debug/Moorea16_013_25_L001_R2_001.fastq.gz: uncompress failed
debug/Moorea16_013_25_L001_R2_001.fastq.gz: NOT OK
debug/Moorea16_014_26_L001_R1_001.fastq.gz: OK
gunzip: debug/Moorea16_014_27_L001_R2_001.fastq.gz: unexpected end of file
gunzip: debug/Moorea16_014_27_L001_R2_001.fastq.gz: uncompress failed
debug/Moorea16_014_27_L001_R2_001.fastq.gz: NOT OK
gunzip: debug/Moorea16_015_28_L001_R1_001.fastq.gz: unexpected end of file
gunzip: debug/Moorea16_015_28_L001_R1_001.fastq.gz: uncompress failed
debug/Moorea16_015_28_L001_R1_001.fastq.gz: NOT OK
gunzip: debug/Moorea16_015_29_L001_R2_001.fastq.gz: unexpected end of file
gunzip: debug/Moorea16_015_29_L001_R2_001.fastq.gz: uncompress failed
debug/Moorea16_015_29_L001_R2_001.fastq.gz: NOT OK
debug/Moorea16_016_30_L001_R1_001.fastq.gz: OK
debug/Moorea16_016_31_L001_R2_001.fastq.gz: OK
debug/Moorea16_017_32_L001_R1_001.fastq.gz: OK
debug/Moorea16_017_33_L001_R2_001.fastq.gz: OK
debug/Moorea16_018_34_L001_R1_001.fastq.gz: OK
debug/Moorea16_018_35_L001_R2_001.fastq.gz: OK
debug/Moorea16_019_36_L001_R1_001.fastq.gz: OK
debug/Moorea16_019_37_L001_R2_001.fastq.gz: OK
debug/Moorea16_020_38_L001_R1_001.fastq.gz: OK
debug/Moorea16_020_39_L001_R2_001.fastq.gz: OK
debug/Moorea16_021_40_L001_R1_001.fastq.gz: OK
debug/Moorea16_021_41_L001_R2_001.fastq.gz: OK
debug/Moorea16_022_42_L001_R1_001.fastq.gz: OK
debug/Moorea16_022_43_L001_R2_001.fastq.gz: OK
debug/Moorea16_023_44_L001_R1_001.fastq.gz: OK
debug/Moorea16_023_45_L001_R2_001.fastq.gz: OK
debug/Moorea16_024_46_L001_R1_001.fastq.gz: OK
debug/Moorea16_024_47_L001_R2_001.fastq.gz: OK

[thermokarst]

Hi @chacha! Do you still have the source directory that has your original demuxed reads in it? This would be the directory that you imported into QIIME 2. If so:

cd $DIRECTORY_WITH_CHACHAS_ORIGINAL_DATA
for f in *.fastq.gz; do gunzip -t -v $f; done

If the source files weren’t zipped (ending in fastq.gz), then that step won’t work. If that is the case, go ahead and re-import your data - it is possible that something went crazy during the first import. If the above command does work, please copy and paste the results here. If everything is good, then it means something went wrong when importing, so you could try to import again. Thanks!

[chacha]

Hi @thermokarst !

Okay I re-imported my data and it was that !

Now It’s good

Thanks !!