error while running QIIME demux summary

User Support
1

Hi,

I am new to QIIME. I have a dataset downloaded from SRA, where each folder for each sample. No barcode file. So is the data demultiplexed?
This is how the sequences in each file look like,
@SRR9680011.1.1 length=441

TGGGGAATATTGCACAATGGGCGCAAGCCTGATGCAGCCATGCCGCGTGTATGAAGAAGGCCTTCGGGTTGTAAAGTACTTTCAGCGGGGAGGAAGGGAGTAAAGTTAATACCTTTGCTCATTGACGTTACACGCAGATGAAGCACCGGCTAACTCCGTGCCAGCAGCCGCGGTAATACGGAGGGTGCAAGCGTTAAGCGGAATTACTGGGCGTAAAGCGCACGCAGGCGGTTGGTTAAGTCAGATGTGAAATCCCCGGGCTCAACCGGGGAACTGCATCTGATACGGGAAAGCTGGAGTCTCGTAGAGGGGGGTAGAATGCCAGGTGTAGCGGTGAAATGCGTAGAGATCTGGAGGAATACCGGTGGCGAAGGCGGCCCCCTGGACGAAGACTGACGCTCAGGTGCGAAAGCGTGGGGAGCAAACAGGATTAGATACCCT

+SRR9680011.1.1 length=441

EFFFGG7F@EFFCAGFFFG8FFGGGD7DFCGGGFGDEGFDFFGGG7FFGFFGGGGCFGFG8EFGFCGEFGGGGFGFEDFGGEFFFGGGCFBF=CE<F+4BFADC9FF,C<FFA<F<BDGFF==9DFFB+3>,>3CCGG,3,@FDEFFCGD@1@E$F:FEFEEFFGGFGGDDCEC@EGC>;3B>=FFGGGGEFGGGCC#@EE)EC80A:??DCE/C>8>+:55CG;=CDEF>&C#CCFGFF@F=F7EFDB,?+<7CC8?F?>9(E'F<76GGDGGFF9,?FDGBEFB,,@,@DD<@,<D@++77FF8C7C+=DEGGDGFB4,,F?GGFGGEGGGGFCFGGEBGGFFE;A9,A=CCFFE?DC=F@6:CFC<FGGEECCFFF7FCF@C6<C9FFFFCGGGFBE@GEEGGGGGGEF7F,FF@FFCFEGGGFFGFF,GGGGCCCCC

@SRR9680011.2.1 length=441

TGGGGAATATTGCACAATGGGCGCAAGCCTGATGCAGCCATGCCGCGTGTATGAAGAAGGCCTTCGGGTTGTAAAGTACTTTCAGCGGGGAGGAAGGGAGTAAAGTTAATACCTTTGCTCATTGACGTTACCCGCAGAAGAAGCACCGGCTAACTCCGTGCCAGCAGCCGCGGTAATACGGAGGGTGCAAGCGTTAATCGGAATTACTGGGCGTAAAGCGCACGCAGGCGGTTTGTTAAGTCAGATGTGAAATCCCCGGGCTCAACCTGGGAACTGCATCTGATACTGGCAAGCTTGAGTCTCGTAGAGGGGGGTAGAATTCCAGGTGTAGCGGTGAAATGCGTAGAGATCTGGAGGAATACCGGTGGCGAAGGCGGCCCCCTGGACGAAGACTGACGCTCAGGTGCGAAAGCGTGGGGAGCAAACAGGATTAGATACCCT

+SRR9680011.2.1 length=441

The library layout is shown as SINGLE

I imported just 2 files coresponding to 2 samples using the command
qiime tools import --type 'SampleData[SequencesWithQuality]' --input-path manifestFile --output-path single-end-demux.qza --input-format SingleEndFastqManifestPhred33

my manifest file is
sample-id,absolute-filepath,direction
SRR9679959,/Users/syama/Downloads/sequences/SRR9679959.fastq,forward
SRR9680011,/Users/syama/Downloads/sequences/SRR9680011.fastq,forward

It successfully run and I got single-end-demux.qza

But when I tried demux summary using
qiime demux summarize \
--i-data single-end-demux.qza
--o-visualization demux.qzv

I got the error Plugin error from demux:

'NoneType' object has no attribute 'strip'

Debug info has been saved to /var/folders/hr/9vcxtv2961bgx43wk4sqww3m0000gn/T/qiime2-q2cli-err-pdlqug93.log

I could not find the above said log file also.
While I open single-end-demux.qza using vi editor it is like,

P??P?G?2c150c2ea-4422-40b4-b3d0-c3d81d9fd4b9/metadata.yaml%˱? ?ᝧ?H1??֩C????- P>???P??PlÃ?4;2c150c2ea-4422-40b4-b3d0-c3d81d9fd4b9/checksums.md5???jA
????k?[?jsp?۰Ԓ:,????<}fɆx!???R_?e?fHñ??v?Q?;Oӗ?????nS??%lR?R??b-y?D??󘦇X?u??w}?????z)?h(?y`H?R?4??W??n>^??o???"T??dV??J?Aq??w;?,Rd???;?W?}]??lx??
b,?[ࠜ?uf??>??$?????"
CSf?4y???9?O?S??????G=Z???=?@@??)?7<p?5?^@vXt9<?????+??? ?[ư5???{cvNE??/P??ͳ??????(,c150c2ea-4422-40b4-b3d0-c3d81d9fd4b9/VERSION
???uU0?J,J??,K?R0?J+J?M-?/ʶR020??34?3?P??P?G?=c150c2ea-4422-40b4-b3d0-c3d81d9fd4b9/provenance/metadata.yaml%˱? >???P??PZ@?????7G?????????Lׄ#??*? ?1ZK????A%]?????+??q
=c150c2ea-4422-40b4-b3d0-c3d81d9fd4b9/provenance/citations.bib}VKv??s??ز%???BQ??L2??]?r b?1>?(ۇ?2???K5???M6Q?h????;Y/?J|[X?œ???_???????????^o??^g???+c?ߊo'F

are the fastq files properly imported?
I am using Mac Os Catalina version 10.15.4 (19E287)
qiime2-2019.10,
R version 3.5.1

It would be great if someone has the answer.
Tnanks

2

Hi Syama,
I'm not sure about the error, it may be easier to debug if you can post the contents of the log file

Debug info has been saved to /var/folders/hr/9vcxtv2961bgx43wk4sqww3m0000gn/T/qiime2-q2cli-err-pdlqug93.log

FYI your data looks like it is paired end data that was cleaned and merged before uploading to NCBI, so you may want to use deblur rather than dada2 for the denoising.

Devin

3

Hello @syama. In the samples you pasted it looks like there are a lot of empty lines in your file separating all of your actual data lines. Are these lines actually present in your file? If so that could be the problem.

It also looks like you have your sample ids repeated on your + separator lines. In all honesty I'm not sure whether that's allowed or not, but I don't think I've ever seen it before. It definitely breaks the spec for fastq files laid out here where it says the separator is just a +. EDIT: It looks like having the sample id after the plus is valid.

The reason the .qza opened in vi looks so goofy is that the .qza itself isn't a text file, it's a zipped directory. It's not surprising that all the compressed data when viewed directly with a text editor just looks like nonsense.

Thank you.

5

@Devin thomas,

I did beblur and it worked,
qiime deblur denoise-16S
--i-demultiplexed-seqs sequences/fj-joined-demux.qza
--p-trim-length 120
--o-representative-sequences sequences/rep-seqs-deblur.qza
--o-table sequences/table-deblur.qza
--p-sample-stats
--o-stats sequences/deblur-stats.qza

also I could do the summary of the deblur table
qiime feature-table summarize
--i-table sequences/table-deblur.qza
--o-visualization sequences/table-deblur.qzv

6

@Oddant1 Thanks for your reply.
The empty lines are not present in the actual file.

7

Hi,

I imported my file with this command
qiime tools import
--type 'SampleData[SequencesWithQuality]'
--input-path se-33-manifest.txt
--output-path single-end-demux.qza \
--input-format SingleEndFastqManifestPhred33V2
But I could not do summary,
When I run
qiime demux summarize
--i-data single-end-demux.qza
--o-visualization demux.qzv
I get the error
Plugin error from demux:

'NoneType' object has no attribute 'strip'

Debug info has been saved to /var/folders/hr/9vcxtv2961bgx43wk4sqww3m0000gn/T/qiime2-q2cli-err-2lvb2v_6.log

The contents of the log file are
Traceback (most recent call last):
File "/Users/syama/opt/anaconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/q2cli/commands.py", line 329, in call
results = action(**arguments)
File "", line 2, in summarize
File "/Users/syama/opt/anaconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/qiime2/sdk/action.py", line 245, in bound_callable
output_types, provenance)
File "/Users/syama/opt/anaconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/qiime2/sdk/action.py", line 452, in callable_executor
ret_val = self._callable(output_dir=temp_dir, **view_args)
File "/Users/syama/opt/anaconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/q2_demux/_summarize/_visualizer.py", line 138, in summarize
for seq in _read_fastq_seqs(filename):
File "/Users/syama/opt/anaconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/q2_demux/_demux.py", line 37, in _read_fastq_seqs
yield (seq_header.strip(), seq.strip(), qual_header.strip(),
AttributeError: 'NoneType' object has no attribute 'strip'

When I do quality-filter,

qiime quality-filter q-score
--i-demux single-end-demux.qza
--o-filtered-sequences demux-filtered.qza
--o-filter-stats demux-filter-stats.qza

the same error coming

Plugin error from quality-filter:
'NoneType' object has no attribute 'strip'
Debug info has been saved to /var/folders/hr/9vcxtv2961bgx43wk4sqww3m0000gn/T/qiime2-q2cli-err-vnxzf2d1.log

The contents of the log file

/Users/syama/opt/anaconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/q2_quality_filter/_filter.py:89: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
phred_offset = yaml.load(metadata_view)['phred-offset']
Traceback (most recent call last):
File "/Users/syama/opt/anaconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/q2cli/commands.py", line 329, in call
results = action(**arguments)
File "", line 2, in q_score
File "/Users/syama/opt/anaconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/qiime2/sdk/action.py", line 245, in bound_callable
output_types, provenance)
File "/Users/syama/opt/anaconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/qiime2/sdk/action.py", line 390, in callable_executor
output_views = self._callable(**view_args)
File "/Users/syama/opt/anaconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/q2_quality_filter/_filter.py", line 115, in q_score
for sequence_record in _read_fastq_seqs(str(fp), phred_offset):
File "/Users/syama/opt/anaconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/q2_quality_filter/_filter.py", line 25, in _read_fastq_seqs
qual = qual.strip()
AttributeError: 'NoneType' object has no attribute 'strip'

please help, I am stuck
Thanks in advance

8

This part suggests there may be an issue with the fastq file, it looks like it is expecting a line of quality scores but not finding that line. I would check that your reads were fully downloaded when imported and that there are not issues with the read files. It's a bit strange that deblur would work while the summarize visualization would not.

Check that the number of lines are the same for the forward and reverse reads
zcat read_file.fastq.gz | wc
And then check that there is nothing weird at the end of the file
zcat read_file.fastq.gz | tail

1 Like
9

Thanks @dwt - I agree, there is an issue with these data.

I'm less surprised about this - deblur doesn't use the quality scores at all.

@syama - please double check what @dwt provided above - thanks!

10

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.