deblur help:Plugin error from deblur: max() arg is an empty sequence

Trodgers · September 29, 2023, 5:51pm

Hello,
I am trying to run deblur on some NextSeq amplicon data. I usually use DADA2 for denoising (with MiSeq data), but I was informed that DADA2 does not work properly on NextSeq data.
I merged my paired end data with vsearch, and than ran this.

qiime quality-filter q-score
--i-demux andean_bear_12S_trimmed_merged.qza
--o-filtered-sequences andean_bear_12S_filtered.qza
--o-filter-stats andean_bear_12S_filtered-stats.qza

qiime deblur denoise-16S
--i-demultiplexed-seqs andean_bear_12S_filtered.qza
--p-trim-length 100
--o-representative-sequences andean_bear_12S_rep-seqs-deblur.qza
--o-table andean_bear_12S_table-deblur.qza
--p-sample-stats
--o-stats andean_bear_12S_deblur-stats.qza

The quality filter step seemed to run fine, but I am getting this error after the denoising step:
Plugin error from deblur: max() arg is an empty sequence

Can anyone tell me what this error means, and what I need to do?
Thanks,

Torrey

Oddant1 · September 29, 2023, 9:20pm

Hello @Trodgers. Can you please rerun the failing command with the --verbose flag and post the entire output here. Thank you.

Trodgers · September 29, 2023, 11:33pm

Thank you,
Here is the entire output:

/uufs/chpc.utah.edu/sys/installdir/r8/qiime2/2023.5) [u6019869@notch180:deblur]$ qiime deblur denoise-16S --i-demultiplexed-seqs andean_bear_12S_filtered.qza --p-trim-length 100 --o-representative-sequences andean_bear_12S_rep-seqs-deblur.qza --o-table andean_bear_12S_table-deblur.qza --p-sample-stats --o-stats andean_bear_12S_deblur-stats.qza --verbose
/uufs/chpc.utah.edu/sys/installdir/r8/qiime2/2023.5/bin/deblur:534: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
logger.warn('deblur version %s workflow started on %s' %
/uufs/chpc.utah.edu/sys/installdir/r8/qiime2/2023.5/bin/deblur:536: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
logger.warn('parameters: %s' % locals())
/uufs/chpc.utah.edu/sys/installdir/r8/qiime2/2023.5/lib/python3.8/site-packages/deblur/workflow.py:847: UserWarning: Problem removing artifacts from file /scratch/local/u6019869/8605630/qiime2/u6019869/data/567fef4e-8245-4a70-9393-aa5b79959e1f/data/NTC1_0_L001_R1_001.fastq.gz
warnings.warn('Problem removing artifacts from file %s' %
/uufs/chpc.utah.edu/sys/installdir/r8/qiime2/2023.5/bin/deblur:637: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
logger.warn('deblurring failed for file %s' % input_fp)
/uufs/chpc.utah.edu/sys/installdir/r8/qiime2/2023.5/lib/python3.8/site-packages/deblur/workflow.py:847: UserWarning: Problem removing artifacts from file /scratch/local/u6019869/8605630/qiime2/u6019869/data/567fef4e-8245-4a70-9393-aa5b79959e1f/data/eDNA2070_4_L001_R1_001.fastq.gz
warnings.warn('Problem removing artifacts from file %s' %
/uufs/chpc.utah.edu/sys/installdir/r8/qiime2/2023.5/bin/deblur:637: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
logger.warn('deblurring failed for file %s' % input_fp)
/uufs/chpc.utah.edu/sys/installdir/r8/qiime2/2023.5/lib/python3.8/site-packages/deblur/workflow.py:147: UserWarning: Vast majority of sequences (26464 / 26476) are shorter than the trim length (100). Are you using the correct -t trim length?
warnings.warn(errmsg, UserWarning)
/uufs/chpc.utah.edu/sys/installdir/r8/qiime2/2023.5/lib/python3.8/site-packages/deblur/workflow.py:847: UserWarning: Problem removing artifacts from file /scratch/local/u6019869/8605630/qiime2/u6019869/data/567fef4e-8245-4a70-9393-aa5b79959e1f/data/eDNA2074_8_L001_R1_001.fastq.gz
warnings.warn('Problem removing artifacts from file %s' %
/uufs/chpc.utah.edu/sys/installdir/r8/qiime2/2023.5/bin/deblur:637: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
logger.warn('deblurring failed for file %s' % input_fp)
/uufs/chpc.utah.edu/sys/installdir/r8/qiime2/2023.5/lib/python3.8/site-packages/deblur/workflow.py:847: UserWarning: Problem removing artifacts from file /scratch/local/u6019869/8605630/qiime2/u6019869/data/567fef4e-8245-4a70-9393-aa5b79959e1f/data/eDNA2090_24_L001_R1_001.fastq.gz
warnings.warn('Problem removing artifacts from file %s' %
/uufs/chpc.utah.edu/sys/installdir/r8/qiime2/2023.5/bin/deblur:637: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
logger.warn('deblurring failed for file %s' % input_fp)
/uufs/chpc.utah.edu/sys/installdir/r8/qiime2/2023.5/lib/python3.8/site-packages/deblur/workflow.py:92: UserWarning: input file /scratch/local/u6019869/8605630/tmp6r2_ed22/all.seqs.fa.no_artifacts does not appear to be FASTA or FASTQ
warnings.warn(msg, UserWarning)
Traceback (most recent call last):
File "/uufs/chpc.utah.edu/sys/installdir/r8/qiime2/2023.5/lib/python3.8/site-packages/q2cli/commands.py", line 468, in call
results = action(**arguments)
File "", line 2, in denoise_16S
File "/uufs/chpc.utah.edu/sys/installdir/r8/qiime2/2023.5/lib/python3.8/site-packages/qiime2/sdk/action.py", line 274, in bound_callable
outputs = self.callable_executor(
File "/uufs/chpc.utah.edu/sys/installdir/r8/qiime2/2023.5/lib/python3.8/site-packages/qiime2/sdk/action.py", line 509, in callable_executor
output_views = self._callable(**view_args)
File "/uufs/chpc.utah.edu/sys/installdir/r8/qiime2/2023.5/lib/python3.8/site-packages/q2_deblur/_denoise.py", line 89, in denoise_16S
return _denoise_helper(
File "/uufs/chpc.utah.edu/sys/installdir/r8/qiime2/2023.5/lib/python3.8/site-packages/q2_deblur/_denoise.py", line 194, in _denoise_helper
table = _load_table(tmp)
File "/uufs/chpc.utah.edu/sys/installdir/r8/qiime2/2023.5/lib/python3.8/site-packages/q2_deblur/_denoise.py", line 61, in _load_table
table.update_ids(sid_map, axis='sample', inplace=True)
File "/uufs/chpc.utah.edu/sys/installdir/r8/qiime2/2023.5/lib/python3.8/site-packages/biom/table.py", line 1405, in update_ids
str_dtype = 'U%d' % max([len(v) for v in id_map.values()])
ValueError: max() arg is an empty sequence

Plugin error from deblur:

max() arg is an empty sequence

See above for debug info.

Oddant1 · October 2, 2023, 4:52pm

It looks like the biom table that came out of your deblur run may have been empty, but it's hard to tell why. Can you DM me your input .qza file so I can try to replicate this error and figure out what is causing it?

Charuka2370 · October 3, 2023, 10:37am

Hi Oddant1,

I'm commenting here as I have the same issue with qiime deblur denoise-16S . I'm running Qiime2 2023.7
I ran the following commands and have attached my .qzv files at each step.

1) generating the .qza

demultiplexed_sequences = Artifact.import_data(
'SampleData[PairedEndSequencesWithQuality]', # Change this to 'SampleData[SequencesWithQuality]'
renamed_dir, # the sub folder
CasavaOneEightSingleLanePerSampleDirFmt,
)

Saving the .qza file

demultiplexed_sequences.save(result_dir)

** 2) Remove primers from the sequencing data **

cmd = """
qiime cutadapt trim-paired
--i-demultiplexed-sequences /home/charuka/PilotSequencing/demultiplexed_sequences.qza
--p-cores 40
--p-front-f GTGCCAGCMGCCGCGGTAA
--p-front-r GGACTACHVGGGTWTCTAAT
--p-match-adapter-wildcards
--o-trimmed-sequences /home/charuka/PilotSequencing/trimmed_demultiplexed_sequences.qza
"""
subprocess.run(cmd, shell=True)

3) Joining the pair end reads

cmd = """
qiime vsearch merge-pairs
--i-demultiplexed-seqs trimmed_demultiplexed_sequences.qza
--p-threads 8
--o-merged-sequences demux-trimed-joined.qza
--verbose
"""
subprocess.run(cmd, shell=True)

** 4) Filtering **

cmd = """
qiime quality-filter q-score
--i-demux demux-trimed-joined.qza
--p-min-quality 20
--p-min-length-fraction 0.5
--o-filtered-sequences demux-trimed-joined-filtered.qza
--o-filter-stats demux-trimed-joined-filtered_Stats.qza

"""
subprocess.run(cmd, shell=True)

** 5) Denoising**

cmd = """\

qiime deblur denoise-16S
--i-demultiplexed-seqs /home/charuka/PilotSequencing/demux-trimed-joined-filtered.qza
--p-trim-length 464
--p-sample-stats
--o-representative-sequences /home/charuka/PilotSequencing/rep-seqs-deblur.qza
--o-table /home/charuka/PilotSequencing/table-deblur.qza
--o-stats /home/charuka/PilotSequencing/deblur-stats.qza
--verbose
"""
subprocess.run(cmd, shell=True)

Then I hit the following error

/home/charuka/anaconda3/envs/qiime2-2023.7/bin/deblur:534: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
logger.warn('deblur version %s workflow started on %s' %
/home/charuka/anaconda3/envs/qiime2-2023.7/bin/deblur:536: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
logger.warn('parameters: %s' % locals())
/home/charuka/anaconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/deblur/workflow.py:847: UserWarning: Problem removing artifacts from file /tmp/qiime2/charuka/data/52ba129f-37a7-41a4-b19b-054afc6b8956/data/D110z2005DJ502_389_L001_R1_001.fastq.gz
warnings.warn('Problem removing artifacts from file %s' %
/home/charuka/anaconda3/envs/qiime2-2023.7/bin/deblur:637: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
logger.warn('deblurring failed for file %s' % input_fp)
/home/charuka/anaconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/deblur/workflow.py:847: UserWarning: Problem removing artifacts from file /tmp/qiime2/charuka/data/52ba129f-37a7-41a4-b19b-054afc6b8956/data/37z1C8C102W_182_L001_R1_001.fastq.gz
warnings.warn('Problem removing artifacts from file %s' %
/home/charuka/anaconda3/envs/qiime2-2023.7/bin/deblur:637: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
logger.warn('deblurring failed for file %s' % input_fp)
/home/charuka/anaconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/deblur/workflow.py:147: UserWarning: Vast majority of sequences (21 / 21) are shorter than the trim length (464). Are you using the correct -t trim length?
warnings.warn(errmsg, UserWarning)
/home/charuka/anaconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/deblur/workflow.py:847: UserWarning: Problem removing artifacts from file /tmp/qiime2/charuka/data/52ba129f-37a7-41a4-b19b-054afc6b8956/data/496z1C0C511W_283_L001_R1_001.fastq.gz
warnings.warn('Problem removing artifacts from file %s' %
/home/charuka/anaconda3/envs/qiime2-2023.7/bin/deblur:637: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
logger.warn('deblurring failed for file %s' % input_fp)
/home/charuka/anaconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/deblur/workflow.py:847: UserWarning: Problem removing artifacts from file /tmp/qiime2/charuka/data/52ba129f-37a7-41a4-b19b-054afc6b8956/data/562z2C0C31W_340_L001_R1_001.fastq.gz
warnings.warn('Problem removing artifacts from file %s' %
/home/charuka/anaconda3/envs/qiime2-2023.7/bin/deblur:637: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
logger.warn('deblurring failed for file %s' % input_fp)
/home/charuka/anaconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/deblur/workflow.py:847: UserWarning: Problem removing artifacts from file /tmp/qiime2/charuka/data/52ba129f-37a7-41a4-b19b-054afc6b8956/data/466z2AH501B_257_L001_R1_001.fastq.gz
warnings.warn('Problem removing artifacts from file %s' %
/home/charuka/anaconda3/envs/qiime2-2023.7/bin/deblur:637: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
...

max() arg is an empty sequence

See above for debug info.
CompletedProcess(args='\nqiime deblur denoise-16S --i-demultiplexed-seqs /home/charuka/PilotSequencing/demux-trimed-joined-filtered.qza --p-trim-length 464 --p-sample-stats --o-representative-sequences /home/charuka/PilotSequencing/rep-seqs-deblur.qza --o-table /home/charuka/PilotSequencing/table-deblur.qza --o-stats /home/charuka/PilotSequencing/deblur-stats.qza --verbose\n\n\n', returncode=1)

Can you please help me with this?

Results_qzvs.zip (1.2 MB)

Thank you

Oddant1 · October 4, 2023, 5:03pm

@Trodgers some other moderators are suggesting that the fact that you are using 12S data in a 16S workflow could be causing some problems. Deblur was not designed to handle 12S. Additionally, your reads seem awfully short after running merge pairs, and when I run demux summarize on the data you DM'd me the quality scores on the ends are very noisy. It seems likely that this workflow is not going to work on your data. You could try using dada2, but I'm not sure if you willl see better results there.

Oddant1 · October 4, 2023, 5:10pm

Hello @Charuka2370. Your trim length is very long and is causing somewhere between 75% and 91% of f your reads to be discarded based on your demux-trimmed-joined-filtered.qzv. It looks like you could set your trim length down to 300 and only lose about 10% of your data

Oddant1 · October 6, 2023, 3:53pm

An addendum on this, you may be able to use some parts of deblur, but denoise-16S may not work correctly on 12S due to the positive filter.

system · November 6, 2023, 9:53pm

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.