Plug-in error from deblur: Duplicate sample IDs

Hi! I'm trying to import my data, but I'm not sure where I've gone wrong. I have this error after 5 hours of running the command:

Blockquote Plugin error from deblur:

Blockquote Duplicate sample IDs!

Blockquote Debug info has been saved to /var/folders/yg/wtspl3kd78z1mymc99k3zkl00000gp/T/qiime2-q2cli-err-pi766zjc.log

Here are the commands I have used that have run OK until the last command:
qiime tools import
--type 'SampleData[PairedEndSequencesWithQuality]'
--input-path fastq_manifest_16S_sarah.txt
--output-path 16S-demux.qza
--input-format PairedEndFastqManifestPhred33

qiime demux summarize
--i-data 16S-demux.qza
--o-visualization 16S-demux.qzv

qiime quality-filter q-score
--i-demux 16S-demux.qza
--o-filtered-sequences demux-filtered-16S.qza
--o-filter-stats demux-filter-stats-16S.qza

qiime demux summarize
--i-data demux-filtered-16S.qza
--o-visualization demux-filtered-16S.qzv

qiime deblur denoise-16S
--i-demultiplexed-seqs demux-filtered-16S.qza
--p-trim-length 150
--p-left-trim-len 13
--p-sample-stats
--o-table table-deblur-16s.qza
--o-representative-sequences rep-seqs-deblur-16s.qza
--o-stats stats-deblur-16s.qza

I've opened the demux files (both filtered and unfiltered) and downloaded the .csv for the sample names and double checked for duplicates and there weren't any. Here are the relevant files:

fastq_manifest_16S_sarah.txt (23.3 KB)
per-sample-fastq-counts.csv (1.6 KB)
16S-demux.qzv (289.4 KB)
demux-filtered-16S.qzv (292.2 KB)

Hi @sarah_i,
Two questions:

  1. what version of QIIME 2 are you using?
  2. could you please post the full error message? (re-run with the --verbose flag, or open the log file where debug info has been saved)

It sounds like you are possibly running an outdated version of deblur that could not handle sample IDs with underscores in the names, as described here:

If that is the case, just install the latest version of QIIME 2 and your problems should dissolve away!

I hope that helps.

Hi Nicholas,

  1. I'm definitely using the 2019.4 version of QIIME2. Deblur version is definitely 2019.4.

qiime info
System versions
Python version: 3.6.7
QIIME 2 release: 2019.4
QIIME 2 version: 2019.4.0
q2cli version: 2019.4.0

Installed plugins
alignment: 2019.4.0
composition: 2019.4.0
cutadapt: 2019.4.0
dada2: 2019.4.0
deblur: 2019.4.0
demux: 2019.4.0
diversity: 2019.4.0
emperor: 2019.4.0
feature-classifier: 2019.4.0
feature-table: 2019.4.0
fragment-insertion: 2019.4.0
gneiss: 2019.4.0
longitudinal: 2019.4.0
metadata: 2019.4.0
phylogeny: 2019.4.0
quality-control: 2019.4.0
quality-filter: 2019.4.0
sample-classifier: 2019.4.0
taxa: 2019.4.0
types: 2019.4.0
vsearch: 2019.4.0

  1. I'm currently re-running with --verbose as you've said, but here is the file with all the debug info copied and pasted into a text document (as I can't upload .log files): deblur_log.txt (199.2 KB)

Is the worst case scenario just me changing all the underscores to dashes?

Thanks for all your help!

Update: I've tried denoising my data with DADA2, and it's worked. I think it did take a bit less time than deblur. I still came back with the same error for the deblur run though.

Yes, it looks like that is the solution here if you want to use deblur.

I was mistaken about that bug having been fixed in more recent releases; I did a little more digging, and it looks like underscores in sample ids are still not supported in q2-deblur. This is specific to q2-deblur; you can use q2-dada2 and other plugins in QIIME 2 without an issue (as you found) if your sample ids contain underscores.

Good luck!

1 Like