Q2-ITSxpress dada2 keeps giving an error

Hi,

I'm relatively new to qiime2 but familiar with 16s part. Now I wanted to make use of the q2-itsxpress. I demultiplexed my paired end reads demuxITS.qzv (293.6 KB) and then trimmed by ITSxpress with the following command:

qiime itsxpress trim-pair-output-unmerged
--i-per-sample-sequences demuxITS.qza
--p-region ITS1
--p-taxa F
--o-trimmed trimmedITS1.qza

Subsequently, I wanted to use Dada2 with this command:

qiime dada2 denoise-paired
--i-demultiplexed-seqs trimmedITS1.qza
--p-trunc-len-r 182
--p-trunc-len-f 0
--verbose
--output-dir dada2out

However, this gives the following error:

Plugin error from dada2:

No reads passed the filter. trunc_len_f (0) or trunc_len_r (0) may be individually longer than read lengths, or trunc_len_f + trunc_len_r may be shorter than the length of the amplicon + 20 nucleotides (the length of the overlap). Alternatively, other arguments (such as max_ee or trunc_q) may be preventing reads from passing the filter.

See above for debug info.

Then I tried:

qiime dada2 denoise-paired
--i-demultiplexed-seqs trimmedITS1.qza
--o-representative-sequences rep-seqs.qza
--o-table table.qza --o-denoising-stats denoising-stats.qza
--verbose
--p-n-threads 0
--p-trunc-len-f 0
--p-trunc-len-r 182
--p-trunc-q 10

But this gave the exact same error message.
I've tried multiple versions of qiime2 (2018.6, 2018.11), but I'm using 2019.1 at the moment with the hope that that would help. Unfortunately for some reason I can't get dada2 to work.

Is there

Version specs:

System versions
Python version: 3.6.5
QIIME 2 release: 2019.1
QIIME 2 version: 2019.1.0
q2cli version: 2019.1.0

Installed plugins
alignment: 2019.1.0
composition: 2019.1.0
cutadapt: 2019.1.0
dada2: 2019.1.0
deblur: 2019.1.0
demux: 2019.1.0
diversity: 2019.1.0
emperor: 2019.1.0
feature-classifier: 2019.1.0
feature-table: 2019.1.0
fragment-insertion: 2019.1.0
gneiss: 2019.1.0
longitudinal: 2019.1.0
metadata: 2019.1.0
phylogeny: 2019.1.0
quality-control: 2019.1.0
quality-filter: 2019.1.0
sample-classifier: 2019.1.0
taxa: 2019.1.0
types: 2019.1.0
vsearch: 2019.1.0

Name Version Build Channel

_r-mutex 1.0.0 anacondar_1
appnope 0.1.0 py36_1000 conda-forge
arb-bio-tools 6.0.6 4 bioconda
asn1crypto 0.24.0 py36_1003 conda-forge
atomicwrites 1.2.1 py_0 conda-forge
attrs 18.2.0 py_0 conda-forge
backcall 0.1.0 py_0 conda-forge
bibtexparser 1.1.0 py_0 conda-forge
bioconductor-biobase 2.38.0 r341h219a75b_1 bioconda
bioconductor-biocgenerics 0.24.0 r341_1 bioconda
bioconductor-biocparallel 1.12.0 r341h219a75b_1 bioconda
bioconductor-biostrings 2.46.0 r3.4.1_0 bioconda
bioconductor-dada2 1.6.0 r3.4.1_0 bioconda
bioconductor-delayedarray 0.4.1 r3.4.1_0 bioconda
bioconductor-genomeinfodb 1.14.0 r3.4.1_0 bioconda
bioconductor-genomeinfodbdata 1.0.0 r3.4.1_1 bioconda
bioconductor-genomicalignments 1.14.1 r3.4.1_0 bioconda
bioconductor-genomicranges 1.30.3 r3.4.1_0 bioconda
bioconductor-iranges 2.12.0 r3.4.1_0 bioconda
bioconductor-rsamtools 1.30.0 r3.4.1_0 bioconda
bioconductor-s4vectors 0.16.0 r3.4.1_0 bioconda
bioconductor-shortread 1.36.0 r3.4.1_0 bioconda
bioconductor-summarizedexperiment 1.8.0 r3.4.1_0 bioconda
bioconductor-xvector 0.18.0 r3.4.1_0 bioconda
bioconductor-zlibbioc 1.24.0 r3.4.1_0 bioconda
biom-format 2.1.7 py36h917ab60_1002 conda-forge
blas 1.0 mkl conda-forge
blast 2.6.0 boost1.64_2 bioconda
bleach 3.1.0 py_0 conda-forge
bokeh 1.0.4 py36_1000 conda-forge
boost 1.67.0 py36h3e44d54_0 conda-forge
boost-cpp 1.67.0 h3a22d5f_0 conda-forge
bz2file 0.98 py_0 conda-forge
bzip2 1.0.6 h1de35cc_1002 conda-forge
ca-certificates 2018.11.29 ha4d7672_0 conda-forge
cachecontrol 0.12.5 py_0 conda-forge
cairo 1.14.12 he6fea26_5 conda-forge
certifi 2018.11.29 py36_1000 conda-forge
cffi 1.11.5 py36h342bebf_1001 conda-forge
chardet 3.0.4 py36_1003 conda-forge
click 7.0 py_0 conda-forge
cryptography 2.3.1 py36hdbc3d79_1000 conda-forge
curl 7.63.0 heae2a1f_1000 conda-forge
cutadapt 1.18 py36h1de35cc_1 bioconda
cycler 0.10.0 py_1 conda-forge
deblur 1.1.0 py36_0 bioconda
decorator 4.3.2 py_0 conda-forge
emperor 1.0.0beta18 py36_1001 conda-forge
entrypoints 0.3 py36_1000 conda-forge
fastcluster 1.1.25 py36h1702cab_1000 conda-forge
fasttree 2.1.10 0 bioconda
fontconfig 2.13.1 h1e4e890_1000 conda-forge
freetype 2.9.1 h5a5313f_1004 conda-forge
future 0.17.1 py36_1000 conda-forge
gettext 0.19.8.1 hcca000d_1001 conda-forge
glib 2.55.0 h464dc38_2 conda-forge
gneiss 0.4.4 py_0 bioconda
graphite2 1.3.13 h2098e52_1000 conda-forge
gsl 2.1 2 conda-forge
h5py 2.9.0 nompi_py36h35dc34b_1102 conda-forge
harfbuzz 1.9.0 h08d66d9_0 conda-forge
hdf5 1.10.4 nompi_h646315f_1105 conda-forge
hdmedians 0.13 py36h917ab60_1000 conda-forge
icu 58.2 h0a44026_1000 conda-forge
idna 2.8 py36_1000 conda-forge
ijson 2.3 py_1 conda-forge
intel-openmp 2019.1 144
ipykernel 5.1.0 py36h24bf2e0_1002 conda-forge
ipython 7.2.0 py36h24bf2e0_1000 conda-forge
ipython_genutils 0.2.0 py_1 conda-forge
ipywidgets 7.4.2 py_0 conda-forge
iqtree 1.6.9 he941832_0 bioconda
jedi 0.13.2 py36_1000 conda-forge
jinja2 2.10 py_1 conda-forge
jpeg 9c h1de35cc_1001 conda-forge
jsonschema 3.0.0a3 py36_1000 conda-forge
jupyter_client 5.2.4 py_3 conda-forge
jupyter_core 4.4.0 py_0 conda-forge
kiwisolver 1.0.1 py36h04f5b5a_1002 conda-forge
krb5 1.16.3 h24a3359_1000 conda-forge
libarbdb 6.0.6 4 bioconda
libcurl 7.63.0 h76de61e_1000 conda-forge
libcxx 7.0.0 h2d50403_1 conda-forge
libedit 3.1.20170329 0 conda-forge
libffi 3.2.1 h0a44026_1005 conda-forge
libgcc 4.8.5 hdbeacc1_10 conda-forge
libgfortran 3.0.1 0 conda-forge
libiconv 1.15 h1de35cc_1004 conda-forge
libpng 1.6.34 ha92aebf_2 conda-forge
libsodium 1.0.16 h1de35cc_1001 conda-forge
libssh2 1.8.0 hf30b1f0_1003 conda-forge
libtiff 4.0.9 h79f4b77_1002 conda-forge
libxml2 2.9.8 hf14e9c8_1005 conda-forge
llvm-meta 7.0.0 0 conda-forge
lockfile 0.12.2 py_1 conda-forge
mafft 7.310 1 bioconda
markupsafe 1.1.0 py36h1de35cc_1000 conda-forge
matplotlib 2.2.3 py36h0e0179f_0 conda-forge
mistune 0.8.4 py36h1de35cc_1000 conda-forge
mkl 2019.1 144
mkl_fft 1.0.10 py36h1de35cc_1 conda-forge
mkl_random 1.0.2 py36h1702cab_2 conda-forge
more-itertools 4.3.0 py36_1000 conda-forge
msgpack-python 0.6.1 py36h04f5b5a_0 conda-forge
natsort 5.5.0 py_0 conda-forge
nbconvert 5.3.1 py36_0
nbformat 4.4.0 py_1 conda-forge
ncurses 5.9 10 conda-forge
nose 1.3.7 py36_1002 conda-forge
notebook 5.7.4 py36_1 conda-forge
numpy 1.15.4 py36hacdab7b_0
numpy-base 1.15.4 py36h6575580_0
olefile 0.46 py_0 conda-forge
openjdk 11.0.1 h1de35cc_1014 conda-forge
openmp 7.0.0 h2d50403_0 conda-forge
openssl 1.0.2p h1de35cc_1002 conda-forge
packaging 19.0 py_0 conda-forge
pandas 0.23.4 py36h1702cab_1000 conda-forge
pandoc 2.5 1 conda-forge
pandocfilters 1.4.2 py_1 conda-forge
pango 1.40.14 h1401c9d_1002 conda-forge
parso 0.3.2 py_0 conda-forge
patsy 0.5.1 py_0 conda-forge
pcre 8.39 0 conda-forge
pexpect 4.6.0 py36_1000 conda-forge
pickleshare 0.7.5 py36_1000 conda-forge
pigz 2.3.4 0 conda-forge
pillow 5.4.1 py36hbddbef0_1000 conda-forge
pip 19.0.1 py36_0 conda-forge
pixman 0.34.0 h1de35cc_1003 conda-forge
pluggy 0.8.1 py_0 conda-forge
prometheus_client 0.5.0 py_0 conda-forge
prompt_toolkit 2.0.8 py_0 conda-forge
psutil 5.4.8 py36h1de35cc_1000 conda-forge
ptyprocess 0.6.0 py36_1000 conda-forge
py 1.7.0 py_0 conda-forge
pycparser 2.19 py_0 conda-forge
pygments 2.3.1 py_0 conda-forge
pyopenssl 18.0.0 py36_1000 conda-forge
pyparsing 2.3.0 py_0 conda-forge
pyrsistent 0.14.9 py36h1de35cc_1000 conda-forge
pysocks 1.6.8 py36_1002 conda-forge
pytest 4.1.1 py36_1000 conda-forge
python 3.6.5 1 conda-forge
python-dateutil 2.7.5 py_0 conda-forge
pytz 2018.9 py_0 conda-forge
pyyaml 3.13 py36h1de35cc_1001 conda-forge
pyzmq 17.1.2 py36h111632d_1001 conda-forge
q2-alignment 2019.1.0 py36_0 qiime2/label/r2019.1
q2-composition 2019.1.0 py36_0 qiime2/label/r2019.1
q2-cutadapt 2019.1.0 py36_0 qiime2/label/r2019.1
q2-dada2 2019.1.0 py36_0 qiime2/label/r2019.1
q2-deblur 2019.1.0 py36_0 qiime2/label/r2019.1
q2-demux 2019.1.0 py36_0 qiime2/label/r2019.1
q2-diversity 2019.1.0 py36_0 qiime2/label/r2019.1
q2-emperor 2019.1.0 py36_0 qiime2/label/r2019.1
q2-feature-classifier 2019.1.0 py36_0 qiime2/label/r2019.1
q2-feature-table 2019.1.0 py36_0 qiime2/label/r2019.1
q2-fragment-insertion 2019.1.0 py36_0 qiime2/label/r2019.1
q2-gneiss 2019.1.0 py36_0 qiime2/label/r2019.1
q2-longitudinal 2019.1.0 py36_0 qiime2/label/r2019.1
q2-metadata 2019.1.0 py36_0 qiime2/label/r2019.1
q2-phylogeny 2019.1.0 py36_0 qiime2/label/r2019.1
q2-quality-control 2019.1.0 py36_0 qiime2/label/r2019.1
q2-quality-filter 2019.1.0 py36_0 qiime2/label/r2019.1
q2-sample-classifier 2019.1.0 py36_0 qiime2/label/r2019.1
q2-taxa 2019.1.0 py36_0 qiime2/label/r2019.1
q2-types 2019.1.0 py36_0 qiime2/label/r2019.1
q2-vsearch 2019.1.0 py36_0 qiime2/label/r2019.1
q2cli 2019.1.0 py36_0 qiime2/label/r2019.1
q2templates 2019.1.0 py36_0 qiime2/label/r2019.1
qiime2 2019.1.0 py36_0 qiime2/label/r2019.1
r-assertthat 0.2.0 r341h6115d3f_1 conda-forge
r-base 3.4.1 4 conda-forge
r-bh 1.66.0_1 r341_1001 conda-forge
r-bitops 1.0_6 r341hc070d10_2 conda-forge
r-cli 1.0.0 r341h6115d3f_1 conda-forge
r-cluster 2.0.7_1 r341h364d78e_0 conda-forge
r-colorspace 1.3_2 r341hc070d10_2 conda-forge
r-crayon 1.3.4 r341h6115d3f_1 conda-forge
r-data.table 1.11.4 r341hc070d10_2 conda-forge
r-digest 0.6.18 r341hc070d10_0 conda-forge
r-fansi 0.3.0 r341hc070d10_0 conda-forge
r-formatr 1.5 r341h6115d3f_1 conda-forge
r-futile.logger 1.4.3 r341h6115d3f_1 conda-forge
r-futile.options 1.0.1 r341h6115d3f_0 conda-forge
r-ggplot2 3.1.0 r341h6115d3f_0 conda-forge
r-glue 1.3.0 r341h470a237_2 conda-forge
r-gtable 0.2.0 r341h6115d3f_1 conda-forge
r-hwriter 1.3.2 r341h6115d3f_1 conda-forge
r-labeling 0.3 r341h6115d3f_1 conda-forge
r-lambda.r 1.2.3 r341h6115d3f_0 conda-forge
r-lattice 0.20_35 r341hc070d10_0 conda-forge
r-latticeextra 0.6_28 r341h6115d3f_1 conda-forge
r-lazyeval 0.2.1 r341hc070d10_2 conda-forge
r-magrittr 1.5 r341h6115d3f_1 conda-forge
r-mass 7.3_50 r341hc070d10_2 conda-forge
r-matrix 1.2_14 r341hc070d10_2 conda-forge
r-matrixstats 0.54.0 r341hc070d10_0 conda-forge
r-mgcv 1.8_24 r341hc070d10_2 conda-forge
r-munsell 0.5.0 r341h6115d3f_1 conda-forge
r-nlme 3.1_137 r341h364d78e_0 conda-forge
r-permute 0.9_4 r341_1002 conda-forge
r-pillar 1.3.0 r341h6115d3f_0 conda-forge
r-plyr 1.8.4 r341h9d2a408_2 conda-forge
r-r6 2.2.2 r341h6115d3f_1 conda-forge
r-rcolorbrewer 1.1_2 r341h6115d3f_1 conda-forge
r-rcpp 1.0.0 r341h9d2a408_0 conda-forge
r-rcppparallel 4.4.1 r341h9d2a408_0 conda-forge
r-rcurl 1.95_4.11 r341hc070d10_3 conda-forge
r-reshape2 1.4.3 r341h9d2a408_2 conda-forge
r-rlang 0.3.0.1 r341h470a237_0 conda-forge
r-scales 1.0.0 r341h9d2a408_1 conda-forge
r-snow 0.4_3 r341h6115d3f_0 conda-forge
r-stringi 1.2.4 r341h9d2a408_1 conda-forge
r-stringr 1.3.1 r341h6115d3f_1 conda-forge
r-tibble 1.4.2 r341hc070d10_2 conda-forge
r-utf8 1.1.4 r341hc070d10_0 conda-forge
r-vegan 2.5_3 r341h364d78e_0 conda-forge
r-viridislite 0.3.0 r341h6115d3f_1 conda-forge
r-withr 2.1.2 r341h6115d3f_0 conda-forge
raxml 8.2.12 h470a237_0 bioconda
readline 7.0 0 conda-forge
requests 2.21.0 py36_1000 conda-forge
scikit-bio 0.5.5 py36h917ab60_1000 conda-forge
scikit-learn 0.20.2 py36h27c97d8_0
scipy 1.2.0 py36h1410ff5_0
seaborn 0.9.0 py_0 conda-forge
send2trash 1.5.0 py_0 conda-forge
setuptools 40.7.1 py36_0 conda-forge
sina 1.4.0 h4ef8376_0 bioconda
six 1.12.0 py36_1000 conda-forge
sortmerna 2.0 he941832_3 bioconda
sqlite 3.20.1 2 conda-forge
statsmodels 0.9.0 py36h917ab60_1000 conda-forge
tbb 2019.3 h04f5b5a_1000 conda-forge
terminado 0.8.1 py36_1001 conda-forge
testpath 0.4.2 py36_1000 conda-forge
tk 8.6.9 ha441bb4_1000 conda-forge
tornado 5.1.1 py36h1de35cc_1000 conda-forge
traitlets 4.3.2 py36_1000 conda-forge
tzlocal 1.5.1 py_0 conda-forge
unifrac 0.9.3 py36h284470d_0 bioconda
urllib3 1.24.1 py36_1000 conda-forge
vsearch 2.7.0 1 bioconda
wcwidth 0.1.7 py_1 conda-forge
webencodings 0.5.1 py_1 conda-forge
wget 1.19.5 hf30b1f0_0
wheel 0.32.3 py36_0 conda-forge
widgetsnbextension 3.4.2 py36_1000 conda-forge
xopen 0.4.1 py_0 bioconda
xz 5.2.4 h1de35cc_1001 conda-forge
yaml 0.1.7 h1de35cc_1001 conda-forge
zeromq 4.2.5 h0a44026_1006 conda-forge
zlib 1.2.11 h1de35cc_1004 conda-forge

Hey there @persaj!

The error message is telling you that your reads aren't being merged --- with the parameters provided there is insufficient overlap to merge forward and reverse.

Let's take a step back --- do you have a demux summarize viz from after trimming your reads with itsxpress? It would be helpful to look at that, in order to contextualize your choice of truncation parameters.

Thanks! :qiime2:

Hi @thermokarst!

Trying to visualize gives me another error message:

qiime demux summarize
--i-data trimmedITS1.qza
--o-visualization trimmedITS1.qzv

Cannot describe a DataFrame without columns

Thanks!

Hi @persaj,

I had the same error. I lost the original post but the problem arises from a incomplete manifest file produced by ITSxpress for the trimmed sequences. In this manifest file there is no read 2 listed.
You can see this if you use qiime2 export command on your ‘trimmedITS1.qza’. In which you may find the trimmed sequences and the wrong manifest file. I solved the issue by correcting this manifest file and importing the ITS trimmed sequences into a new qza object. This should work properly both with the ‘demux summarize’ comand and the dada2 denoising step.

Hope it makes sense.

Luca

Hi @llenzi

Thanks for that, it does make sense and the manifest file was indeed incomplete. I tried to fix it, import it and then summarize it again, but it still gives me the same error. Interestingly, every file is quite small as well, for example: ‘1AL_30_L001_R1_001.fastq.gz’ is 50 bytes
I then also checked the size of the other files, my ‘demuxITS.qza’ for example is 138 MB (and emp-paired-end-sequences-ITS.qza 2.2 GB), while the ‘trimmedITS1.qza’ is 33 KB. It seems to me like a big loss of data… is that right?

Thanks!

Hi @persaj,
not sure about that, if you open the trimmed files, how many sequences are there?
It may be the case to tag @Adam_Rivers for help!
Luca

Thanks @llenzi! I have submitted a bugfix for that problem on the q2-itsxpress code repo, hopefully that fix will be available some day!

Sorry for the delay on reviewing the pull request. The changes are available on pip now.

pip install --upgrade q2-itsxpress
2 Likes

Thank you for your help!
@Adam_Rivers I have upgraded q2-itsxpress, but unfortunately my trimmedITS.qza still becomes 33 KB. It doesn’t matter whether I choose region ITS1, ITS2 or ALL. The files within the trimmed files seem empty. Subsequent dada2 gives the following error: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

Hi @Adam_Rivers,
I have no better luck with the update too. Tested on qiime2-2018.11 and qiime2-2019.1.
Installing collected packages: q2-itsxpress
Found existing installation: q2-itsxpress 1.7.2
Uninstalling q2-itsxpress-1.7.2:
Successfully uninstalled q2-itsxpress-1.7.2
Successfully installed q2-itsxpress-1.7.3

When I export the trimmed sequences I got:
“Exported paired-end-demux.trim.qza as SingleLanePerSamplePairedEndFastqDirFmt to directory TEST”

The resulting MANIFEST is still:
sample-id,filename,direction
mock-24,mock-24_0_L001_R1_001.fastq.gz,forward

(I’m testing with mock24 in mockrobiota database)
Thanks
Luca

ITSxpress currently only supports primer sets in the “normal” forward orientation where for ITS1, the forward primer is in the 18S and the reverse is in the 5.8S, or for ITS2 the forward primer is in the 5.8s and the reverse primer is in the 28S. Your mock data set uses the primers from Taylor et al. (2016) which are in the reverse orientation. This limitation is documented in the README section.

ITSxpress does not currently support reversed primer sets, but it is on the roadmap for the next incremental release, probably in May.

2 Likes

Sorry I did not realised that!
Thanks !
Luca

Sorry, It’s a weird limitation and presently there are no checks to let you know you are using incompatible primers.

Hi Adam, no problem and most of all thanks for your work, there is a need for your plug-in!
At least is was written and I should check this details more carefully!

Luca

1 Like

Thanks @Adam_Rivers, I also didn’t realise that q2-itsxpress used only forward reads.

itsxpress can use both forward and reverse reads. The issue here is that it expects the forward reads to be in the same orientation as the reference sequences that it uses. Whether you are using single-end or paired-end sequences, orientation is important.

Thanks, I was just about to clarify that.

2 Likes