Problems with dada2 - Qiime2 installed locally on a cluster

User Support
,
1

Hi,

I installed the latest version of q2 (2018.4) locally on a cluster. It installed without any errors. When I tried to use dada2 to quality filter my data, I got the following error:

WARNING: ignoring environment value of R_HOME
R version 3.4.1 (2017-06-30)
Loading required package: Rcpp
Error: package or namespace load failed for ‘dada2’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/apps/lib-osver/R-mkl/3.3.2/lib64/R/library/RCurl/libs/RCurl.so':
/lib64/libk5crypto.so.3: symbol krb5int_buf_len, version krb5support_0_MIT not defined in file libkrb5support.so.0 with link time reference
Execution halted
Plugin error from dada2:

My code:
qiime dada2 denoise-paired
--i-demultiplexed-seqs demux-paired-end.qza
--o-table table
--o-representative-sequences rep-seqs
--p-trim-left-f 0
--p-trim-left-r 0
--p-trunc-len-f 180
--p-trunc-len-r 180

There are a few different versions of R on the cluster. Do I need to install dada2 separately in the latest R version (R version 3.4.1 (2017-06-30))? As far as I can gather, the dada2 package is not installed in this version of R.

Thanks.

2

Hi,

I was attempting to run dada2 on a cluster to denoise my data and received this error:
Plugin error from dada2:

An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

log file details:
Command: run_dada_paired.R /tmp/tmpv_oc6ava/forward /tmp/tmpv_oc6ava/reverse /tmp/tmpv_oc6ava/output.tsv.biom /tmp/tmpv_oc6ava/track.tsv /tmp/tmpv_oc6ava/filt_f /tmp/tmpv_oc6ava/filt_r 180 180 0 0 2.0 2 consensus 1.0 1 1000000

R version 3.4.1 (2017-06-30)
Loading required package: Rcpp
Error: package or namespace load failed for ‘dada2’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/PHShome/sy282/miniconda2/envs/qiime2-2018.4/lib/R/library/RCurl/libs/RCurl.so':
/PHShome/sy282/miniconda2/envs/qiime2-2018.4/lib/R/library/RCurl/libs/../../../.././libcom_err.so.3: symbol k5_strerror_r, version krb5support_0_MIT not defined in file libkrb5support.so.0 with link time reference
Execution halted
Traceback (most recent call last):
File "/PHShome/sy282/miniconda2/envs/qiime2-2018.4/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 229, in denoise_paired
run_commands([cmd])
File "/PHShome/sy282/miniconda2/envs/qiime2-2018.4/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 36, in run_commands
subprocess.run(cmd, check=True)
File "/PHShome/sy282/miniconda2/envs/qiime2-2018.4/lib/python3.5/subprocess.py", line 398, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['run_dada_paired.R', '/tmp/tmpv_oc6ava/forward', '/tmp/tmpv_oc6ava/reverse', '/tmp/tmpv_oc6ava/output.tsv.biom', '/tmp/tmpv_oc6ava/track.tsv', '/tmp/tmpv_oc6ava/filt_f', '/tmp/tmpv_oc6ava/filt_r', '180', '180', '0', '0', '2.0', '2', 'consensus', '1.0', '1', '1000000']' returned non-zero exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/PHShome/sy282/miniconda2/envs/qiime2-2018.4/lib/python3.5/site-packages/q2cli/commands.py", line 274, in call
results = action(**arguments)
File "", line 2, in denoise_paired
File "/PHShome/sy282/miniconda2/envs/qiime2-2018.4/lib/python3.5/site-packages/qiime2/sdk/action.py", line 231, in bound_callable
output_types, provenance)
File "/PHShome/sy282/miniconda2/envs/qiime2-2018.4/lib/python3.5/site-packages/qiime2/sdk/action.py", line 366, in callable_executor
output_views = self._callable(**view_args)
File "/PHShome/sy282/miniconda2/envs/qiime2-2018.4/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 244, in denoise_paired
" and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

My qiime command was:
qiime dada2 denoise-paired
--i-demultiplexed-seqs demux-paired-end.qza
--o-table table
--o-representative-sequences rep-seqs
--o-denoising-stats denoising-stats.qza
--p-trim-left-f 0
--p-trim-left-r 0
--p-trunc-len-f 180
--p-trunc-len-r 180

conda 4.5.2
qiime2-2018.4
R version 3.4.1 (2017-06-30)

This error is similar to: Qiime2-2018.2, dada2 denoising, error (return code 1) Qiime2-2018.2, dada2 denoising, error (return code 1)

I have uninstalled and re-installed q2 twice already with no change.

Would appreciate any suggestions.

4

Hi @sbudree,

I suspect something funny is going on here. Our goal isn't going to be to install DADA2 somewhere else, so much as to make sure that other R's aren't interfering with the QIIME 2 installation (lots of ways for that to happen).

Since your cluster already has a few R installations, there's probably a mechanism managing it. Do you have anything like a module load command that sets up your session with different programs?

Also, when inside a QIIME 2 environment, what do the following commands show:

env

which R

R -e ".libPaths()"

I think one of those should give us some insight as to what is going on.

6

Hi,

Yes, I can use 'module load' to load different versions of R or other programs.

output from env (qiime2-2018.4) [sy282@eris1n3 ~] env
MANPATH=:/hptc_cluster/lsf/8.0/man:
HOSTNAME=eris1n3.research.partners.org
SELINUX_ROLE_REQUESTED=
TERM=xterm-256color
SHELL=/bin/bash
PYTHONNOUSERSITE=/PHShome/sy282/miniconda2/envs/qiime2-2018.4/lib/python*/site-packages/
HISTSIZE=1000
SSH_CLIENT=10.251.35.219 60683 22
CONDA_SHLVL=1
PERL5LIB=/PHShome/sy282/perl5/lib/perl5/x86_64-linux-thread-multi:/PHShome/sy282/perl5/lib/perl5
CONDA_PROMPT_MODIFIER=(qiime2-2018.4)
SELINUX_USE_CURRENT_RANGE=
QTDIR=/usr/lib64/qt-3.3
LSF_SERVERDIR=/hptc_cluster/lsf/8.0/linux2.6-glibc2.3-x86_64/etc
QTINC=/usr/lib64/qt-3.3/include
SSH_TTY=/dev/pts/58
LSF_LIBDIR=/hptc_cluster/lsf/8.0/linux2.6-glibc2.3-x86_64/lib
USER=sy282
LD_LIBRARY_PATH=/lib64:/usr/lib64:/lib:/usr/lib:/hptc_cluster/lsf/8.0/linux2.6-glibc2.3-x86_64/lib
LS_COLORS=rs=0:di=38;5;27:ln=38;5;51:mh=44;38;5;15:pi=40;38;5;11:so=38;5;13:do=38;5;5:bd=48;5;232;38;5;11:cd=48;5;232;38;5;3:or=48;5;232;38;5;9:mi=05;48;5;232;38;5;15:su=48;5;196;38;5;15:sg=48;5;11;38;5;16:ca=48;5;196;38;5;226:tw=48;5;10;38;5;16:ow=48;5;10;38;5;21:st=48;5;21;38;5;15:ex=38;5;34:.tar=38;5;9:.tgz=38;5;9:.arj=38;5;9:.taz=38;5;9:.lzh=38;5;9:.lzma=38;5;9:.tlz=38;5;9:.txz=38;5;9:.zip=38;5;9:.z=38;5;9:.Z=38;5;9:.dz=38;5;9:.gz=38;5;9:.lz=38;5;9:.xz=38;5;9:.bz2=38;5;9:.tbz=38;5;9:.tbz2=38;5;9:.bz=38;5;9:.tz=38;5;9:.deb=38;5;9:.rpm=38;5;9:.jar=38;5;9:.rar=38;5;9:.ace=38;5;9:.zoo=38;5;9:.cpio=38;5;9:.7z=38;5;9:.rz=38;5;9:.jpg=38;5;13:.jpeg=38;5;13:.gif=38;5;13:.bmp=38;5;13:.pbm=38;5;13:.pgm=38;5;13:.ppm=38;5;13:.tga=38;5;13:.xbm=38;5;13:.xpm=38;5;13:.tif=38;5;13:.tiff=38;5;13:.png=38;5;13:.svg=38;5;13:.svgz=38;5;13:.mng=38;5;13:.pcx=38;5;13:.mov=38;5;13:.mpg=38;5;13:.mpeg=38;5;13:.m2v=38;5;13:.mkv=38;5;13:.ogm=38;5;13:.mp4=38;5;13:.m4v=38;5;13:.mp4v=38;5;13:.vob=38;5;13:.qt=38;5;13:.nuv=38;5;13:.wmv=38;5;13:.asf=38;5;13:.rm=38;5;13:.rmvb=38;5;13:.flc=38;5;13:.avi=38;5;13:.fli=38;5;13:.flv=38;5;13:.gl=38;5;13:.dl=38;5;13:.xcf=38;5;13:.xwd=38;5;13:.yuv=38;5;13:.cgm=38;5;13:.emf=38;5;13:.axv=38;5;13:.anx=38;5;13:.ogv=38;5;13:.ogx=38;5;13:.aac=38;5;45:.au=38;5;45:.flac=38;5;45:.mid=38;5;45:.midi=38;5;45:.mka=38;5;45:.mp3=38;5;45:.mpc=38;5;45:.ogg=38;5;45:.ra=38;5;45:.wav=38;5;45:.axa=38;5;45:.oga=38;5;45:.spx=38;5;45:.xspf=38;5;45:
CONDA_EXE=/PHShome/sy282/miniconda2/bin/conda
MODULEBUILDRC=/PHShome/sy282/perl5/.modulebuildrc
MAIL=/var/spool/mail/sy282
PATH=/PHShome/sy282/miniconda2/envs/qiime2-2018.4/bin:/PHShome/sy282/miniconda2/bin:/usr/lib64/qt-3.3/bin:/PHShome/sy282/perl5/bin:/hptc_cluster/lsf/8.0/linux2.6-glibc2.3-x86_64/etc:/hptc_cluster/lsf/8.0/linux2.6-glibc2.3-x86_64/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/ibutils/bin:/opt/puppetlabs/bin:/PHShome/sy282/bin
CONDA_PREFIX=/PHShome/sy282/miniconda2/envs/qiime2-2018.4
PWD=/PHShome/sy282
MPLBACKEND=Agg
EDITOR=nano
LANG=en_US.UTF-8
MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/etc/modulefiles/:/usr/share/Modules/modulefiles:/apps/modulefiles/base
LOADEDMODULES=
SELINUX_LEVEL_REQUESTED=
LSF_BINDIR=/hptc_cluster/lsf/8.0/linux2.6-glibc2.3-x86_64/bin
HISTCONTROL=ignoredups
KRB5CCNAME=FILE:/tmp/krb5cc_3658451_Z2Ii7Y
SHLVL=1
HOME=/PHShome/sy282
BINARY_TYPE_HPC=
CONDA_PYTHON_EXE=/PHShome/sy282/miniconda2/bin/python
LOGNAME=sy282
QTLIB=/usr/lib64/qt-3.3/lib
CVS_RSH=ssh
SSH_CONNECTION=10.251.35.219 60683 172.27.81.81 22
MODULESHOME=/usr/share/Modules
CONDA_DEFAULT_ENV=qiime2-2018.4
LESSOPEN=||/usr/bin/lesspipe.sh %s
LSF_ENVDIR=/hptc_cluster/lsf/conf
R_LIBS_USER=/PHShome/sy282/miniconda2/envs/qiime2-2018.4/lib/R/library/
PERL_MM_OPT=INSTALL_BASE=/PHShome/sy282/perl5
G_BROKEN_FILENAMES=1
BASH_FUNC_module()=() { eval /usr/bin/modulecmd bash $*
}
_=/bin/env

output from: $which R
~/miniconda2/envs/qiime2-2018.4/bin/R

output from: R -e ".libPaths()"
R version 3.4.1 (2017-06-30) -- "Single Candle"
Copyright (C) 2017 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

.libPaths()
[1] "/PHShome/sy282/miniconda2/envs/qiime2-2018.4/lib/R/library"

10

Hey there @sbudree, sorry we didn't get back to you sooner.

Thanks for the details, nothing is jumping out at me that seems out of place (although @ebolyen might have picked up on something here).

Are you loading a module before activating your qiime 2 env? If so, this seems like a likely culprit to me.

This little bit is why I think that - this path has nothing to do with our QIIME 2 environment you created, but somehow info about your host system is leaking into your env...

The weird thing is this error is slightly different - did you do anything to your env between those two commands? Maybe it is worth trying to uninstall and reinstall the q2 env. You can remove by running

source deactivate qiime2-2018.4
conda env remove -n qiime2-2018.4

I am still a bit puzzled though, this might be a good question for your cluster sysadmin, they might have some specific thoughts as to global env vars/config being injected into your conda installation. Just a thought. Keep us posted! :t_rex:

1 Like
12

Hi,

One of the administrators of the cluster were able to sort this out for me. He advised me to use the following path:
$export LD_LIBRARY_PATH=/PHShome/sy282/miniconda2/envs/qiime2-2018.4/lib:$LD_LIBRARY_PATH

This solved the issue.

Thanks.

3 Likes
13

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.