Environment and installation issues

Technical Support
1

Yes. I installed it on the public domain to set for group users. Where is the ‘run_dada_paired.R’ located. Did I miss some packages to install?
Thanks! Wei

2

Yep! You missed all of QIIME 2. How did you install QIIME 2 and the plugins?

3

I see.
I Installed it by miniconda according the following link:
https://docs.qiime2.org/2018.11/install/native/#install-miniconda

Do you mean I missed the following package: How can I get all of QIIME 2?

https://docs.qiime2.org/2018.11/install/

QIIME 2 Core 2018.11 distribution

The QIIME 2 Core 2018.11 distribution includes the QIIME 2 framework, q2cli (a QIIME 2 command-line interface) and the following plugins:

  • q2-alignment
  • q2-composition
  • q2-cutadapt
  • q2-dada2
  • q2-deblur
  • q2-demux
  • q2-diversity
  • q2-emperor
  • q2-feature-classifier
  • q2-feature-table
  • q2-fragment-insertion
  • q2-gneiss
  • q2-longitudinal
  • q2-metadata
  • q2-phylogeny
  • q2-quality-control
  • q2-quality-filter
  • q2-sample-classifier
  • q2-taxa
  • q2-types
  • q2-vsearch
    Thanks!

wei

4

You mentioned above that you "installed it on the public domain to set for group users" - does that mean that you installed into the root/base environment? If so, something went wrong with the install. Either way though, it looks like your environment is incomplete --- I suggest you remove the environment and try again.

5

I don't have root permission on cluster. Does it require as root? If not, I plan to reinstalled it from scratch. Thanks for your quick response. Wei

6

Nope, as long as you follow our installation recommendation of installing into a separate conda env.

7

I run into following errors:

[swadmin@sirius qiime]$ which wget
/usr/public/qiime/miniconda3//bin/wget
[swadmin@sirius qiime]$ wget http://data.qiime2.org/distro/core/qiime2-2018.4-py35-osx-conda.yml
--2019-03-05 17:19:30--  http://data.qiime2.org/distro/core/qiime2-2018.4-py35-osx-conda.yml
Resolving data.qiime2.org... 52.35.38.247
Connecting to data.qiime2.org|52.35.38.247|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://data.qiime2.org:443/distro/core/qiime2-2018.4-py35-osx-conda.yml [following]
--2019-03-05 17:19:30--  https://data.qiime2.org/distro/core/qiime2-2018.4-py35-osx-conda.yml
Disabling SSL due to encountered errors.
[swadmin@sirius qiime]$

any suggestion?

Thanks
wei

8

Please just download the file directly, by copying and pasting the URL into a browser. conda's wget appears to be broken, we are in the process of publishing revised install guides.

PS - why are you installing 2018.4? The latest release is 2019.1 --- I highly recommend installing the newest version.

9

Update - the revised docs are now live:

https://docs.qiime2.org/2019.1/install/native/

10

Thanks! I used the curl to get qiime2-2019.1-py36-linux-conda.yml file.
I run another error: "An unexpected error has occurred"

[swadmin@sirius qiime]$ conda env create -n qiime2-2019.1 --file qiime2-2019.1-py36-linux-conda.yml

>>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

Traceback (most recent call last):
  File "/gsfs0/public/qiime/miniconda3/lib/python3.6/site-packages/conda/exceptions.py", line 1002, in __call__
    return func(*args, **kwargs)
  File "/gsfs0/public/qiime/miniconda3/lib/python3.6/site-packages/conda_env/cli/main.py", line 73, in do_call
    exit_code = getattr(module, func_name)(args, parser)
  File "/gsfs0/public/qiime/miniconda3/lib/python3.6/site-packages/conda_env/cli/main_create.py", line 77, in execute
    directory=os.getcwd())
  File "/gsfs0/public/qiime/miniconda3/lib/python3.6/site-packages/conda_env/specs/__init__.py", line 40, in detect
    if spec.can_handle():
  File "/gsfs0/public/qiime/miniconda3/lib/python3.6/site-packages/conda_env/specs/yaml_file.py", line 18, in can_handle
    self._environment = env.from_file(self.filename)
  File "/gsfs0/public/qiime/miniconda3/lib/python3.6/site-packages/conda_env/env.py", line 131, in from_file
    return from_yaml(yamlstr, filename=filename)
  File "/gsfs0/public/qiime/miniconda3/lib/python3.6/site-packages/conda_env/env.py", line 116, in from_yaml
    data = yaml_load_standard(yamlstr)
  File "/gsfs0/public/qiime/miniconda3/lib/python3.6/site-packages/conda/common/serialize.py", line 76, in yaml_load_standard
    return yaml.load(string, Loader=yaml.Loader, version="1.2")
  File "/gsfs0/public/qiime/miniconda3/lib/python3.6/site-packages/ruamel_yaml/main.py", line 924, in load
    return loader._constructor.get_single_data()
  File "/gsfs0/public/qiime/miniconda3/lib/python3.6/site-packages/ruamel_yaml/constructor.py", line 106, in get_single_data
    node = self.composer.get_single_node()
  File "/gsfs0/public/qiime/miniconda3/lib/python3.6/site-packages/ruamel_yaml/composer.py", line 78, in get_single_node
    document = self.compose_document()
  File "/gsfs0/public/qiime/miniconda3/lib/python3.6/site-packages/ruamel_yaml/composer.py", line 104, in compose_document
    self.parser.get_event()
  File "/gsfs0/public/qiime/miniconda3/lib/python3.6/site-packages/ruamel_yaml/parser.py", line 163, in get_event
    self.current_event = self.state()
  File "/gsfs0/public/qiime/miniconda3/lib/python3.6/site-packages/ruamel_yaml/parser.py", line 239, in parse_document_end
    token = self.scanner.peek_token()
  File "/gsfs0/public/qiime/miniconda3/lib/python3.6/site-packages/ruamel_yaml/scanner.py", line 182, in peek_token
    self.fetch_more_tokens()
  File "/gsfs0/public/qiime/miniconda3/lib/python3.6/site-packages/ruamel_yaml/scanner.py", line 282, in fetch_more_tokens
    return self.fetch_value()
  File "/gsfs0/public/qiime/miniconda3/lib/python3.6/site-packages/ruamel_yaml/scanner.py", line 655, in fetch_value
    self.reader.get_mark(),
ruamel_yaml.scanner.ScannerError: mapping values are not allowed here
  in "<unicode string>", line 4, column 56:
     ... cted automatically to target URL: <a href="https://raw.githubuse ...
                                         ^ (line: 4)

$ /usr/public/qiime/miniconda3/bin/conda-env create -n qiime2-2019.1 --file qiime2-2019.1-py36-linux-conda.yml

environment variables:
CIO_TEST=
CONDA_AUTO_UPDATE_CONDA=false
CONDA_ROOT=/gsfs0/public/qiime/miniconda3
LD_LIBRARY_PATH=/cm/shared/apps/torque/lib:/cm/shared/apps/torque/6.1.1/lib/
MANPATH=/cm/shared/apps/torque/current/share/man:/cm/shared/apps/torque/6.1.1/
share/man:/usr/local/share/man:/usr/share/man/overrides:/usr/share/man
:/cm/local/apps/environment-modules/current/share/man
MODULEPATH=/cm/local/modulefiles:/cm/shared/modulefiles
PATH=/usr/public/qiime/miniconda3//bin:/gsfs0/public/fsl/5.0.11/bin:/data/s
wadmin/bin:/cm/shared/apps/torque/bin:/cm/shared/apps/torque/sbin:/cm/
shared/apps/moab/bin:/cm/shared/apps/moab/sbin:/usr/public/moaB/script
s:/cm/shared/apps/torque/6.1.1/sbin:/cm/shared/apps/torque/6.1.1/bin:/
usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/sbin:
/cm/local/apps/environment-
modules/3.2.10/bin:/opt/dell/srvadmin/bin:/data/swadmin/abin
REQUESTS_CA_BUNDLE=
SSL_CERT_FILE=

 active environment : None
   user config file : /data/swadmin/.condarc

populated config files : /data/swadmin/.condarc
conda version : 4.6.7
conda-build version : not installed
python version : 3.6.7.final.0
base environment : /gsfs0/public/qiime/miniconda3 (writable)
channel URLs : conda-forge/linux-64
conda-forge/noarch
bioconda/linux-64
bioconda/noarch
main/linux-64
main/noarch
Anaconda packages for Linux x86_64 (64-bit)
Anaconda packages (noarch)
r/linux-64
r/noarch
package cache : /gsfs0/public/qiime/miniconda3/pkgs
/data/swadmin/.conda/pkgs
envs directories : /gsfs0/public/qiime/miniconda3/envs
/data/swadmin/.conda/envs
platform : linux-64
user-agent : conda/4.6.7 requests/2.21.0 CPython/3.6.7 Linux/3.10.0-693.11.6.el7.x86_64 rhel/7.4 glibc/2.17
UID:GID : 9100:9100
netrc file : None
offline mode : False

An unexpected error has occurred. Conda has prepared the above report.

11

I don't think you actually downloaded the file --- curl might not have followed the redirects. You can check by running:

cat qiime2-2019.1-py36-linux-conda.yml
12
Redirecting...

Redirecting...

You should be redirected automatically to target URL: https://raw.githubusercontent.com/qiime2/environment-files/master/2019.1/release/qiime2-2019.1-py36-linux-conda.yml. If not click the link.[swadmin@sirius qiime]$

13

Yep, just as I mentioned above, curl didn't follow the URL redirects.

14

Thanks
I think I have installed this successfully. ( I remember I have gone through this first time as well)

Verifying transaction: done
Executing transaction: - Enabling notebook extension jupyter-js-widgets/extension...
- Validating: OK
done

To activate this environment, use:

> source activate qiime2-2019.1

To deactivate an active environment, use:

> source deactivate

I qsub a pbs file. I have the same error

[qiuwa@sirius qiime2]$ more /scratch/qiuwa/qiime2-q2cli-err-0kdnj7tn.log
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_paired.R /scratch/qiuwa/tmpcjsadpp8/forward /scratch/qiuwa/tmpcjsadpp8/reverse /scratch/qiuwa/tmpcjsadpp8/output.tsv.biom /scratch/qiuwa/tmpcjsadpp8/track.tsv /scratch/qiuwa/tmpcjs
adpp8/filt_f /scratch/qiuwa/tmpcjsadpp8/filt_r 150 150 5 5 2.0 2 consensus 1.0 1 1000000

Traceback (most recent call last):
File "/gsfs0/public/qiime/miniconda3/envs/qiime2-2018.6/lib/python3.5/site-packages/q2cli/commands.py", line 274, in call
results = action(**arguments)
File "", line 2, in denoise_paired
File "/gsfs0/public/qiime/miniconda3/envs/qiime2-2018.6/lib/python3.5/site-packages/qiime2/sdk/action.py", line 232, in bound_callable
output_types, provenance)
File "/gsfs0/public/qiime/miniconda3/envs/qiime2-2018.6/lib/python3.5/site-packages/qiime2/sdk/action.py", line 367, in callable_executor
output_views = self._callable(**view_args)
File "/gsfs0/public/qiime/miniconda3/envs/qiime2-2018.6/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 229, in denoise_paired
run_commands([cmd])
File "/gsfs0/public/qiime/miniconda3/envs/qiime2-2018.6/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 36, in run_commands
subprocess.run(cmd, check=True)
File "/gsfs0/public/qiime/miniconda3/envs/qiime2-2018.6/lib/python3.5/subprocess.py", line 383, in run
with Popen(*popenargs, **kwargs) as process:
File "/gsfs0/public/qiime/miniconda3/envs/qiime2-2018.6/lib/python3.5/subprocess.py", line 676, in init
restore_signals, start_new_session)
File "/gsfs0/public/qiime/miniconda3/envs/qiime2-2018.6/lib/python3.5/subprocess.py", line 1289, in _execute_child
raise child_exception_type(errno_num, err_msg)
PermissionError: [Errno 13] Permission denied

somehow I had error for running:
[swadmin@sirius qiime] source activate qiime2-2019.1 activate: No such file or directory. [swadmin@sirius qiime] which activate
/usr/public/qiime/miniconda3//bin/activate

Should I put "source activate qiime2-2019.1" inside pbs file?

I run the following on the terminal. I had the same error.
[qiuwa@sirius qiime2]$ qiime dada2 denoise-paired
? --i-demultiplexed-seqs demux.qza
? --p-trim-left-f 5
? --p-trim-left-r 5
? --p-trunc-len-f 150
? --p-trunc-len-r 150
? --o-table table.qza
? --o-representative-sequences rep-seqs.qza
? --o-denoising-stats denoising-stats.qza

Plugin error from dada2:

[Errno 13] Permission denied

Debug info has been saved to /scratch/qiuwa/qiime2-q2cli-err-a7mwxm5i.log

Thanks

Wei

15

BTW. I can have this run.
[qiuwa@sirius qiime2]$ qiime --help
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
Usage: qiime [OPTIONS] COMMAND [ARGS]...

QIIME 2 command-line interface (q2cli)

To get help with QIIME 2, visit https://qiime2.org.

To enable tab completion in Bash, run the following command or add it to
your .bashrc/.bash_profile:

  source tab-qiime

To enable tab completion in ZSH, run the following commands or add them to
your .zshrc:

  autoload bashcompinit && bashcompinit && source tab-qiime

Options:
--version Show the version and exit.
--help Show this message and exit.

Commands:
info Display information about current deployment.
tools Tools for working with QIIME 2 files.
dev Utilities for developers and advanced users.
alignment Plugin for generating and manipulating alignments.
composition Plugin for compositional data analysis.
cutadapt Plugin for removing adapter sequences, primers, and
other unwanted sequence from sequence data.
dada2 Plugin for sequence quality control with DADA2.
deblur Plugin for sequence quality control with Deblur.
demux Plugin for demultiplexing & viewing sequence quality.
diversity Plugin for exploring community diversity.
emperor Plugin for ordination plotting with Emperor.
feature-classifier Plugin for taxonomic classification.
feature-table Plugin for working with sample by feature tables.
gneiss Plugin for building compositional models.
longitudinal Plugin for paired sample and time series analyses.
metadata Plugin for working with Metadata.
phylogeny Plugin for generating and manipulating phylogenies.
quality-control Plugin for quality control of feature and sequence data.
quality-filter Plugin for PHRED-based filtering and trimming.
sample-classifier Plugin for machine learning prediction of sample
metadata.
taxa Plugin for working with feature taxonomy annotations.
vsearch Plugin for clustering and dereplicating with vsearch.

16

The error is some kind of permission error - this isn't anything we can help with, unfortunately --- I suggest you talk to your system administrator.

17

I checked. All user have permission to access qiime. Can you point to me what kind of permission we need to set for which package or program? Where is the path of run_dada_paired.R? Wei

18

Yep, but that error indicates a disk permission error.

The disk permission error is likely one of two errors: you don't have permission to write locally (for example, you can't generate output files in your working directory) or the second is that you don't have permission to write to $TMPDIR.

Can you re-run the command with the --verbose flag?

19

I did set $TMPDIR . It worked well. Do you mind we have a quick chart?
Can you re-run the command with the --verbose flag? which command?

Thanks!

20

The failing command.