I am running into an issue calculating core diversity metrics.
Here is my current configuration:
System versions
Python version: 3.8.13
QIIME 2 release: 2022.8
QIIME 2 version: 2022.8.3
q2cli version: 2022.8.0
Installed plugins
alignment: 2022.8.0
composition: 2022.8.0
cutadapt: 2022.8.0
dada2: 2022.8.0
deblur: 2022.8.0
demux: 2022.8.0
diversity: 2022.8.0
diversity-lib: 2022.8.0
emperor: 2022.8.0
feature-classifier: 2022.8.0
feature-table: 2022.8.1
fragment-insertion: 2022.8.0
gneiss: 2022.8.0
longitudinal: 2022.8.0
metadata: 2022.8.0
phylogeny: 2022.8.0
quality-control: 2022.8.0
quality-filter: 2022.8.0
sample-classifier: 2022.8.0
taxa: 2022.8.0
types: 2022.8.0
vsearch: 2022.8.0
I originally ran the following command:
qiime diversity core-metrics-phylogenetic \
--i-phylogeny rooted-tree-dada2.qza \
--i-table table-dada2.qza \
--p-sampling-depth 5260 \
--m-metadata-file ../16s.its.9.26.2022.Metadata.Hamilton_corrected_data.tab \
--output-dir core-metrics-results \
--verbose
Here is the verbose output:
/home/vinson/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/sklearn/metrics/pairwise.py:1776: DataConversionWarning: Data was converted to boolean for metric jaccard
warnings.warn(msg, DataConversionWarning)
Running external command line application. This may print messages to stdout and/or stderr.
The command being run is below. This command cannot be manually re-run as it will depend on temporary files that no longer exist.Command:
faithpd -i /tmp/qiime2/vinson/data/c4c939d9-94a7-472d-91d4-7ab17408b769/data/feature-table.biom -t /tmp/qiime2/vinson/data/64121c21-c122-40f5-8391-9359c55a7e09/data/tree.nwk -o /tmp/q2-AlphaDiversityFormat-bejy74cw
Running external command line application. This may print messages to stdout and/or stderr.
The command being run is below. This command cannot be manually re-run as it will depend on temporary files that no longer exist.Command:
ssu -i /tmp/qiime2/vinson/data/c4c939d9-94a7-472d-91d4-7ab17408b769/data/feature-table.biom -t /tmp/qiime2/vinson/data/64121c21-c122-40f5-8391-9359c55a7e09/data/tree.nwk -m unweighted -o /tmp/q2-LSMatFormat-6aa_twx3
Current file: /opt/conda/conda-bld/unifrac-binaries_1659558422263/work/src/unifrac_cmp.cpp
function: _ZN6su_acc21UnifracUnweightedTaskIdE4_runEjPKd
line: 558
This file was compiled: -ta=tesla:cc35,cc50,cc60,cc60,cc70,cc75,cc80,cc80
Traceback (most recent call last):
File "/home/vinson/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/q2cli/commands.py", line 339, in call
results = action(**arguments)
File "", line 2, in core_metrics_phylogenetic
File "/home/vinson/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/qiime2/sdk/action.py", line 234, in bound_callable
outputs = self.callable_executor(scope, callable_args,
File "/home/vinson/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/qiime2/sdk/action.py", line 475, in callable_executor
outputs = self._callable(scope.ctx, **view_args)
File "/home/vinson/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/q2_diversity/_core_metrics.py", line 66, in core_metrics_phylogenetic
dms += unweighted_unifrac(table=cr.rarefied_table, phylogeny=phylogeny,
File "", line 2, in unweighted_unifrac
File "/home/vinson/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/qiime2/sdk/action.py", line 234, in bound_callable
outputs = self.callable_executor(scope, callable_args,
File "/home/vinson/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/qiime2/sdk/action.py", line 381, in callable_executor
output_views = self._callable(**view_args)
File "", line 2, in unweighted_unifrac
File "/home/vinson/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/q2_diversity_lib/_util.py", line 69, in _disallow_empty_tables
return wrapped_function(*args, **kwargs)
File "", line 2, in unweighted_unifrac
File "/home/vinson/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/q2_diversity_lib/_util.py", line 112, in _validate_requested_cpus
return wrapped_function(*bound_arguments.args, **bound_arguments.kwargs)
File "/home/vinson/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/q2_diversity_lib/beta.py", line 221, in unweighted_unifrac
_omp_cmd_wrapper(threads, cmd)
File "/home/vinson/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/q2_diversity_lib/_util.py", line 128, in _omp_cmd_wrapper
return _run_external_cmd(cmd, verbose=verbose, env=env)
File "/home/vinson/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/q2_diversity_lib/_util.py", line 122, in _run_external_cmd
return subprocess.run(cmd, check=True, env=env)
File "/home/vinson/miniconda3/envs/qiime2-2022.8/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ssu', '-i', '/tmp/qiime2/vinson/data/c4c939d9-94a7-472d-91d4-7ab17408b769/data/feature-table.biom', '-t', '/tmp/qiime2/vinson/data/64121c21-c122-40f5-8391-9359c55a7e09/data/tree.nwk', '-m', 'unweighted', '-o', '/tmp/q2-LSMatFormat-6aa_twx3']' returned non-zero exit status 1.Plugin error from diversity:
Command '['ssu', '-i', '/tmp/qiime2/vinson/data/c4c939d9-94a7-472d-91d4-7ab17408b769/data/feature-table.biom', '-t', '/tmp/qiime2/vinson/data/64121c21-c122-40f5-8391-9359c55a7e09/data/tree.nwk', '-m', 'unweighted', '-o', '/tmp/q2-LSMatFormat-6aa_twx3']' returned non-zero exit status 1.
See above for debug info.
I then decided to run several alpha and beta-diversity metrics individually. There were no problems with qiime diversity alpha or alpha-phylogenetic distances (observed_features, faith_pd, simpson_e). There were no problems with qiime diversity beta (braycurtis - once I removed samples with zero features). However, I did run into a problem when trying to run qiime diversity beta-phylogenetic (unweighted unifrac).
Here is the command:
qiime diversity beta-phylogenetic \
--i-table table-dada2.qza \
--i-phylogeny rooted-tree-dada2.qza \
--p-metric unweighted_unifrac \
--o-distance-matrix diversity-metrics/unweighted_unifrac_distance_matrix.qza
Here is the error:
Command '['ssu', '-i', '/tmp/qiime2/vinson/data/17621874-0a08-46c4-9593-cda227972ecc/data/feature-table.biom', '-t', '/tmp/qiime2/vinson/data/64121c21-c122-40f5-8391-9359c55a7e09/data/tree.nwk', '-m', 'unweighted', '-o', '/tmp/q2-LSMatFormat-twh939bi']' returned non-zero exit status 1.
Debug info has been saved to /tmp/qiime2-q2cli-err-4losvsrf.log
Not verbose, but looks the same as the previous error when calculating all metrics.
SO, the next thing I checked was if ssu was installed and running properly. It is. There is no version information for ssu, but here is the usage to confirm I have it installed:
usage: ssu -i -o <out.dm> -m [METHOD] -t [-n threads] [-a alpha] [-f] [--vaw]
[--mode [MODE]] [--start starting-stripe] [--stop stopping-stripe] [--partial-pattern ]
[--n-partials number_of_partitions] [--report-bare] [--format|-r out-mode]
So, I converted my feature table to BIOM and tree (qza) to newick with qiime before running ssu.
mkdir biom
qiime tools export \
--input-path table-dada2.qza \
--output-path biom
mkdir rooted-tree-dada2-nwk
qiime tools export \
--input-path rooted-tree-dada2.qza \
--output-path rooted-tree-dada2-nwk
mkdir ssu.output
ssu -i biom/feature-table.biom -t rooted-tree-dada2-nwk/tree.nwk -m unweighted -o /home/vinson/qiime2analyses/qiime.phragmites/16s.its.9.26.2022/Q1.analyses/ssu.output/ssu.out.dm
It seems to have run fine, given the output:
Current file: /opt/conda/conda-bld/unifrac-binaries_1659558422263/work/src/unifrac_cmp.cpp
function: _ZN6su_acc21UnifracUnweightedTaskIdE4_runEjPKd
line: 558
This file was compiled: -ta=tesla:cc35,cc50,cc60,cc60,cc70,cc75,cc80,cc80
However, I can't find a distance matrix anywhere.
Furthermore, I am not sure where to go from here.
I saw another posting about this at plugin error from diversity - going through tutorial, but it was closed without resolution because the OP did not respond, I think.
Thanks for any help.