Hello, I am from China, I am a newbie in qiime2, I am learning the data of qiime2, I have a problem when facing qiime diversity core-metrics-phylogenetic, I found that many people in the forum have encountered this problem It has not been solved yet. I would like to ask if there is a clear solution now.
(qiime2-2023.9) zhang_chao 16:29:12 ~/qiime2/test/mp
$ qiime diversity core-metrics-phylogenetic \
> --i-phylogeny rooted-tree.qza \
> --i-table table.qza \
> --p-sampling-depth 1103 \
> --m-metadata-file sample-metadata.tsv \
> --output-dir core-metrics-results
Plugin error from diversity:
Command '['ssu', '-i', '/tmp/qiime2/zhang_chao/data/7e26fa27-228d-4058-b087-fd74caee89e7/data/feature-table.biom', '-t', '/tmp/qiime2/zhang_chao/data/efd5331a-8cad-4464-b567-085caa2ade33/data/tree.nwk', '-m', 'unweighted', '-o', '/tmp/q2-LSMatFormat-zhlmh9vk']' returned non-zero exit status 1.
Debug info has been saved to /tmp/qiime2-q2cli-err-64ca1r3w.log
Hello! Thank you for your reply. Due to the time difference, I am willing to spoof you for being slower. Sorry! The following is the code information
(qiime2-2023.9) zhang_chao 08:39:30 ~/qiime2/test/mp
$ qiime diversity core-metrics-phylogenetic \
> --i-phylogeny rooted-tree.qza \
> --i-table table.qza \
> --p-sampling-depth 1103 \
> --m-metadata-file sample-metadata.tsv \
> --output-dir core-metrics-results \
> --verbose
Running external command line application. This may print messages to stdout and/or stderr.
The command being run is below. This command cannot be manually re-run as it will depend on temporary files that no longer exist.
Command:
faithpd -i /tmp/qiime2/zhang_chao/data/7e26fa27-228d-4058-b087-fd74caee89e7/data/feature-table.biom -t /tmp/qiime2/zhang_chao/data/efd5331a-8cad-4464-b567-085caa2ade33/data/tree.nwk -o /tmp/q2-AlphaDiversityFormat-50k_jrf9
Running external command line application. This may print messages to stdout and/or stderr.
The command being run is below. This command cannot be manually re-run as it will depend on temporary files that no longer exist.
Command:
ssu -i /tmp/qiime2/zhang_chao/data/7e26fa27-228d-4058-b087-fd74caee89e7/data/feature-table.biom -t /tmp/qiime2/zhang_chao/data/efd5331a-8cad-4464-b567-085caa2ade33/data/tree.nwk -m unweighted -o /tmp/q2-LSMatFormat-9au4ecps
Failing in Thread:0
call to cuInit returned error 804: Other
Traceback (most recent call last):
File "/titan/zhang_chao/anaconda3/envs/qiime2-2023.9/lib/python3.8/site-packages/q2cli/commands.py", line 520, in __call__
results = self._execute_action(
File "/titan/zhang_chao/anaconda3/envs/qiime2-2023.9/lib/python3.8/site-packages/q2cli/commands.py", line 586, in _execute_action
results = action(**arguments)
File "<decorator-gen-189>", line 2, in core_metrics_phylogenetic
File "/titan/zhang_chao/anaconda3/envs/qiime2-2023.9/lib/python3.8/site-packages/qiime2/sdk/action.py", line 342, in bound_callable
outputs = self._callable_executor_(
File "/titan/zhang_chao/anaconda3/envs/qiime2-2023.9/lib/python3.8/site-packages/qiime2/sdk/action.py", line 647, in _callable_executor_
outputs = self._callable(scope.ctx, **view_args)
File "/titan/zhang_chao/anaconda3/envs/qiime2-2023.9/lib/python3.8/site-packages/q2_diversity/_core_metrics.py", line 66, in core_metrics_phylogenetic
dms += unweighted_unifrac(table=cr.rarefied_table, phylogeny=phylogeny,
File "/titan/zhang_chao/anaconda3/envs/qiime2-2023.9/lib/python3.8/site-packages/qiime2/sdk/context.py", line 140, in deferred_action
return action_obj._bind(
File "<decorator-gen-711>", line 2, in unweighted_unifrac
File "/titan/zhang_chao/anaconda3/envs/qiime2-2023.9/lib/python3.8/site-packages/qiime2/sdk/action.py", line 342, in bound_callable
outputs = self._callable_executor_(
File "/titan/zhang_chao/anaconda3/envs/qiime2-2023.9/lib/python3.8/site-packages/qiime2/sdk/action.py", line 566, in _callable_executor_
output_views = self._callable(**view_args)
File "<decorator-gen-143>", line 2, in unweighted_unifrac
File "/titan/zhang_chao/anaconda3/envs/qiime2-2023.9/lib/python3.8/site-packages/q2_diversity_lib/_util.py", line 69, in _disallow_empty_tables
return wrapped_function(*args, **kwargs)
File "<decorator-gen-142>", line 2, in unweighted_unifrac
File "/titan/zhang_chao/anaconda3/envs/qiime2-2023.9/lib/python3.8/site-packages/q2_diversity_lib/_util.py", line 112, in _validate_requested_cpus return wrapped_function(*bound_arguments.args, **bound_arguments.kwargs)
File "/titan/zhang_chao/anaconda3/envs/qiime2-2023.9/lib/python3.8/site-packages/q2_diversity_lib/beta.py", line 221, in unweighted_unifrac
_omp_cmd_wrapper(threads, cmd)
File "/titan/zhang_chao/anaconda3/envs/qiime2-2023.9/lib/python3.8/site-packages/q2_diversity_lib/_util.py", line 128, in _omp_cmd_wrapper
return _run_external_cmd(cmd, verbose=verbose, env=env)
File "/titan/zhang_chao/anaconda3/envs/qiime2-2023.9/lib/python3.8/site-packages/q2_diversity_lib/_util.py", line 122, in _run_external_cmd
return subprocess.run(cmd, check=True, env=env)
File "/titan/zhang_chao/anaconda3/envs/qiime2-2023.9/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ssu', '-i', '/tmp/qiime2/zhang_chao/data/7e26fa27-228d-4058-b087-fd74caee89e7/data/feature-table.biom', '-t', '/tmp/qiime2/zhang_chao/data/efd5331a-8cad-4464-b567-085caa2ade33/data/tree.nwk', '-m', 'unweighted', '-o', '/tmp/q2-LSMatFormat-9au4ecps']' returned non-zero exit status 1.
Plugin error from diversity:
Command '['ssu', '-i', '/tmp/qiime2/zhang_chao/data/7e26fa27-228d-4058-b087-fd74caee89e7/data/feature-table.biom', '-t', '/tmp/qiime2/zhang_chao/data/efd5331a-8cad-4464-b567-085caa2ade33/data/tree.nwk', '-m', 'unweighted', '-o', '/tmp/q2-LSMatFormat-9au4ecps']' returned non-zero exit status 1.
See above for debug info.
Failing in Thread:0
call to cuInit returned error 804: Other
suggests that this is an Nvidia related issue. cuInit stands for "cuda init" or something along those lines and cuda is Nvidia's language for programming their GPUs. Am I correct in assuming you have access to an Nvidia GPU in that machine? If so, ssu will attempt to use it by default, but that doesn't appear to be working.
I can't get it to link directly to your specific error, but if you go on this page and look for "804" you can see the error you're getting. It looks like it has something to do with an incompatibility between the physical GPU and the drivers on the machine. I would start by uninstalling the current GPU drivers and installing whatever the latest driver version is for the specific GPU model you have. If that doesn't work then you'll probably need to post on the Nvidia help forum and ask them.
Could you describe the system you're using? A few things that would be helpful are the output of head /proc/cpuinfo and uname -a. Any information about the operating system version (requesting this can vary on flavors of Linux) would be helpful as well. And are there GPUs on the system where the command was performed, and if this is a cluster, is the associated submission requesting GPU resources?
Additionally, could you try running export UNIFRAC_USE_GPU=N, and then rerun the call to core-metrics-phylogenetic?
Thanks for the answer, here is the configuration of my server
Welcome to Ubuntu 20.04.4 LTS (GNU/Linux 5.14.0-1054-oem x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
105 更新可以立即应用。
这些更新中有 1 个是标准安全更新。
要查看这些附加更新，请运行：apt list --upgradable
1 updates could not be installed automatically. For more details,
see /var/log/unattended-upgrades/unattended-upgrades.log
Your Hardware Enablement Stack (HWE) is supported until April 2025.
*** System restart required ***
Last login: Fri Jan 5 13:20:24 2024 from 10.1.42.2
架构： x86_64
CPU 运行模式： 32-bit, 64-bit
字节序： Little Endian
Address sizes: 46 bits physical, 48 bits virtual
CPU: 20
在线 CPU 列表： 0-19
每个核的线程数： 2
每个座的核数： 10
座： 1
NUMA 节点： 1
厂商 ID： GenuineIntel
CPU 系列： 6
型号： 85
型号名称： Intel(R) Xeon(R) Silver 4210R CPU @ 2.40GHz
步进： 7
CPU MHz： 2400.000
CPU 最大 MHz： 3200.0000
CPU 最小 MHz： 1000.0000
BogoMIPS： 4800.00
虚拟化： VT-x
L1d 缓存： 320 KiB
L1i 缓存： 320 KiB
L2 缓存： 10 MiB
L3 缓存： 13.8 MiB
NUMA 节点0 CPU： 0-19
Vulnerability Itlb multihit: KVM: Mitigation: VMX disabled
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Mitigation; Clear CPU buffers; SMT vulnerable
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Enhanced IBRS, IBPB conditional, RSB filling
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Mitigation; TSX disabled
标记： fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid ap erfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2 apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_ l3 invpcid_single intel_ppin ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsg sbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm mpx rdt_a avx512f avx512dq rdseed adx smap clflushopt clwb int el_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dt herm ida arat pln pts hwp hwp_act_window hwp_epp hwp_pkg_req pku ospke avx512_vnni md_clear flush_l1d arch_capabilit ies
I gave up on qiime2 because it has many bugs that cannot be solved. I don't understand why so many people are still requesting to use qiime2. I switched to vserch and userch, and qiime2 is a waste of my time
To clarify here @linke0909 the problem you're seeing has nothing to do with QIIME 2 itself. Your GPU isn't configured correctly. It looks like the wrong version of your GPU drivers was installed. It's unlikely that any application attempting to use the GPU in your system will work until this is resolved.
Also to clarify what @wasade was requesting here. Please first run the command export UNIFRAC_USE_GPU=N as its own command. This will tell unifrac not to use your GPU because your GPU appears to be misconfigured. Then rerun your qiime core-metrics-phylogenetic command as a separate command.
Thank you for your reply. Yes, I have listened to your suggestion. I did not use a server with GPU configuration issues. Instead, I tried installing Ubuntu using the Linux subsystem under Windows, but encountered a new problem while using Dada2. I checked the forum and found that the r path was incorrect, but there is only one r path in my Ubuntu. I have also tried to modify it, change the version, and reinstall it, In short, during the use of qiime2, I feel that it is very unsuitable, and I have also found many unresolved similar issues while browsing forums. The above is my personal opinion.
(qiime2-2023.9) zhangchao@DESKTOP-NV32H8O:~/mp$ qiime dada2 denoise-single \
> --i-demultiplexed-seqs demux.qza \
> --p-trim-left 0 \
> --p-trunc-len 120 \
> --o-representative-sequences rep-seqs-dada2.qza \
> --o-table table-dada2.qza \
> --o-denoising-stats stats-dada2.qza\
> --verbose
Plugin error from dada2:
An error was encountered while running DADA2 in R (return code 255), please inspect stdout and stderr to learn more.
Debug info has been saved to /tmp/qiime2-q2cli-err-yyak3ydm.log
(qiime2-2023.9) zhangchao@DESKTOP-NV32H8O:~/mp$ which R
/home/zhangchao/anaconda3/envs/qiime2-2023.9/bin/R
Because there is no space before the \ your terminal interpreted this as --o-denoising-stats stats-dada2.qza--verbose. What the \ in a terminal does is basically tell the terminal "the next line should be treated as part of this same line," so in the end all the lines with \ get combined into one line, so you still need to put a space there.
If you are still interested in debugging this please run the command with the --verbose flag passed correctly.
If you do run the command with --verbose my guess is you will see a line saying something along the lines of Rscript execution error: No such file or directory. This could be due to QIIME 2 having installed incorrectly, but we most often see errors like this when there are competing versions of R installed on a system. R does not like to respect people's virtual environments and will frequently cause headaches by deciding not to let QIIME 2 run the version of R it wants to because R thinks it knows better (it doesn't).
If you run which R with your environment active you should see what you are seeing
(qiime2-2023.9) zhangchao@DESKTOP-NV32H8O:~/mp$ which R
/home/zhangchao/anaconda3/envs/qiime2-2023.9/bin/R
You should see something similar if you run which Rscript with your environment active. If you run which R or which Rscript without your environment active then, assuming you don't have R installed on your computer outside of your conda environment, you should see nothing.
If you do see nothing then I would remove your environment and recreate it making sure to follow the install instructions found here under "WIndows (via WSL)" (we frequently have people follow install instructions that were created by people other than us then have issues with their environment).
If you do see versions of R and Rscript outside of your conda environment, then remove them if you don't need them. If you do still need them then it will take a bit more fiddling to convince R to listen, and we can deal with that if needed.
Thank you for your reply. The result is as you guessedRscript execution error: No such file or directory. How can I solve this problem
(qiime2-2023.9) zhangchao@DESKTOP-NV32H8O:~/mp$ qiime dada2 denoise-single \
> --i-demultiplexed-seqs demux.qza \
> --p-trim-left 0 \
> --p-trunc-len 120 \
> --o-representative-sequences rep-seqs-dada2.qza \
> --o-table table-dada2.qza \
> --o-denoising-stats stats-dada2.qza --verbose
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.
Command: run_dada.R --input_directory /tmp/qiime2/zhangchao/data/e59ec413-3ba5-45c7-93f7-cc03b0b607ce/data --output_path /tmp/tmpe7zj4i1g/output.tsv.biom --output_track /tmp/tmpe7zj4i1g/track.tsv --filtered_directory /tmp/tmpe7zj4i1g --truncation_length 120 --trim_left 0 --max_expected_errors 2.0 --truncation_quality_score 2 --max_length Inf --pooling_method independent --chimera_method consensus --min_parental_fold 1.0 --allow_one_off False --num_threads 1 --learn_min_reads 1000000 --homopolymer_gap_penalty NULL --band_size 16
Rscript execution error: No such file or directory
Traceback (most recent call last):
File "/home/zhangchao/anaconda3/envs/qiime2-2023.9/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 220, in _denoise_single
run_commands([cmd])
File "/home/zhangchao/anaconda3/envs/qiime2-2023.9/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 36, in run_commands
subprocess.run(cmd, check=True)
File "/home/zhangchao/anaconda3/envs/qiime2-2023.9/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['run_dada.R', '--input_directory', '/tmp/qiime2/zhangchao/data/e59ec413-3ba5-45c7-93f7-cc03b0b607ce/data', '--output_path', '/tmp/tmpe7zj4i1g/output.tsv.biom', '--output_track', '/tmp/tmpe7zj4i1g/track.tsv', '--filtered_directory', '/tmp/tmpe7zj4i1g', '--truncation_length', '120', '--trim_left', '0', '--max_expected_errors', '2.0', '--truncation_quality_score', '2', '--max_length', 'Inf', '--pooling_method', 'independent', '--chimera_method', 'consensus', '--min_parental_fold', '1.0', '--allow_one_off', 'False', '--num_threads', '1', '--learn_min_reads', '1000000', '--homopolymer_gap_penalty', 'NULL', '--band_size', '16']' returned non-zero exit status 255.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/zhangchao/anaconda3/envs/qiime2-2023.9/lib/python3.8/site-packages/q2cli/commands.py", line 520, in __call__
results = self._execute_action(
File "/home/zhangchao/anaconda3/envs/qiime2-2023.9/lib/python3.8/site-packages/q2cli/commands.py", line 581, in _execute_action
results = action(**arguments)
File "<decorator-gen-70>", line 2, in denoise_single
File "/home/zhangchao/anaconda3/envs/qiime2-2023.9/lib/python3.8/site-packages/qiime2/sdk/action.py", line 342, in bound_callable
outputs = self._callable_executor_(
File "/home/zhangchao/anaconda3/envs/qiime2-2023.9/lib/python3.8/site-packages/qiime2/sdk/action.py", line 566, in _callable_executor_
output_views = self._callable(**view_args)
File "/home/zhangchao/anaconda3/envs/qiime2-2023.9/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 244, in denoise_single
return _denoise_single(
File "/home/zhangchao/anaconda3/envs/qiime2-2023.9/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 229, in _denoise_single
raise Exception("An error was encountered while running DADA2"
Exception: An error was encountered while running DADA2 in R (return code 255), please inspect stdout and stderr to learn more.
Plugin error from dada2:
An error was encountered while running DADA2 in R (return code 255), please inspect stdout and stderr to learn more.
See above for debug info.
Can you clarify whether the issues you're running into with the DADA2 plugin are in the environment where you made these changes, or if they are in a freshly installed QIIME 2 environment?
Hello, thank you for your reply. I have encountered an issue with Dada2 in the newly installed environment. Currently, I have tried three methods to test the data: WSL, virtual machine, server, server graphics card issues, virtual machine and WSL, and Dada2 errors. Now I can only run it separately, using the server in the first half and importing files into WSL for analysis in the second half. The entire process runs quickly in 5 minutes, but this problem still persists. I want to know how to avoid dada2 errors because there are also many people who encounter these problems without clear answers, which will discourage many beginners and ultimately give up qiime2. This is something I don't want to see, After all, qiime2 is indeed a great invention!
Please do the following: With the failing conda environment active run both which R and which Rscript then run env and post the entire output from all three commands here. It should look something like this
1: Your R installation in your environment is correct
2: You do not have an R installation outside of your environment that is potentially competing with the R installation inside of your environment
