Help in performing alpha-rarefaction diversity analysis

vasanth_mandla · August 30, 2023, 3:41pm

I am using the emp-single-end-sequences dataset to run the Qiime2 workflow.

I was able to generate the phylogenetic tree for the diversity analysis. But, after this step I was unable to generate the visualization file (qzv) for the alpha-rarefaction. It is giving the error, as following. I have removed and reinstalled Qiime2, even then the error is not solved.

Error
Plugin error from diversity:

[Errno 8] Exec format error: 'faithpd'

Debug info has been saved to /tmp/qiime2-q2cli-err-_i4csh25.log

Command used:
qiime diversity alpha-rarefaction --i-table table-dada2.qza --i-phylogeny rooted-tree.qza --p-max-depth 4000 --m-metadata-file sample-metadata.tsv --o-visualization alpha-rarefaction.qzv --qverbose

Verbose Output:
(qiime2-2023.7) bioinfo@AHLL09533:~/meta_geneome$ qiime diversity alpha-rarefaction --i-table table-dada2.qza --i-phylogeny rooted-tree.qza --p-max-depth 4000 --m-metadata-file sample-metadata.tsv --o-visualization alpha-rarefaction.qzv --verbose
Running external command line application. This may print messages to stdout and/or stderr.
The command being run is below. This command cannot be manually re-run as it will depend on temporary files that no longer exist.

Command:

faithpd -i /tmp/qiime2/bioinfo/data/991d9934-d26f-4aa9-8f42-f228edadc328/data/feature-table.biom -t /tmp/qiime2/bioinfo/data/f51690f0-8a8a-46b3-a425-39e1d3b31dbd/data/tree.nwk -o /tmp/q2-AlphaDiversityFormat-u0w7r459

Traceback (most recent call last):
File "/home/bioinfo/anaconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/q2cli/commands.py", line 478, in call
results = self._execute_action(
File "/home/bioinfo/anaconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/q2cli/commands.py", line 539, in _execute_action
results = action(**arguments)
File "", line 2, in alpha_rarefaction
File "/home/bioinfo/anaconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/qiime2/sdk/action.py", line 342, in bound_callable
outputs = self.callable_executor(
File "/home/bioinfo/anaconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/qiime2/sdk/action.py", line 615, in callable_executor
ret_val = self._callable(output_dir=temp_dir, **view_args)
File "/home/bioinfo/anaconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/q2_diversity/_alpha/_visualizer.py", line 375, in alpha_rarefaction
data = _compute_rarefaction_data(table, min_depth, max_depth,
File "/home/bioinfo/anaconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/q2_diversity/_alpha/_visualizer.py", line 306, in _compute_rarefaction_data
vector, = alpha_phylo(table=rt, metric=metric,
File "/home/bioinfo/anaconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/qiime2/sdk/context.py", line 140, in deferred_action
return action_obj._bind(
File "", line 2, in alpha_phylogenetic
File "/home/bioinfo/anaconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/qiime2/sdk/action.py", line 342, in bound_callable
outputs = self.callable_executor(
File "/home/bioinfo/anaconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/qiime2/sdk/action.py", line 647, in callable_executor
outputs = self._callable(scope.ctx, **view_args)
File "/home/bioinfo/anaconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/q2_diversity/_alpha/_pipeline.py", line 16, in alpha_phylogenetic
vector, = action(table, phylogeny)
File "/home/bioinfo/anaconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/qiime2/sdk/context.py", line 140, in deferred_action
return action_obj._bind(
File "", line 2, in faith_pd
File "/home/bioinfo/anaconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/qiime2/sdk/action.py", line 342, in bound_callable
outputs = self.callable_executor(
File "/home/bioinfo/anaconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/qiime2/sdk/action.py", line 566, in callable_executor
output_views = self._callable(**view_args)
File "", line 2, in faith_pd
File "/home/bioinfo/anaconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/q2_diversity_lib/_util.py", line 69, in _disallow_empty_tables
return wrapped_function(*args, **kwargs)
File "", line 2, in faith_pd
File "/home/bioinfo/anaconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/q2_diversity_lib/_util.py", line 112, in _validate_requested_cpus
return wrapped_function(*bound_arguments.args, **bound_arguments.kwargs)
File "/home/bioinfo/anaconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/q2_diversity_lib/alpha.py", line 53, in faith_pd
_omp_cmd_wrapper(threads, cmd)
File "/home/bioinfo/anaconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/q2_diversity_lib/_util.py", line 128, in _omp_cmd_wrapper
return _run_external_cmd(cmd, verbose=verbose, env=env)
File "/home/bioinfo/anaconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/q2_diversity_lib/_util.py", line 122, in _run_external_cmd
return subprocess.run(cmd, check=True, env=env)
File "/home/bioinfo/anaconda3/envs/qiime2-2023.7/lib/python3.8/subprocess.py", line 493, in run
with Popen(*popenargs, **kwargs) as process:
File "/home/bioinfo/anaconda3/envs/qiime2-2023.7/lib/python3.8/subprocess.py", line 858, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File "/home/bioinfo/anaconda3/envs/qiime2-2023.7/lib/python3.8/subprocess.py", line 1704, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 8] Exec format error: 'faithpd'

Plugin error from diversity:

[Errno 8] Exec format error: 'faithpd'

See above for debug info.

Files in directory
(qiime2-2023.7) bioinfo@AHLL09533:~/meta_geneome$ ls -ltr
total 54624
-rw-r--r-- 1 bioinfo bioinfo 12885 Aug 26 17:00 qiime2-2023.7-py38-linux-conda.yml
-rw-r--r-- 1 bioinfo bioinfo 2094 Aug 26 17:35 sample-metadata.tsv
drwxr-xr-x 1 bioinfo bioinfo 4096 Aug 26 17:45 single_end_sequences
-rw-r--r-- 1 bioinfo bioinfo 29102196 Aug 29 17:15 empsingle-end-sequences.qza
-rw-r--r-- 1 bioinfo bioinfo 20603392 Aug 29 18:07 demux.qza
-rw-r--r-- 1 bioinfo bioinfo 3502245 Aug 29 18:07 demuxdetails.qza
-rw-r--r-- 1 bioinfo bioinfo 302578 Aug 29 18:09 demux.qzv
-rw-r--r-- 1 bioinfo bioinfo 54459 Aug 29 18:30 table-dada2.qza
-rw-r--r-- 1 bioinfo bioinfo 50610 Aug 29 18:30 rep-seqs-dada2.qza
-rw-r--r-- 1 bioinfo bioinfo 17883 Aug 29 18:30 stats-dada2.qza
-rw-r--r-- 1 bioinfo bioinfo 1223783 Aug 29 18:38 stats-dada2.qzv
-rw-r--r-- 1 bioinfo bioinfo 453782 Aug 30 10:29 table.qzv
-rw-r--r-- 1 bioinfo bioinfo 284451 Aug 30 10:57 rep-seqs.qzv
-rw-r--r-- 1 bioinfo bioinfo 62633 Aug 30 11:16 aligned-rep-seqs.qza
-rw-r--r-- 1 bioinfo bioinfo 65293 Aug 30 11:40 masked-aligned-rep-seqs.qza
-rw-r--r-- 1 bioinfo bioinfo 63171 Aug 30 11:40 unrooted-tree.qza
-rw-r--r-- 1 bioinfo bioinfo 67614 Aug 30 11:40 rooted-tree.qza

Qiime2 Version:
(qiime2-2023.7) bioinfo@AHLL09533:~/meta_geneome$ qiime info
System versions
Python version: 3.8.15
QIIME 2 release: 2023.7
QIIME 2 version: 2023.7.0
q2cli version: 2023.7.0

Installed plugins
alignment: 2023.7.0
composition: 2023.7.0
cutadapt: 2023.7.0
dada2: 2023.7.0
deblur: 2023.7.0
demux: 2023.7.0
diversity: 2023.7.0
diversity-lib: 2023.7.0
emperor: 2023.7.0
feature-classifier: 2023.7.0
feature-table: 2023.7.0
fragment-insertion: 2023.7.0
gneiss: 2023.7.0
longitudinal: 2023.7.0
metadata: 2023.7.0
phylogeny: 2023.7.0
quality-control: 2023.7.0
quality-filter: 2023.7.0
sample-classifier: 2023.7.0
taxa: 2023.7.0
types: 2023.7.0
vsearch: 2023.7.0

Application config directory
/home/bioinfo/anaconda3/envs/qiime2-2023.7/var/q2cli

Getting help
To get help with QIIME 2, visit https://qiime2.org

Qiime2 Installation Method
Conda
Windows 10 Pro --> Ubuntu --> conda --> Qiime2

Ubuntu Version
Ubuntu 20.04.6 LTS (GNU/Linux 4.4.0-19041-Microsoft x86_64)

command:
conda env create -n qiime2-2023.7 --file qiime2-2023.7-py38-linux-conda.yml

Please, let me know if you need any further information
Thanks in advance

colinvwood · August 30, 2023, 6:04pm

Hello @vasanth_mandla,

What happens when you try to run the faithpd command alone in your environment?

vasanth_mandla · August 31, 2023, 8:02am

Hi @colinvwood ,

Thank you, for your response.

The following is the output:

-bash: /home/bioinfo/anaconda3/envs/qiime2-2023.7/bin/faithpd: cannot execute binary file: Exec format error

colinvwood · August 31, 2023, 4:13pm

Hello @vasanth_mandla,

It looks like you installed a binary that is incompatible with your machine somehow. You ran all of these commands inside of WSL with the qiime environment activated, correct?

Is it possible you followed installation instructions for a different architecture in the docs?

colinvwood · August 31, 2023, 4:16pm

Hello @5cr34m,

The post you linked discusses a different error message from a different command. We're grateful for help on the forum, but please be a little more careful about making sure your responses are on topic.

vasanth_mandla · September 1, 2023, 7:22am

Dear @colinvwood,

Yes, you are right. I have installed the Ubuntu(20.04.6 LTS) in windows 10, installed the Qiime and followed the tutorial.

I have realised my mistake of installing qiime, by following instructions from the following link:
Natively installing QIIME 2 — QIIME 2 2023.7.0 documentation (Linux)

I have reinstalled qiime by following instruction from the tab Windows (via WSL) and performed the analysis similarly (to my first post) and got stuck at the same step.

The error output when I ran faithpd command alone is same as previously mentioned.

I have provided the "conda info" and files used in rarefaction analysis, for reference.

rooted-tree.qza (66.0 KB)
table.qza (53.2 KB)

conda info from new qiime(WSL) environment

(qiime2-2023.7_WSL) bioinfo@AHLL09533:~/meta_geneome/meta_test$ conda info

 active environment : qiime2-2023.7_WSL
active env location : /home/bioinfo/anaconda3/envs/qiime2-2023.7_WSL
        shell level : 2
   user config file : /home/bioinfo/.condarc

populated config files :
conda version : 23.7.3
conda-build version : 3.26.0
python version : 3.11.4.final.0
virtual packages : __archspec=1=x86_64
__glibc=2.31=0
__linux=4.4.0=0
__unix=0=0
base environment : /home/bioinfo/anaconda3 (writable)
conda av data dir : /home/bioinfo/anaconda3/etc/conda
conda av metadata url : None
channel URLs : main/linux-64
main/noarch
r/linux-64
r/noarch
package cache : /home/bioinfo/anaconda3/pkgs
/home/bioinfo/.conda/pkgs
envs directories : /home/bioinfo/anaconda3/envs
/home/bioinfo/.conda/envs
platform : linux-64
user-agent : conda/23.7.3 requests/2.31.0 CPython/3.11.4 Linux/4.4.0-19041-Microsoft ubuntu/20.04.6 glibc/2.31
UID:GID : 1000:1000
netrc file : None
offline mode : False

5cr34m · September 1, 2023, 1:56pm

Sorry, my fault.
I'll try to.

colinvwood · September 1, 2023, 5:06pm

Hello @vasanth_mandla,

What does file $(which faithpd) output?

vasanth_mandla · September 2, 2023, 5:40am

Hi @colinvwood ,

The following is the output:

(qiime2-2023.7_WSL) bioinfo@AHLL09533:~/meta_geneome/meta_test$ file $(which faithpd)
/home/bioinfo/anaconda3/envs/qiime2-2023.7_WSL/bin/faithpd: symbolic link to faithpd_nv

Looks like, it does take the faithpd from where it supposed to. Did you get a chance to look at the files that I have provided. If so, did you find anything sceptical.

Thanks, for you time.

colinvwood · September 5, 2023, 4:46pm

Hello @vasanth_mandla,

Can you run file $(which faithpd_nv)?

vasanth_mandla · September 7, 2023, 5:06am

Hi @colinvwood ,

Below is the output:

(qiime2-2023.7_WSL) bioinfo@AHLL09533:~/meta_geneome$ file $(which faithpd_nv)
/home/bioinfo/anaconda3/envs/qiime2-2023.7_WSL/bin/faithpd_nv: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.18, with debug_info, not stripped

colinvwood · September 7, 2023, 4:52pm

Hello @vasanth_mandla,

And what does uname -m output in your linux terminal?

vasanth_mandla · September 8, 2023, 5:54am

Hi @colinvwood,

Output:
(qiime2-2023.7_WSL) bioinfo@AHLL09533:~$ uname -m
x86_64

colinvwood · September 8, 2023, 5:37pm

Hello @vasanth_mandla,

A possible solution is found here.

vasanth_mandla · September 14, 2023, 6:31am

Dear @colinvwood ,

I did try to resolve it by following your suggestion but, it did not solve the problem.

So, I have re-installed conda and qiime2 environment after completely removing them. Then the diversity analysis command started working as intended. In addition I have upgraded the WLS to version 2.

Thanks, for your time and patience in helping to resolve it.

system · October 15, 2023, 12:32pm

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.