Qiime Plugin error from fragment-insertion

Noor_Us-Subah · August 15, 2022, 6:47pm

Qiime installed through conda
Version: qiime2-2022.2

qiime fragment-insertion sepp
--i-representative-sequences  ./representative_sequences.qza
--i-reference-database sepp-refs-gg-13-8.qza
--o-tree treeNew.qza
--o-placements tree_placementsNew.qza
--p-threads 1
--verbose
--p-debug

Plugin error from fragment-insertion:

Command '['run-sepp.sh', '/tmp/qiime2-archive-ng78u86d/a733267f-f35c-47be-8ede-9005a96aca8b/data/dna-sequences.fasta', 'q2-fragment-insertion', '-x', '1', '-A', '1000', '-P', '5000', '-a', '/tmp/qiime2-archive-eiz695x7/a14c6180-506b-4ecb-bacb-9cb30bc3044b/data/aligned-dna-sequences.fasta', '-t', '/tmp/qiime2-archive-eiz695x7/a14c6180-506b-4ecb-bacb-9cb30bc3044b/data/tree.nwk', '-r', '/tmp/qiime2-archive-eiz695x7/a14c6180-506b-4ecb-bacb-9cb30bc3044b/data/raxml-info.txt', '-b', '1']' returned non-zero exit status 1.

See above for debug info.

Oddant1 · August 22, 2022, 3:44pm

Hello @Noor_Us-Subah, it looks like you left out the traceback. Can you please rerun the command and post everything here including the part after "Traceback (most recent call last):?" We need that information to debug what's happening. Thank you!

Noor_Us-Subah · August 23, 2022, 7:23am

Hi @Oddant1 I am pasting the Traceback part, the other lines are the same as above but still attaching the text file that contains whole error part

Traceback part:

Traceback (most recent call last):
File "/home/noor/anaconda3/envs/qiime2-2022.2/lib/python3.8/site-packages/sepp/jobs.py", line 150, in run
raise JobError("\n".join([
sepp.scheduler.JobError: The following execution failed:
/home/noor/anaconda3/envs/qiime2-2022.2/bin/pplacer --out-dir /tmp/tmp.fGBexxwP3f/sepp-tempssd-Z9x3LWcQCU/q2-fragment-insertion.z8d9c3_r/root/P_1 -j 1 -r /tmp/tmp.fGBexxwP3f/sepp-tempssd-Z9x3LWcQCU/q2-fragment-insertion.z8d9c3_r/root/P_1/pplacer.backbone.un0pvqjb.fasta -s /tmp/qiime2-archive-l4gp450y/a14c6180-506b-4ecb-bacb-9cb30bc3044b/data/raxml-info.txt -t /tmp/tmp.fGBexxwP3f/sepp-tempssd-Z9x3LWcQCU/q2-fragment-insertion.z8d9c3_r/root/P_1/pplacer.tree.xqlloe53.tre --groups 10 /tmp/tmp.fGBexxwP3f/sepp-tempssd-Z9x3LWcQCU/q2-fragment-insertion.z8d9c3_r/root/P_1/pplacer.extended.0.rdrmprtj.fasta
No error messages available
[12:09:22] jobs.py (line 80): DEBUG: Starting pplacer Job with input: backbone_alignment_file:/tmp/tmp.fGBexxwP3f/sepp-tempssd-Z9x3LWcQCU/q2-fragment-insertion.z8d9c3_r/root/P_2/pplacer.backbone.f_ju2ji4.fasta, tree_file:/tmp/tmp.fGBexxwP3f/sepp-tempssd-Z9x3LWcQCU/q2-fragment-insertion.z8d9c3_r/root/P_2/pplacer.tree.2pvy2nmf.tre, info_file:/tmp/qiime2-archive-l4gp450y/a14c6180-506b-4ecb-bacb-9cb30bc3044b/data/raxml-info.txt, extended alignment:/tmp/tmp.fGBexxwP3f/sepp-tempssd-Z9x3LWcQCU/q2-fragment-insertion.z8d9c3_r/root/P_2/pplacer.extended.0.zm0k4wyo.fasta, output:/tmp/tmp.fGBexxwP3f/sepp-tempssd-Z9x3LWcQCU/q2-fragment-insertion.z8d9c3_r/root/P_2/pplacer.extended.0.zm0k4wyo.jplace
[12:09:22] jobs.py (line 106): DEBUG: Invocation of /home/noor/anaconda3/envs/qiime2-2022.2/bin/pplacer --out-dir /tmp/tmp.fGBexxwP3f/sepp-tempssd-Z9x3LWcQCU/q2-fragment-insertion.z8d9c3_r/root/P_2 -j 1 -r /tmp/tmp.fGBexxwP3f/sepp-tempssd-Z9x3LWcQCU/q2-fragment-insertion.z8d9c3_r/root/P_2/pplacer.backbone.f_ju2ji4.fasta -s /tmp/qiime2-archive-l4gp450y/a14c6180-506b-4ecb-bacb-9cb30bc3044b/data/raxml-info.txt -t /tmp/tmp.fGBexxwP3f/sepp-tempssd-Z9x3LWcQCU/q2-fragment-insertion.z8d9c3_r/root/P_2/pplacer.tree.2pvy2nmf.tre --groups 10 /tmp/tmp.fGBexxwP3f/sepp-tempssd-Z9x3LWcQCU/q2-fragment-insertion.z8d9c3_r/root/P_2/pplacer.extended.0.zm0k4wyo.fasta
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/home/noor/anaconda3/envs/qiime2-2022.2/lib/python3.8/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/home/noor/anaconda3/envs/qiime2-2022.2/lib/python3.8/site-packages/sepp/scheduler.py", line 74, in call
return self.run()
File "/home/noor/anaconda3/envs/qiime2-2022.2/lib/python3.8/site-packages/sepp/jobs.py", line 150, in run
raise JobError("\n".join([
sepp.scheduler.JobError: The following execution failed:
/home/noor/anaconda3/envs/qiime2-2022.2/bin/pplacer --out-dir /tmp/tmp.fGBexxwP3f/sepp-tempssd-Z9x3LWcQCU/q2-fragment-insertion.z8d9c3_r/root/P_1 -j 1 -r /tmp/tmp.fGBexxwP3f/sepp-tempssd-Z9x3LWcQCU/q2-fragment-insertion.z8d9c3_r/root/P_1/pplacer.backbone.un0pvqjb.fasta -s /tmp/qiime2-archive-l4gp450y/a14c6180-506b-4ecb-bacb-9cb30bc3044b/data/raxml-info.txt -t /tmp/tmp.fGBexxwP3f/sepp-tempssd-Z9x3LWcQCU/q2-fragment-insertion.z8d9c3_r/root/P_1/pplacer.tree.xqlloe53.tre --groups 10 /tmp/tmp.fGBexxwP3f/sepp-tempssd-Z9x3LWcQCU/q2-fragment-insertion.z8d9c3_r/root/P_1/pplacer.extended.0.rdrmprtj.fasta
No error messages available
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/noor/anaconda3/envs/qiime2-2022.2/bin/run_sepp.py", line 26, in
ExhaustiveAlgorithm().run()
File "/home/noor/anaconda3/envs/qiime2-2022.2/lib/python3.8/site-packages/sepp/algorithm.py", line 174, in run
if (not JobPool().wait_for_all_jobs()):
File "/home/noor/anaconda3/envs/qiime2-2022.2/lib/python3.8/site-packages/sepp/scheduler.py", line 365, in wait_for_all_jobs
result.get()
File "/home/noor/anaconda3/envs/qiime2-2022.2/lib/python3.8/multiprocessing/pool.py", line 771, in get
raise self._value
sepp.scheduler.JobError: The following execution failed:
/home/noor/anaconda3/envs/qiime2-2022.2/bin/pplacer --out-dir /tmp/tmp.fGBexxwP3f/sepp-tempssd-Z9x3LWcQCU/q2-fragment-insertion.z8d9c3_r/root/P_1 -j 1 -r /tmp/tmp.fGBexxwP3f/sepp-tempssd-Z9x3LWcQCU/q2-fragment-insertion.z8d9c3_r/root/P_1/pplacer.backbone.un0pvqjb.fasta -s /tmp/qiime2-archive-l4gp450y/a14c6180-506b-4ecb-bacb-9cb30bc3044b/data/raxml-info.txt -t /tmp/tmp.fGBexxwP3f/sepp-tempssd-Z9x3LWcQCU/q2-fragment-insertion.z8d9c3_r/root/P_1/pplacer.tree.xqlloe53.tre --groups 10 /tmp/tmp.fGBexxwP3f/sepp-tempssd-Z9x3LWcQCU/q2-fragment-insertion.z8d9c3_r/root/P_1/pplacer.extended.0.rdrmprtj.fasta
No error messages available

Traceback (most recent call last):
File "/home/noor/anaconda3/envs/qiime2-2022.2/lib/python3.8/site-packages/q2cli/commands.py", line 339, in call
results = action(**arguments)
File "", line 2, in sepp
File "/home/noor/anaconda3/envs/qiime2-2022.2/lib/python3.8/site-packages/qiime2/sdk/action.py", line 245, in bound_callable
outputs = self.callable_executor(scope, callable_args,
File "/home/noor/anaconda3/envs/qiime2-2022.2/lib/python3.8/site-packages/qiime2/sdk/action.py", line 391, in callable_executor
output_views = self._callable(**view_args)
File "/home/noor/anaconda3/envs/qiime2-2022.2/lib/python3.8/site-packages/q2_fragment_insertion/_insertion.py", line 71, in sepp
_run(str(representative_sequences.file.view(DNAFASTAFormat)),
File "/home/noor/anaconda3/envs/qiime2-2022.2/lib/python3.8/site-packages/q2_fragment_insertion/_insertion.py", line 53, in _run
subprocess.run(cmd, check=True, cwd=cwd)
File "/home/noor/anaconda3/envs/qiime2-2022.2/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['run-sepp.sh', '/tmp/qiime2-archive-avopoo76/a733267f-f35c-47be-8ede-9005a96aca8b/data/dna-sequences.fasta', 'q2-fragment-insertion', '-x', '1', '-A', '1000', '-P', '5000', '-a', '/tmp/qiime2-archive-l4gp450y/a14c6180-506b-4ecb-bacb-9cb30bc3044b/data/aligned-dna-sequences.fasta', '-t', '/tmp/qiime2-archive-l4gp450y/a14c6180-506b-4ecb-bacb-9cb30bc3044b/data/tree.nwk', '-r', '/tmp/qiime2-archive-l4gp450y/a14c6180-506b-4ecb-bacb-9cb30bc3044b/data/raxml-info.txt', '-b', '1']' returned non-zero exit status 1.

Plugin error from fragment-insertion:

Command '['run-sepp.sh', '/tmp/qiime2-archive-avopoo76/a733267f-f35c-47be-8ede-9005a96aca8b/data/dna-sequences.fasta', 'q2-fragment-insertion', '-x', '1', '-A', '1000', '-P', '5000', '-a', '/tmp/qiime2-archive-l4gp450y/a14c6180-506b-4ecb-bacb-9cb30bc3044b/data/aligned-dna-sequences.fasta', '-t', '/tmp/qiime2-archive-l4gp450y/a14c6180-506b-4ecb-bacb-9cb30bc3044b/data/tree.nwk', '-r', '/tmp/qiime2-archive-l4gp450y/a14c6180-506b-4ecb-bacb-9cb30bc3044b/data/raxml-info.txt', '-b', '1']' returned non-zero exit status 1.

detail error file.txt (1.3 MB)

Oddant1 · August 23, 2022, 4:38pm

Hmmm. Unfortunately that traceback doesn't make what's happening immediately obvious. Can you tell me what kind of environment you're running QIIME 2 on and what kind of resources (CPU and RAM) you're using? It's possible the issue has something to do with running out of resources although I don't think that's likely with a single thread.

Noor_Us-Subah · August 24, 2022, 8:10am

I've installed Qiime2 using conda environment (Windows Subsystem for Linux-64 bits) and CPU and RAM-related information is attached in the below image:

Oddant1 · August 24, 2022, 6:42pm

It looks like you only have ~8gigs of RAM which is most likely not enough to run the analysis you're trying to run. Unfortunately, you're probably going to need access to a system with more RAM if you want to be able to run this job. You can go some ways towards confirming this by running htop in a separate terminal from your QIIME 2 command and watching the RAM usage in htop as your command runs. I suspect you will run out.

Noor_Us-Subah · August 25, 2022, 6:56am

Following are the attached snapshots of Htop and command while running side by side

Oddant1 · August 25, 2022, 5:19pm

Yeah unfortunately I do think that's what's happening. If you look at the top screenshot almost all of your RAM is in use.

Noor_Us-Subah · August 31, 2022, 5:10am

I have also run this command on a 16gig RAM machine but it gives the same error.

Oddant1 · August 31, 2022, 6:56pm

How large is your data? It is possible you need a whole lot of RAM. Can you run htop while running the command on the 16gig machine and see if you run out?