fragment-insertion sepp debug help

dylan · January 22, 2021, 2:13pm

Running this command

qiime fragment-insertion sepp \
--i-representative-sequences output/representative_sequences.qza \
--i-reference-database input/sepp-refs-silva-128.qza \
--o-tree output/rooted_tree \
--o-placements output/placements \
--p-threads 12 \
--verbose \
--p-debug

with the edit to run-sepp.sh mentioned here (is this the same as the debug flag?) yields the following message:

  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/q2cli/commands.py", line 329, in __call__
    results = action(**arguments)
  File "<decorator-gen-303>", line 2, in sepp
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/qiime2/sdk/action.py", line 245, in bound_callable
    output_types, provenance)
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/qiime2/sdk/action.py", line 390, in _callable_executor_
    output_views = self._callable(**view_args)
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/q2_fragment_insertion/_insertion.py", line 77, in sepp
    debug)
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/q2_fragment_insertion/_insertion.py", line 53, in _run
    subprocess.run(cmd, check=True, cwd=cwd)
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['run-sepp.sh', '/tmp/qiime2-archive-05j3igqe/19e4fc78-44a3-443b-9ee3-41ae8102598d/data/dna-sequences.fasta', 'q2-fragment-insertion', '-x', '12', '-A', '1000', '-P', '5000', '-a', '/tmp/qiime2-archive-jsk6s5yy/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/aligned-dna-sequences.fasta', '-t', '/tmp/qiime2-archive-jsk6s5yy/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/tree.nwk', '-r', '/tmp/qiime2-archive-jsk6s5yy/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/raxml-info.txt', '-b', '1']' returned non-zero exit status 1.

Plugin error from fragment-insertion:

  Command '['run-sepp.sh', '/tmp/qiime2-archive-05j3igqe/19e4fc78-44a3-443b-9ee3-41ae8102598d/data/dna-sequences.fasta', 'q2-fragment-insertion', '-x', '12', '-A', '1000', '-P', '5000', '-a', '/tmp/qiime2-archive-jsk6s5yy/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/aligned-dna-sequences.fasta', '-t', '/tmp/qiime2-archive-jsk6s5yy/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/tree.nwk', '-r', '/tmp/qiime2-archive-jsk6s5yy/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/raxml-info.txt', '-b', '1']' returned non-zero exit status 1.

See above for debug info.

I'm running WSL with a 12 core processor and 32 gigs of RAM plus 96 gigs of swap.

Any help would be appreciated.

thermokarst · January 22, 2021, 2:14pm

Hi @dylan!

No, I don't think so - I hate to ask this, but we'll need you to rerun the command above, in a fresh QIIME 2 environment that hasn't had the source files edited, in order to help diagnose. Thanks!

Stefan · January 22, 2021, 3:31pm

you could try to operate on a smaller version of output/representative_sequences.qza. Just take e.g. the first 10 sequences from this file and check if this runs through.

It might also help to reduce --p-threads 12 to just 3. Otherwise each core only gets 32/12 = 2.6 GB per task, which might be too low.

dylan · January 22, 2021, 3:33pm

Ah, I see why now. Here’s the last few lines of debug info from the base version:

[07:23:03] alignment.py (line 205):    DEBUG: Writing alignment of type <class 'sepp.alignment.ReadonlySubalignment'> with length 3616 to file /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_113/pplacer.backbone.tivih9mx.fasta
Traceback (most recent call last):
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/sepp/jobs.py", line 154, in run
    if self.read_stderr() else 'No error messages available']))
sepp.scheduler.JobError: The following execution failed:
/home/dylan/miniconda3/envs/qiime2-2020.11/bin/pplacer --out-dir /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_12 -j 1 -r /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_12/pplacer.backbone.e98bn9jr.fasta -s /tmp/qiime2-archive-u6vrpdmr/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/raxml-info.txt -t /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_12/pplacer.tree.cru8sdu3.tre --groups 10 /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_12/pplacer.extended.0.ha_v8cyc.fasta
No error messages available
Traceback (most recent call last):
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/sepp/jobs.py", line 154, in run
    if self.read_stderr() else 'No error messages available']))
sepp.scheduler.JobError: The following execution failed:
/home/dylan/miniconda3/envs/qiime2-2020.11/bin/pplacer --out-dir /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_12 -j 1 -r /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_12/pplacer.backbone.2cyxu6c9.fasta -s /tmp/qiime2-archive-u6vrpdmr/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/raxml-info.txt -t /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_12/pplacer.tree.ythm3oai.tre --groups 10 /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_12/pplacer.extended.1.ka5e67hk.fasta
No error messages available
Traceback (most recent call last):
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/sepp/jobs.py", line 154, in run
    if self.read_stderr() else 'No error messages available']))
sepp.scheduler.JobError: The following execution failed:
/home/dylan/miniconda3/envs/qiime2-2020.11/bin/pplacer --out-dir /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_11 -j 1 -r /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_11/pplacer.backbone.r72tz0hg.fasta -s /tmp/qiime2-archive-u6vrpdmr/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/raxml-info.txt -t /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_11/pplacer.tree.u87t_ymn.tre --groups 10 /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_11/pplacer.extended.2.06vearn4.fasta
No error messages available
Traceback (most recent call last):
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/sepp/jobs.py", line 154, in run
    if self.read_stderr() else 'No error messages available']))
sepp.scheduler.JobError: The following execution failed:
/home/dylan/miniconda3/envs/qiime2-2020.11/bin/pplacer --out-dir /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_14 -j 1 -r /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_14/pplacer.backbone.hi_337mh.fasta -s /tmp/qiime2-archive-u6vrpdmr/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/raxml-info.txt -t /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_14/pplacer.tree.5yli2nhx.tre --groups 10 /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_14/pplacer.extended.1.402kay9h.fasta
No error messages available
Traceback (most recent call last):
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/sepp/jobs.py", line 154, in run
    if self.read_stderr() else 'No error messages available']))
sepp.scheduler.JobError: The following execution failed:
/home/dylan/miniconda3/envs/qiime2-2020.11/bin/pplacer --out-dir /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_14 -j 1 -r /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_14/pplacer.backbone.3yirsjh3.fasta -s /tmp/qiime2-archive-u6vrpdmr/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/raxml-info.txt -t /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_14/pplacer.tree.ivfr5iul.tre --groups 10 /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_14/pplacer.extended.2.bo5jnz3t.fasta
No error messages available
Traceback (most recent call last):
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/sepp/jobs.py", line 154, in run
    if self.read_stderr() else 'No error messages available']))
sepp.scheduler.JobError: The following execution failed:
/home/dylan/miniconda3/envs/qiime2-2020.11/bin/pplacer --out-dir /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_14 -j 1 -r /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_14/pplacer.backbone.fzbnpg10.fasta -s /tmp/qiime2-archive-u6vrpdmr/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/raxml-info.txt -t /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_14/pplacer.tree.zn1yjivk.tre --groups 10 /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_14/pplacer.extended.0.oh3gbkt2.fasta
No error messages available
Traceback (most recent call last):
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/sepp/jobs.py", line 154, in run
    if self.read_stderr() else 'No error messages available']))
sepp.scheduler.JobError: The following execution failed:
/home/dylan/miniconda3/envs/qiime2-2020.11/bin/pplacer --out-dir /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_8 -j 1 -r /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_8/pplacer.backbone.cf_f4bef.fasta -s /tmp/qiime2-archive-u6vrpdmr/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/raxml-info.txt -t /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_8/pplacer.tree.k1dkt6sw.tre --groups 10 /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_8/pplacer.extended.0.vrofjcb2.fasta
No error messages available
Traceback (most recent call last):
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/sepp/jobs.py", line 154, in run
    if self.read_stderr() else 'No error messages available']))
sepp.scheduler.JobError: The following execution failed:
/home/dylan/miniconda3/envs/qiime2-2020.11/bin/pplacer --out-dir /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_8 -j 1 -r /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_8/pplacer.backbone.bz1bs0gl.fasta -s /tmp/qiime2-archive-u6vrpdmr/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/raxml-info.txt -t /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_8/pplacer.tree.nyz7yrtt.tre --groups 10 /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_8/pplacer.extended.1.67g9xcb6.fasta
No error messages available
Traceback (most recent call last):
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/sepp/jobs.py", line 154, in run
    if self.read_stderr() else 'No error messages available']))
sepp.scheduler.JobError: The following execution failed:
/home/dylan/miniconda3/envs/qiime2-2020.11/bin/pplacer --out-dir /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_6 -j 1 -r /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_6/pplacer.backbone.lpn28ui4.fasta -s /tmp/qiime2-archive-u6vrpdmr/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/raxml-info.txt -t /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_6/pplacer.tree.r3ype17n.tre --groups 10 /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_6/pplacer.extended.2.gszmxfd_.fasta
No error messages available
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/sepp/scheduler.py", line 74, in __call__
    return self.run()
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/sepp/jobs.py", line 154, in run
    if self.read_stderr() else 'No error messages available']))
sepp.scheduler.JobError: The following execution failed:
/home/dylan/miniconda3/envs/qiime2-2020.11/bin/pplacer --out-dir /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_1 -j 1 -r /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_1/pplacer.backbone.fp5d6hyw.fasta -s /tmp/qiime2-archive-u6vrpdmr/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/raxml-info.txt -t /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_1/pplacer.tree.8v7w1eoy.tre --groups 10 /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_1/pplacer.extended.0.z5n04x_e.fasta
No error messages available
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/bin/run_sepp.py", line 26, in <module>
    ExhaustiveAlgorithm().run()
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/sepp/algorithm.py", line 174, in run
    if (not JobPool().wait_for_all_jobs()):
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/sepp/scheduler.py", line 365, in wait_for_all_jobs
    result.get()
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/multiprocessing/pool.py", line 644, in get
    raise self._value
sepp.scheduler.JobError: The following execution failed:
/home/dylan/miniconda3/envs/qiime2-2020.11/bin/pplacer --out-dir /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_1 -j 1 -r /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_1/pplacer.backbone.fp5d6hyw.fasta -s /tmp/qiime2-archive-u6vrpdmr/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/raxml-info.txt -t /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_1/pplacer.tree.8v7w1eoy.tre --groups 10 /tmp/tmp.wHblljACq8/sepp-tempssd-xImh1cyro4/q2-fragment-insertion.5ortsuh_/root/P_1/pplacer.extended.0.z5n04x_e.fasta
No error messages available
=====================================================================
Traceback (most recent call last):
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/q2cli/commands.py", line 329, in __call__
    results = action(**arguments)
  File "<decorator-gen-303>", line 2, in sepp
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/qiime2/sdk/action.py", line 245, in bound_callable
    output_types, provenance)
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/qiime2/sdk/action.py", line 390, in _callable_executor_
    output_views = self._callable(**view_args)
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/q2_fragment_insertion/_insertion.py", line 77, in sepp
    debug)
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/q2_fragment_insertion/_insertion.py", line 53, in _run
    subprocess.run(cmd, check=True, cwd=cwd)
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['run-sepp.sh', '/tmp/qiime2-archive-xwjdyq_0/19e4fc78-44a3-443b-9ee3-41ae8102598d/data/dna-sequences.fasta', 'q2-fragment-insertion', '-x', '24', '-A', '1000', '-P', '5000', '-a', '/tmp/qiime2-archive-u6vrpdmr/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/aligned-dna-sequences.fasta', '-t', '/tmp/qiime2-archive-u6vrpdmr/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/tree.nwk', '-r', '/tmp/qiime2-archive-u6vrpdmr/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/raxml-info.txt', '-b', '1']' returned non-zero exit status 1.

Plugin error from fragment-insertion:

  Command '['run-sepp.sh', '/tmp/qiime2-archive-xwjdyq_0/19e4fc78-44a3-443b-9ee3-41ae8102598d/data/dna-sequences.fasta', 'q2-fragment-insertion', '-x', '24', '-A', '1000', '-P', '5000', '-a', '/tmp/qiime2-archive-u6vrpdmr/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/aligned-dna-sequences.fasta', '-t', '/tmp/qiime2-archive-u6vrpdmr/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/tree.nwk', '-r', '/tmp/qiime2-archive-u6vrpdmr/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/raxml-info.txt', '-b', '1']' returned non-zero exit status 1.

See above for debug info.

dylan · January 22, 2021, 4:00pm

Thanks @stefan, I incorporated both suggestions and still got the ‘non-zero exit status 1’ error.

[07:51:07] alignment.py (line 205):    DEBUG: Writing alignment of type <class 'sepp.alignment.ReadonlySubalignment'> with length 2801 to file /tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_108/pplacer.backbone.hrd32des.fasta
Traceback (most recent call last):
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/sepp/jobs.py", line 154, in run
    if self.read_stderr() else 'No error messages available']))
sepp.scheduler.JobError: The following execution failed:
/home/dylan/miniconda3/envs/qiime2-2020.11/bin/pplacer --out-dir /tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_64 -j 1 -r /tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_64/pplacer.backbone.wei12gfw.fasta -s /tmp/qiime2-archive-n4nno1to/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/raxml-info.txt -t /tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_64/pplacer.tree.dcpmxhdz.tre --groups 10 /tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_64/pplacer.extended.0.uv0iinur.fasta
No error messages available
[07:51:07] jobs.py (line 81):    DEBUG: Starting pplacer Job with input: backbone_alignment_file:/tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_108/pplacer.backbone.hrd32des.fasta, tree_file:/tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_108/pplacer.tree.01lt6ohu.tre, info_file:/tmp/qiime2-archive-n4nno1to/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/raxml-info.txt, extended alignment:/tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_108/pplacer.extended.0.7k4lpgf1.fasta, output:/tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_108/pplacer.extended.0.7k4lpgf1.jplace
[07:51:07] jobs.py (line 108):    DEBUG: Invocation of /home/dylan/miniconda3/envs/qiime2-2020.11/bin/pplacer --out-dir /tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_108 -j 1 -r /tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_108/pplacer.backbone.hrd32des.fasta -s /tmp/qiime2-archive-n4nno1to/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/raxml-info.txt -t /tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_108/pplacer.tree.01lt6ohu.tre --groups 10 /tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_108/pplacer.extended.0.7k4lpgf1.fasta
[07:51:07] jobs.py (line 149):    DEBUG: Finished pplacer Job with input: backbone_alignment_file:/tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_108/pplacer.backbone.hrd32des.fasta, tree_file:/tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_108/pplacer.tree.01lt6ohu.tre, info_file:/tmp/qiime2-archive-n4nno1to/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/raxml-info.txt, extended alignment:/tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_108/pplacer.extended.0.7k4lpgf1.fasta, output:/tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_108/pplacer.extended.0.7k4lpgf1.jplace with:
 return code: -11
 output:
 error:
Traceback (most recent call last):
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/sepp/jobs.py", line 154, in run
    if self.read_stderr() else 'No error messages available']))
sepp.scheduler.JobError: The following execution failed:
/home/dylan/miniconda3/envs/qiime2-2020.11/bin/pplacer --out-dir /tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_82 -j 1 -r /tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_82/pplacer.backbone.00galm9o.fasta -s /tmp/qiime2-archive-n4nno1to/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/raxml-info.txt -t /tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_82/pplacer.tree.s6bqnws0.tre --groups 10 /tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_82/pplacer.extended.0.ga3djgo5.fasta
No error messages available
Traceback (most recent call last):
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/sepp/jobs.py", line 154, in run
    if self.read_stderr() else 'No error messages available']))
sepp.scheduler.JobError: The following execution failed:
/home/dylan/miniconda3/envs/qiime2-2020.11/bin/pplacer --out-dir /tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_87 -j 1 -r /tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_87/pplacer.backbone.uzfxh8wo.fasta -s /tmp/qiime2-archive-n4nno1to/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/raxml-info.txt -t /tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_87/pplacer.tree.wmpiz_4f.tre --groups 10 /tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_87/pplacer.extended.0.vlh105yz.fasta
No error messages available
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/sepp/scheduler.py", line 74, in __call__
    return self.run()
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/sepp/jobs.py", line 154, in run
    if self.read_stderr() else 'No error messages available']))
sepp.scheduler.JobError: The following execution failed:
/home/dylan/miniconda3/envs/qiime2-2020.11/bin/pplacer --out-dir /tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_15 -j 1 -r /tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_15/pplacer.backbone.wqff6xyc.fasta -s /tmp/qiime2-archive-n4nno1to/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/raxml-info.txt -t /tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_15/pplacer.tree.plwylyww.tre --groups 10 /tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_15/pplacer.extended.0.plzerdgy.fasta
No error messages available
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/bin/run_sepp.py", line 26, in <module>
    ExhaustiveAlgorithm().run()
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/sepp/algorithm.py", line 174, in run
    if (not JobPool().wait_for_all_jobs()):
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/sepp/scheduler.py", line 365, in wait_for_all_jobs
    result.get()
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/multiprocessing/pool.py", line 644, in get
    raise self._value
sepp.scheduler.JobError: The following execution failed:
/home/dylan/miniconda3/envs/qiime2-2020.11/bin/pplacer --out-dir /tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_15 -j 1 -r /tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_15/pplacer.backbone.wqff6xyc.fasta -s /tmp/qiime2-archive-n4nno1to/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/raxml-info.txt -t /tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_15/pplacer.tree.plwylyww.tre --groups 10 /tmp/tmp.YE7eIK5BTh/sepp-tempssd-3NqyBhHOxi/q2-fragment-insertion.nufc1fbf/root/P_15/pplacer.extended.0.plzerdgy.fasta
No error messages available
=====================================================================
Traceback (most recent call last):
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/q2cli/commands.py", line 329, in __call__
    results = action(**arguments)
  File "<decorator-gen-303>", line 2, in sepp
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/qiime2/sdk/action.py", line 245, in bound_callable
    output_types, provenance)
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/qiime2/sdk/action.py", line 390, in _callable_executor_
    output_views = self._callable(**view_args)
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/q2_fragment_insertion/_insertion.py", line 77, in sepp
    debug)
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/site-packages/q2_fragment_insertion/_insertion.py", line 53, in _run
    subprocess.run(cmd, check=True, cwd=cwd)
  File "/home/dylan/miniconda3/envs/qiime2-2020.11/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['run-sepp.sh', '/tmp/qiime2-archive-avho24u1/75746a5b-c9d7-49b8-8e44-683cb7c80ca2/data/dna-sequences.fasta', 'q2-fragment-insertion', '-x', '3', '-A', '1000', '-P', '5000', '-a', '/tmp/qiime2-archive-n4nno1to/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/aligned-dna-sequences.fasta', '-t', '/tmp/qiime2-archive-n4nno1to/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/tree.nwk', '-r', '/tmp/qiime2-archive-n4nno1to/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/raxml-info.txt', '-b', '1']' returned non-zero exit status 1.

Plugin error from fragment-insertion:

  Command '['run-sepp.sh', '/tmp/qiime2-archive-avho24u1/75746a5b-c9d7-49b8-8e44-683cb7c80ca2/data/dna-sequences.fasta', 'q2-fragment-insertion', '-x', '3', '-A', '1000', '-P', '5000', '-a', '/tmp/qiime2-archive-n4nno1to/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/aligned-dna-sequences.fasta', '-t', '/tmp/qiime2-archive-n4nno1to/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/tree.nwk', '-r', '/tmp/qiime2-archive-n4nno1to/e44b5e78-31e5-4a0f-9041-494bc3ca2df2/data/raxml-info.txt', '-b', '1']' returned non-zero exit status 1.

See above for debug info.

Stefan · January 22, 2021, 4:02pm

would you be willing to send me your reduced representative_sequences.qza file, such that I can take a look myself?

dylan · January 24, 2021, 2:12am

For posterity:

@Stefan was able to successfully execute the command on his end. I continued to encounter an error even in a fresh conda environment. I also experienced an error when trying on my laptop, again with a clean conda install.

However, I installed Ubuntu and was able to successfully execute the command on test sequences in that OS, so this issue may be related to WSL.

Thanks to both @Stefan and @thermokarst for their help.

thermokarst · January 24, 2021, 8:17pm

Yes, that is what I expected, I just needed to be sure that there wasn't an additional error caused by your manual edits to the environment, make sense? As I mentioned above, this is important for diagnosis, it was not intended as a fix for your problem (this is an important distinction to make for any future readers of this thread).

Sounds to me like maybe you didn't have enough memory allocated in WSL, I'll leave it to @Stefan to follow up with his thoughts.

system · February 25, 2021, 2:17am

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.