Non-zero exit status when performing fragment-insertion sepp

Technical Support
,
1

Hi there,
I’m facing exactly the same problem as the one introduced in this post. Interestingly, I ran the same code chunk both on my laptop and on the server, and the result is that in the laptop run no error occurred, while in the server one the comand was killed. I also tried the solution proposed by @nickscales in this post, by removing both the row containing ‘Partition:0’ and the one with ‘Name: No Name Provided’ (also separately) and re-building the sepp-refs-gg-13-8.qza file I was using, with the same disappointing result: the process stopped after 38 mins of calculation.

Here are the code I ran:

qiime fragment-insertion sepp \
--i-representative-sequences outputs/rep-seqs.qza \
--i-reference-database sepp-refs-gg-13-8.qza \
--p-threads 4 \
--o-tree outputs/insertion-tree.qza \
--o-placements outputs/insertion-placements.qza \
--verbose

and the error I obtained:

Traceback (most recent call last):
  File "/opt/conda/envs/qiime2-2020.2/lib/python3.6/site-packages/q2cli/commands.py", line 328, in __call__
    results = action(**arguments)
  File "</opt/conda/envs/qiime2-2020.2/lib/python3.6/site-packages/decorator.py:decorator-gen-295>", line 2, in sepp
  File "/opt/conda/envs/qiime2-2020.2/lib/python3.6/site-packages/qiime2/sdk/action.py", line 245, in bound_callable
    output_types, provenance)
  File "/opt/conda/envs/qiime2-2020.2/lib/python3.6/site-packages/qiime2/sdk/action.py", line 390, in _callable_executor_
    output_views = self._callable(**view_args)
  File "/opt/conda/envs/qiime2-2020.2/lib/python3.6/site-packages/q2_fragment_insertion/_insertion.py", line 77, in sepp
    debug)
  File "/opt/conda/envs/qiime2-2020.2/lib/python3.6/site-packages/q2_fragment_insertion/_insertion.py", line 53, in _run
    subprocess.run(cmd, check=True, cwd=cwd)
  File "/opt/conda/envs/qiime2-2020.2/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['run-sepp.sh', '/tmp/qiime2-archive-epd62htd/22f8ad0c-5879-4b69-8135-aed65b19719e/data/dna-sequences.fasta', 'q2-fragment-insertion', '-x', '1', '-A', '1000', '-P', '5000', '-a', '/tmp/qiime2-archive-9c66grac/a14c6180-506b-4ecb-bacb-9cb30bc3044b/data/aligned-dna-sequences.fasta', '-t', '/tmp/qiime2-archive-9c66grac/a14c6180-506b-4ecb-bacb-9cb30bc3044b/data/tree.nwk', '-r', '/tmp/qiime2-archive-9c66grac/a14c6180-506b-4ecb-bacb-9cb30bc3044b/data/raxml-info.txt']' returned non-zero exit status 1.

Plugin error from fragment-insertion:

  Command '['run-sepp.sh', '/tmp/qiime2-archive-epd62htd/22f8ad0c-5879-4b69-8135-aed65b19719e/data/dna-sequences.fasta', 'q2-fragment-insertion', '-x', '1', '-A', '1000', '-P', '5000', '-a', '/tmp/qiime2-archive-9c66grac/a14c6180-506b-4ecb-bacb-9cb30bc3044b/data/aligned-dna-sequences.fasta', '-t', '/tmp/qiime2-archive-9c66grac/a14c6180-506b-4ecb-bacb-9cb30bc3044b/data/tree.nwk', '-r', '/tmp/qiime2-archive-9c66grac/a14c6180-506b-4ecb-bacb-9cb30bc3044b/data/raxml-info.txt']' returned non-zero exit status 1.

See above for debug info.

Many thanks for your time and help!

3

Hi @iptz1!

Can you rerun with the following:

qiime fragment-insertion sepp \
  --i-representative-sequences outputs/rep-seqs.qza \
  --i-reference-database sepp-refs-gg-13-8.qza \
  --p-threads 4 \
  --o-tree outputs/insertion-tree.qza \
  --o-placements outputs/insertion-placements.qza \
  --p-debug \
  --verbose

This include the --p-debug flag, which is a special flag for sepp - it will run a bit more slowly, but will produce a much more detailed output log (the log right now basically tells us nothing, its like "hey, i broke."). Please rerun and share the log here - thanks!

:qiime2:

1 Like
5

Hi @thermokarst,
I attach the log file here.
Thank you for your kind help!
err_sepp.txt (2.4 MB)

7

Thanks @iptz1! I see a ton of errors in this log, but I need some help making sense of it - @Stefan, can you lend a hand? Thanks!

10

Hi @iptz1,
to me, it looks like each and every invocation of the pplacer program fails. Unfortunately, I don’t know what exit code -11 stands for. Typically, pplacer fails due to exceeding main memory. May I ask about the characteristics of your input? How many fragments and how long are they on average. Do I understand you correctly that the same input went through on your Laptop, but not on a server? What are the operating systems. Can you find out how much main memory is available for the job on the server? What if you limit the server job to just one core, i.e. --p-threads 1.
Thanks for your report!
Stefan

1 Like
12

what just came to my mind is this issue report about pplacer. Looks like a kernel option might cause errors. What happens on the server if you manually invoke /opt/conda/envs/qiime2-2020.2/bin/pplacer ?

1 Like
13

Hi @Stefan,
thank you for the help.

You’re right, my OS/Kernel are the “problematic” ones highlighted in the post (Operating System: Debian GNU/Linux 10 (buster), Kernel: Linux 4.19.0-6-amd64). And yes, the same input went through on my laptop, but not on the server, despite having the same memory resources (16GB RAM, 4 swap) on both the systems. I also tried to limit the job to a single core, but this did not solve the problem, unfortunately.
But I think you definitely found the needle in the haystack with the pplacer direct invocation hint. Indeed, here is what I obtain:

Segmentation fault (core dumped)

How do you think I can get out of it? I now have to specify that the server is not a physical one, but a lxc virtualization and that I’m working within a Docker container (build-mounting the /proc folder) present within this server.

Thank you again for your time and patience.

1 Like
17

Hi there,
the system manager finally found the problem: vsyscall was disabled and this caused sepp process to die instead of performing a “normal” system call.

Hope this will help others if detecting the same issue.

Thank you for the kind help!

3 Likes
19

A post was split to a new topic: q2-fragment-insertion error