# Problem with Fragment Insertion - SEPP

**URL:** <https://forum.qiime2.org/t/problem-with-fragment-insertion-sepp/11396>\
**Category:** Technical Support\
**Tags:** sepp\
**Created:** [September 5, 2019, 12:31pm UTC](https://forum.qiime2.org/t/problem-with-fragment-insertion-sepp/11396 "2019-09-05T12:31:35Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![Felipe\_Rocha](https://forum.qiime2.org/user_avatar/forum.qiime2.org/felipe_rocha/32/5691_2.png) [@Felipe\_Rocha](https://forum.qiime2.org/u/Felipe_Rocha)\
**Post date:** [September 5, 2019, 12:31pm UTC](https://forum.qiime2.org/t/problem-with-fragment-insertion-sepp/11396/1 "2019-09-05T12:31:35Z")

</div>

Hello, everyone,

I'm trying to run qiime fragment-insertion sepp with a SILVA rooted tree and aligned sequences (99%), but I keep getting an error. First I tried with my own computer, but I noticed there wasn't enough RAM in it. So I went after a remote server with enough memory, but I still get an error.

**My command line is as follows:**

[felipe.rocha@bioinfo Analises] docker run -t -i -v (pwd):/data qiime2/core:2019.7  
qiime fragment-insertion sepp  
--i-representative-sequences Felipe-Deblur-RepSeqs.qza  
--i-reference-alignment silva99\_alignedseqs.qza  
--i-reference-phylogeny silva99\_rootedtree.qza  
--p-threads 2  
--o-tree FelipeSilva-FragmInsert-RootedTree.qza  
--o-placements FelipeSilva-FragmInsert-Placements.qza  
--verbose

**And the error I get:**

Removing /tmp/tmp.4taskGv3iQ/sepp-tmp-i9N6X  
Traceback (most recent call last):  
File "/opt/conda/envs/qiime2-2019.7/lib/python3.6/site-packages/q2cli/commands.py", line 327, in **call**  
results = action(\*\*arguments)  
File "\</opt/conda/envs/qiime2-2019.7/lib/python3.6/site-packages/decorator.py:decorator-gen-299\>", line 2, in sepp  
File "/opt/conda/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/sdk/action.py", line 240, in bound\_callable  
output\_types, provenance)  
File "/opt/conda/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/sdk/action.py", line 383, in _callable\_executor_  
output\_views = self.\_callable(\*\*view\_args)  
File "/opt/conda/envs/qiime2-2019.7/lib/python3.6/site-packages/q2\_fragment\_insertion/\_insertion.py", line 179, in sepp  
reference\_alignment, reference\_phylogeny, debug)  
File "/opt/conda/envs/qiime2-2019.7/lib/python3.6/site-packages/q2\_fragment\_insertion/\_insertion.py", line 137, in \_run  
subprocess.run(cmd, check=True, cwd=cwd)  
File "/opt/conda/envs/qiime2-2019.7/lib/python3.6/subprocess.py", line 418, in run  
output=stdout, stderr=stderr)  
subprocess.CalledProcessError: Command '['run-sepp.sh', '/tmp/qiime2-archive-bfgihcj8/b712abec-2b25-418a-b569-b8f8ad4be511/data/dna-sequences.fasta', 'q2-fragment-insertion', '-x', '2', '-A', '1000', '-P', '5000', '-a', '/tmp/qiime2-archive-fizomboa/b35759dc-0bdc-4c8a-a122-3496ab0a0222/data/aligned-dna-sequences.fasta', '-t', '/tmp/qiime2-archive-5e4vsrma/50638235-87e6-44f0-a9ce-302bf1d7294d/data/tree.nwk']' returned non-zero exit status 1.

Plugin error from fragment-insertion:

Command '['run-sepp.sh', '/tmp/qiime2-archive-bfgihcj8/b712abec-2b25-418a-b569-b8f8ad4be511/data/dna-sequences.fasta', 'q2-fragment-insertion', '-x', '2', '-A', '1000', '-P', '5000', '-a', '/tmp/qiime2-archive-fizomboa/b35759dc-0bdc-4c8a-a122-3496ab0a0222/data/aligned-dna-sequences.fasta', '-t', '/tmp/qiime2-archive-5e4vsrma/50638235-87e6-44f0-a9ce-302bf1d7294d/data/tree.nwk']' returned non-zero exit status 1.

See above for debug info.

Everything seems fine in my command, and I have sufficient memory, so please help me out on this one. Thank you all, Felipe.

---

<div class="post-metadata">

**Author:** ![jwdebelius](https://forum.qiime2.org/user_avatar/forum.qiime2.org/jwdebelius/32/9655_2.png) [@jwdebelius](https://forum.qiime2.org/u/jwdebelius)\
**Post date:** [September 5, 2019, 3:18pm UTC](https://forum.qiime2.org/t/problem-with-fragment-insertion-sepp/11396/2 "2019-09-05T15:18:49Z")

</div>

Hi @Felipe_Rocha,

How did you get the Silva trees? There's a fun issues with sepp in that you can't use the trees that come in the QIIME release, you need to get the SEPP-specific trees. There's a thread about that here:

> [@Q2-fragment-insertion reference info](https://forum.qiime2.org/t/q2-fragment-insertion-reference-info/4360/2):
>
> Hi Ryan, you are doing the right things in principle. However, the branch length of the original reference phylogeny need to be reestimated to be in sync with the alignment distances. This is not necessarily guaranteed by Silva. SEPP's developer explains what to do in detail in the file "sepp-package/buildref/README.md" of this archive: [https://github.com/smirarab/sepp-refs/blob/master/gg/sepp-package.tar.bz?raw=true](https://github.com/smirarab/sepp-refs/blob/master/gg/sepp-package.tar.bz?raw=true) Alternatively, you might want to use a pre-compiled version of Silva 12.8 whi…

If switching trees doesn't solve your problem, you might consider the `--debug` flag next time? It wont fix anything, but it might give more answers?

Best,  
Justine

---

<div class="post-metadata">

**Author:** ![Felipe\_Rocha](https://forum.qiime2.org/user_avatar/forum.qiime2.org/felipe_rocha/32/5691_2.png) [@Felipe\_Rocha](https://forum.qiime2.org/u/Felipe_Rocha)\
**Post date:** [September 5, 2019, 7:58pm UTC](https://forum.qiime2.org/t/problem-with-fragment-insertion-sepp/11396/3 "2019-09-05T19:58:53Z")

</div>

Hi @jwdebelius,

I got my Silva 99% tree from the zip file they have on their website, it is something like silva\_99.tre, which must be the QIIME release you are talking about. I'll try to use the --debug flag again today, let's see what happens.

I cannot find this reference info that Ryan on that other thread is talking about, all I see as input is:  
Usage: qiime fragment-insertion sepp [OPTIONS]

Perform fragment insertion of 16S sequences using the SEPP algorithm  
against the Greengenes 13\_8 99% tree.

Inputs:  
**--i-representative-sequences** ARTIFACT FeatureData[Sequence]  
The sequences to insert [required]  
**--i-reference-alignment** ARTIFACT FeatureData[AlignedSequence]  
The reference multiple nucleotide alignment used to  
construct the reference phylogeny. [optional]  
**--i-reference-phylogeny** ARTIFACT  
Phylogeny[Rooted] The rooted reference phylogeny. Must be in sync with  
reference-alignment, i.e. each tip name must have  
exactly one corresponding record. [optional]

If things go wrong again I'll try to read the README.md, what sort of program should I use to open .md files? I wasn't able to do it.

Thanks for helping,  
Felipe.

---

<div class="post-metadata">

**Author:** ![jwdebelius](https://forum.qiime2.org/user_avatar/forum.qiime2.org/jwdebelius/32/9655_2.png) [@jwdebelius](https://forum.qiime2.org/u/jwdebelius)\
**Post date:** [September 5, 2019, 11:20pm UTC](https://forum.qiime2.org/t/problem-with-fragment-insertion-sepp/11396/4 "2019-09-05T23:20:24Z")

</div>

Hi @Felipe_Rocha,

THere should be .qza files already imported in the share/qiime2-fragment-insertion directory (or something similar); the exact path is given on the github link, and you should be able to use that and the alignment file instead of the tree.

Im currently running those files for hte first time with my own fragment insertion, and the `--debug` flag has been my saving grace. ("I'll do insertion against Silva," she said. "It will make my new boss happy," she said. "How hard can it be?" ...Two weeks of expensive debugging later, and I'll let you know hopefully by sometime Saturday whether or not these trees worked for me.) But, in the meantime, seriously, reading the output from the debug flag on the terminal helped me so much.

As far as the .md files go,

> [@Felipe\_Rocha](#):
>
> If things go wrong again I’ll try to read the README.md, what sort of program should I use to open .md files? I wasn’t able to do it.

A .md file is a markdown file, it's like HMTL light. You _can_ read it using a text editor, but some of the annotation will be different. (Actually, a lot like text formatting here). Or, my favorite mac program for markdown is [MacDown](https://macdown.uranusjr.com/) because it's a relatively nice side-by-side rendering.

Best,  
Justine

---

<div class="post-metadata">

**Author:** ![Stefan](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/s/bbce88/32.png) [@Stefan](https://forum.qiime2.org/u/Stefan)\
**Post date:** [September 6, 2019, 8:18am UTC](https://forum.qiime2.org/t/problem-with-fragment-insertion-sepp/11396/5 "2019-09-06T08:18:03Z")

</div>

Hi @Felipe_Rocha and @jwdebelius,  
I am sorry that using Silva is currently such a pain in the a\*\*. I am working on measures to make it way easier in future qiime2 releases: [Silva 2019 by sjanssen2 · Pull Request #63 · qiime2/q2-fragment-insertion · GitHub](https://github.com/qiime2/q2-fragment-insertion/pull/63)

For now, changing the reference is impossible via the q2 wrapper since does not expose the parameter to set the file path for the _info_ file. You need three files for your reference: 1. tree, 2. alignment and 3. info.

The funny thing is, that those three files are already shipped with qiiime2-2019.7: unzip [https://anaconda.org/qiime2/q2-fragment-insertion/2019.7.0/download/linux-64/q2-fragment-insertion-2019.7.0-py36\_0.tar.bz2](https://anaconda.org/qiime2/q2-fragment-insertion/2019.7.0/download/linux-64/q2-fragment-insertion-2019.7.0-py36_0.tar.bz2) and navigate to /share/fragment-insertion/ref/ you will find three files silva12.8\_99otus\_aligned\_masked1977.tre silva12.8\_99otus\_aligned\_masked1977.fasta silva12.8\_99otus\_aligned\_masked1977.info

You will have to run SEPP manually (i.e. not using the qiime plugin) via command line: `run-sepp.sh yourfragments.fasta q2-fragment-insertion -x 2 -A 1000 -P 5000 -a silva12.8_99otus_aligned_masked1977.fasta -t files silva12.8_99otus_aligned_masked1977.tre -r silva12.8_99otus_aligned_masked1977.info -b 1`

Best,  
Stefan

---

<div class="post-metadata">

**Author:** ![jwdebelius](https://forum.qiime2.org/user_avatar/forum.qiime2.org/jwdebelius/32/9655_2.png) [@jwdebelius](https://forum.qiime2.org/u/jwdebelius)\
**Post date:** [September 6, 2019, 8:22am UTC](https://forum.qiime2.org/t/problem-with-fragment-insertion-sepp/11396/6 "2019-09-06T08:22:00Z")

</div>

Thanks @Stefan, I'll take a look at the manual command.

> [@Stefan](#):
>
> For now, changing the reference is impossible via the q2 wrapper since does not expose the parameter to set the file path for the _info_ file. You need three files for your reference: 1. tree, 2. alignment and 3. info.

Perhaps a stupid question, but if thats the case, why is it exposed in the documentation? It might be good to just update the docs to make that clear.

Best,  
Justine

---

<div class="post-metadata">

**Author:** ![jwdebelius](https://forum.qiime2.org/user_avatar/forum.qiime2.org/jwdebelius/32/9655_2.png) [@jwdebelius](https://forum.qiime2.org/u/jwdebelius)\
**Post date:** [September 6, 2019, 10:10am UTC](https://forum.qiime2.org/t/problem-with-fragment-insertion-sepp/11396/7 "2019-09-06T10:10:28Z")

</div>

Hi @Stefan,

Continuing with live debugging, I'm getting an error with I try the suggested command:

```auto
========= Execution of SEPP failed with exit code 1 =================                                                                                                                 
temporary working directories are NOT deleted for further inspection:                                                                                                                 
  $tmp = /tmp/tmp.o0ZesBHaZU/sepp-tmp-QoU2w                                                                                                                                   
  $tmpssd = /tmp/tmp.o0ZesBHaZU/sepp-tempssd-K86g
--------- Content of STDOUT ----------------------------------------- 
cat: sepp-./data/trees/silva128_insertion-out.log: No such file or directory 

```

I'm trying to run it outside the share directory; Id like to be able to instead reference the file I need.

Best,  
Justine

---

<div class="post-metadata">

**Author:** ![Stefan](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/s/bbce88/32.png) [@Stefan](https://forum.qiime2.org/u/Stefan)\
**Post date:** [September 6, 2019, 10:21am UTC](https://forum.qiime2.org/t/problem-with-fragment-insertion-sepp/11396/8 "2019-09-06T10:21:22Z")

</div>

Hi @jwdebelius,  
can you please post the full command that led to the error, and perhaps also the result of `pwd` and `ls -la` to give us a feeling of your path "environment" and presence / absence of necessary reference files?

---

<div class="post-metadata">

**Author:** ![Stefan](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/s/bbce88/32.png) [@Stefan](https://forum.qiime2.org/u/Stefan)\
**Post date:** [September 6, 2019, 10:27am UTC](https://forum.qiime2.org/t/problem-with-fragment-insertion-sepp/11396/9 "2019-09-06T10:27:57Z")

</div>

I just recall that there was another little bug in SEPP itself with providing the info file reference. We fixed that in the original sources and also compiled a new conda package for SEPP: [Sepp | Anaconda.org](https://anaconda.org/bioconda/sepp) but I am not sure if the latest qiime version uses this fixed package :-/ Thus, you might want to install (and thus overwrite) `conda install -c bioconda sepp>=4.3.10` prior to further debugging.

---

<div class="post-metadata">

**Author:** ![jwdebelius](https://forum.qiime2.org/user_avatar/forum.qiime2.org/jwdebelius/32/9655_2.png) [@jwdebelius](https://forum.qiime2.org/u/jwdebelius)\
**Post date:** [September 6, 2019, 11:38am UTC](https://forum.qiime2.org/t/problem-with-fragment-insertion-sepp/11396/10 "2019-09-06T11:38:59Z")

</div>

My command was

```auto
run-sepp.sh \
 ./data/trees/deblur_rep_seq.fasta \
 ./data/trees/silva128_insertion \
 -x 2 \
 -a /home/justine.debelius/trees/sepp-ref-tree/silva128/share/fragment-insertion/ref/silva12.8_99otus_aligned_masked1977.fasta \
 -t /home/justine.debelius/trees/sepp-ref-tree/silva128/share/fragment-insertion/ref/silva12.8_99otus_aligned_masked1977.tre \
 -r /home/justine.debelius/trees/sepp-ref-tree/silva128/share/fragment-insertion/ref/silva12.8_99otus_aligned_masked1977.info \
 -b 1

```

And, I realize I didnt print the error (or I got a new one...):

```auto
/home/justine.debelius/miniconda3/envs/qiime2-2019.7-mini/bin/run-sepp.sh: line 137: sepp-./data/trees/silva128_insertion-out.log: No such file or directory

```

The sub directories and referenced files exist, and Im working out of the right location, so I dont think its a path issue?

I tried the conda update, my current version is `sepp=4.3.10=py36_0`.

Best,  
Justine

---

<div class="post-metadata">

**Author:** ![Stefan](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/s/bbce88/32.png) [@Stefan](https://forum.qiime2.org/u/Stefan)\
**Post date:** [September 6, 2019, 11:57am UTC](https://forum.qiime2.org/t/problem-with-fragment-insertion-sepp/11396/11 "2019-09-06T11:57:12Z")

</div>

The second argument to `run-sepp.sh` is expected to be a _name_, not a real prefix as in a path :-/

Which is hard to see from the script [sepp/sepp-package/run-sepp.sh at bd26318e7857a98c5917a1b0c7b97aa4a9096e2c · smirarab/sepp · GitHub](https://github.com/smirarab/sepp/blob/bd26318e7857a98c5917a1b0c7b97aa4a9096e2c/sepp-package/run-sepp.sh#L4)

The script now tries to first combine the word _sepp-_ with your prefix `./data/trees/silva128_insertion` and than checks if the file `sepp-./data/trees/silva128_insertion` can be written - which is not the case due to the included `.` and `/`. I suggest you change the second argument to just: `silva128_insertion`

---

<div class="post-metadata">

**Author:** ![Felipe\_Rocha](https://forum.qiime2.org/user_avatar/forum.qiime2.org/felipe_rocha/32/5691_2.png) [@Felipe\_Rocha](https://forum.qiime2.org/u/Felipe_Rocha)\
**Post date:** [September 6, 2019, 12:16pm UTC](https://forum.qiime2.org/t/problem-with-fragment-insertion-sepp/11396/12 "2019-09-06T12:16:06Z")

</div>

Hi @jwdebelius and @Stefan,

well, it didn't work out, but now I have a huge debug file which I don't understand. 😅  
I think there is more to it above but I couldn't rescue in my terminal.

[debug-sepp.txt](https://forum.qiime2.org/uploads/short-url/1jAnuPMPqcHuYWZVb7Szy0QooXY.txt) (124.1 KB)

> [@jwdebelius](#):
>
> (“I’ll do insertion against Silva,” she said. “It will make my new boss happy,” she said. “How hard can it be?” …Two weeks of expensive debugging later, and I’ll let you know hopefully by sometime Saturday whether or not these trees worked for me.)

I hope you can find your way through fragment-insertion!

> [@jwdebelius](#):
>
> A .md file is a markdown file, it’s like HMTL light. You _can_ read it using a text editor, but some of the annotation will be different. (Actually, a lot like text formatting here). Or, my favorite mac program for markdown is [MacDown](https://macdown.uranusjr.com/) because it’s a relatively nice side-by-side rendering.

Thanks for the explanation, I'll download some markdown program to read that file.  
Thank you both for the discussion here, I'll do my best to understand and try each of the steps you both proposed here, unfortunately, I'm still not well versed in python, conda, and so on.

I see you are suggesting to use Silva 12.8 files, I was using the latest release (13.2). Should I try with the previous release then?

Cheers,  
Felipe.

---

<div class="post-metadata">

**Author:** ![jwdebelius](https://forum.qiime2.org/user_avatar/forum.qiime2.org/jwdebelius/32/9655_2.png) [@jwdebelius](https://forum.qiime2.org/u/jwdebelius)\
**Post date:** [September 6, 2019, 12:28pm UTC](https://forum.qiime2.org/t/problem-with-fragment-insertion-sepp/11396/13 "2019-09-06T12:28:22Z")

</div>

> [@Stefan](#):
>
> The script now tries to first combine the word _sepp-_ with your prefix `./data/trees/silva128_insertion` and than checks if the file `sepp-./data/trees/silva128_insertion` can be written - which is not the case due to the included `.` and `/` . I suggest you change the second argument to just: `silva128_insertion`

Won't that then move my path to `sepp-data/trees/silva128_insertion`, creating a new folder and data structure? Does it mean I have to go to my target directory (`data/trees/` to run the command and can't run it from a parent directory?

---

<div class="post-metadata">

**Author:** ![jwdebelius](https://forum.qiime2.org/user_avatar/forum.qiime2.org/jwdebelius/32/9655_2.png) [@jwdebelius](https://forum.qiime2.org/u/jwdebelius)\
**Post date:** [September 6, 2019, 1:20pm UTC](https://forum.qiime2.org/t/problem-with-fragment-insertion-sepp/11396/14 "2019-09-06T13:20:54Z")

</div>

It looks like the `run-sepp.sh` script won't write if you try to run it from a directory other than the target directory with a relative path. `¯\_(ツ)_/¯`

---

<div class="post-metadata">

**Author:** ![Stefan](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/s/bbce88/32.png) [@Stefan](https://forum.qiime2.org/u/Stefan)\
**Post date:** [September 6, 2019, 1:32pm UTC](https://forum.qiime2.org/t/problem-with-fragment-insertion-sepp/11396/15 "2019-09-06T13:32:40Z")

</div>

@Felipe_Rocha from scrolling through your debug report, I figure your computer has too few memory: `OSError: [Errno 12] Cannot allocate memory`

---

<div class="post-metadata">

**Author:** ![Felipe\_Rocha](https://forum.qiime2.org/user_avatar/forum.qiime2.org/felipe_rocha/32/5691_2.png) [@Felipe\_Rocha](https://forum.qiime2.org/u/Felipe_Rocha)\
**Post date:** [September 6, 2019, 3:17pm UTC](https://forum.qiime2.org/t/problem-with-fragment-insertion-sepp/11396/16 "2019-09-06T15:17:43Z")

</div>

> [@jwdebelius](#):
>
> It looks like the `run-sepp.sh` script won’t write if you try to run it from a directory other than the target directory with a relative path. `¯\_(ツ)_/¯`

OK, so I should try to run manually the script with my target directory open in the terminal?

> [@Stefan](#):
>
> @Felipe_Rocha from scrolling through your debug report, I figure your computer has too few memory: `OSError: [Errno 12] Cannot allocate memory`

Wow, I've been using a remote server with ~380GB of RAM, isn't that enough? Or is there something going on while I use this remote server which prevents me of using its memory?

---

<div class="post-metadata">

**Author:** ![jwdebelius](https://forum.qiime2.org/user_avatar/forum.qiime2.org/jwdebelius/32/9655_2.png) [@jwdebelius](https://forum.qiime2.org/u/jwdebelius)\
**Post date:** [September 7, 2019, 11:55am UTC](https://forum.qiime2.org/t/problem-with-fragment-insertion-sepp/11396/17 "2019-09-07T11:55:28Z")

</div>

> [@Felipe\_Rocha](#):
>
> OK, so I should try to run manually the script with my target directory open in the terminal?

Yes, navigate to your target directory (wherever you want the tree to come out) and then run the come. If you use path (like I was), you'll get the same "can't make" error.
