ghost-tree index out of range error

Dear all,

I am trying to use ghost-tree through conda. I have made the files to use based on https://github.com/JTFouquier/q2-ghost-tree/tree/master/small_test_files/original-non-qiime-files.

However,
I am getting this errors:
Traceback (most recent call last):
File “/Users/Irene/miniconda3/bin/ghost-tree”, line 403, in
cli()
File “/Users/Irene/miniconda3/lib/python3.7/site-packages/click/core.py”, line 764, in call
return self.main(*args, **kwargs)
File “/Users/Irene/miniconda3/lib/python3.7/site-packages/click/core.py”, line 717, in main
rv = self.invoke(ctx)
File “/Users/Irene/miniconda3/lib/python3.7/site-packages/click/core.py”, line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/Users/Irene/miniconda3/lib/python3.7/site-packages/click/core.py”, line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/Users/Irene/miniconda3/lib/python3.7/site-packages/click/core.py”, line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/Users/Irene/miniconda3/lib/python3.7/site-packages/click/core.py”, line 555, in invoke
return callback(*args, **kwargs)
File “/Users/Irene/miniconda3/bin/ghost-tree”, line 365, in hybrid_tree
foundation_taxonomy_file)
File “/Users/Irene/miniconda3/lib/python3.7/site-packages/ghosttree/scaffold/hybridtree.py”, line 104, in extensions_onto_foundation
graft_level)
File “/Users/Irene/miniconda3/lib/python3.7/site-packages/ghosttree/scaffold/hybridtree.py”, line 196, in _extension_genus_accession_dict
graft_level)
File “/Users/Irene/miniconda3/lib/python3.7/site-packages/ghosttree/scaffold/hybridtree.py”, line 237, in _create_taxonomy_dict
taxonomy_line = splitline[1].strip()
IndexError: list index out of range

Coudl anyone help understand what I am doing wrong?
Thanks!

Hi Irene, unfortunately, my school schedule is really packed this week. I’ll have a chance to get to this later tomorrow or Thursday. If for some reason I forget, please remind me. I’ve had a lot going on lately. Thank you!

2 Likes

Hi Irene! Welcome to the QIIME2 forum and community. :wave: Sorry this took so long, my week has been a little crazy. :slight_smile: Thank you so much for always providing such clear and detailed help requests! Posting on the forum is great because other users can get help. The formatting here is better, too!

After looking at the files that I have from you in my email, It looks like you are providing a taxonomy line like this 'OTU_207 k__Fungi;p__Basidiomycota;c__Agaricomycetes;o__Agaricales;f__Amanitaceae' in your extension tree taxonomy file which does not go down to genus and species. Normally, the taxonomy file would have g__unclassified;s__unclassified or g__unidentified;s__unidentified or something along those lines as placeholders so that a lot of the microbiome bioinformatics tools see things as consistent. This is a "standard", as you can imagine consistency is easier to work with in computer programming. You will need to repair your taxonomy file to have ';g__unclassified;s__unclassified' on all the lines just as placeholders. You can find concatenate functions in Excel to add this string to each line or do this in Python. I'm not sure your computational level, so if you cannot do this or find help in your lab, let me know.

Now, normally ghost-tree uses genera to make smaller groups of related fungi, but since you do not have genera (they're all unclassified), you will have to do a custom tree. I added this feature: Options for plugin inputs/arguments (ghost-tree progress) a while back but I have not tested it too much besides on smaller trees. You don't need to read the details, but you can see the cool images showing how the graft level option/flag creates trees at different taxonomic groupings. When you're in command line, type --help after your ghost-tree command, then you will see the option to graft at a higher level. So you will do --graft-level f in your case.

Screen Shot 2020-01-24 at 7.09.21 AM


Luckily your tree will be fairly small, so make sure to open in a tree viewer and make sure the groups make sense to you after you create your custom tree. OTUs in the same family should be grouped together, of course! :sweat_smile:. If they aren't let me know. I would also try to compare your diversity analysis results if you plan to do them to non-phylogenetic or taxonomic diversity analysis methods such as bray-curtis or binary-jaccard.

Let me know if you have any other questions, and feel free to email me any cool results. Would love to see them. :star_struck:

3 Likes

Dear Jennifer,

thanks a lot for your answer! it is very clear!

Do I need to change the taxonomy also in the foundation taxonomy with genus and species space as unidentified?

Can I use --graft-level f with the ghost-tree scaffold hybrid-tree-foundation-tree, and do I use after the command or in a specific place in the command line?

I will be happy to show the final results!!

Thanks a lot again for your answer!

1 Like

I can’t remember off the top of my head. I might not have enforced that check for the foundation tree, but I should have required the taxonomy to go down to the species level. It’s best to keep things consistent so I would repair both.

If you type ghost-tree scaffold hybrid-tree-foundation-tree --help you will see the option for other graft levels, so yes it works. ghost-tree on its own (used outside of the QIIME2 environment) is picky about the order of things so just put --graft-level f at the end.

Perfect!
so I changed both the extensions and foundation with g__unidentified;s__unidentified for the extension taxonomy and unidentified;unidentified for the foundation taxonomy.

then I ran
ghost-tree scaffold hybrid-tree-foundation-tree EXTENSION_TREES_OTU_MAP.txt EXTENSION_TREES_TAXONOMY_FILE.txt EXTENSION_TREES_SEQUENCE_FILE.fasta FOUNDATION_TREE_FILE.nwk FOUNDATION_TAXONOMY_FILE.txt GHOST_TREE_OUTPUT_FOLDER --graft-level f
I understood this is the way I should do it , but I still get the same exact error.. I have tried to move the option --graft-level but still same error. I have checked the code in github and other than the graft level i cannot see any other error. I really would like to get the hybrid tree done since in my case this is the best method.
thanks!!!!

EXTENSION_TREES_TAXONOMY_FILE.txt (28.9 KB) FOUNDATION_TAXONOMY_FILE.txt (8.7 KB)

1 Like

Hi Irene,

It looks like there is an issue with some inconsistency in the tabs in your extension file but you can verify the other files as well.

If I type cat -t EXTENSION_TREES_TAXONOMY_FILE.txt into the terminal, I can see some tabs and some without tabs on line to line. I'm not sure how these were introduced. So you can search for how to repair the tabs and spacing in your file.

The '^I' shows the inconsistency. This inconsistency will cause errors in the code because it's expecting the same organization on each line.

1 Like

Dear Jennifer,

you were totally right! now it works perfectly! I will be happy to show the results once I annotate the tree, but for now I can tell you that all the OTUs were correctly placed!

Thank you so much for all your help!!!

2 Likes