Issues with q2-ghost-tree .nwk file

I am trying to import a pre-built ghost-tree following the guidelines in this tutorial, but I get the error:

There was a problem importing ghost_tree.nwk:

ghost_tree.nwk is not a(n) NewickFormat file

I have selected this pre-built tree to try & get further in my analysis, but I cannot proceed. I feel like I’m missing something obvious, but I cannot figure out where I’m going wrong. I am having difficulty getting back in the groove of things after recovering from “the crud”, so I’m probably overlooking something simple.

Any help is much appreciated.

2 Likes

Hey there @jhines1! Can you provide some more specifics, like, what version of QIIME 2 are you running, what is the exact command you ran, and what is the complete error message observed? Thanks!

1 Like

Hi @jhines1! :slight_smile:,

I just tried importing it and I get the green light from QIIME 2. I'm using QIIME 2019.1. I just wanted to make sure that file specifically works, but please follow up with Matt's questions if it's still not working for you. -Jennifer

wget https://raw.githubusercontent.com/JTFouquier/ghost-tree-trees/master/silva-132-foundation/01.12.2017/ghost_tree_80_qiime_ver7_99_01.12.2017/ghost_tree.nwk
--2019-03-04 14:50:00--  https://raw.githubusercontent.com/JTFouquier/ghost-tree-trees/master/silva-132-foundation/01.12.2017/ghost_tree_80_qiime_ver7_99_01.12.2017/ghost_tree.nwk
Resolving raw.githubusercontent.com... 151.101.68.133
Connecting to raw.githubusercontent.com|151.101.68.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1347120 (1.3M) [text/plain]
Saving to: 'ghost_tree.nwk'

ghost_tree.nwk                    100%[==========================================================>]   1.28M  4.61MB/s    in 0.3s

2019-03-04 14:50:01 (4.61 MB/s) - 'ghost_tree.nwk' saved [1347120/1347120]

(qiime2-2019.1) jenniferfouquier@MacBook Pro:~/Desktop$ qiime tools import --input-path ghost
ghost-tree-collapse/ ghost_tree.nwk
(qiime2-2019.1) jenniferfouquier@MacBook Pro:~/Desktop$ qiime tools import --input-path ghost
ghost-tree-collapse/ ghost_tree.nwk
(qiime2-2019.1) jenniferfouquier@MacBook Pro:~/Desktop$ qiime tools import --input-path ghost_tree.nwk --type Phylogeny[Unrooted] --output-path ghost-tree-unrooted.qza
Imported ghost_tree.nwk as NewickDirectoryFormat to ghost-tree-unrooted.qza 

2 Likes

I am using qiime2-2019.1. The error message I received was in my OP. That was all it said. I am about to head to a meeting, then home, but I will respond with the exact code first thing tomorrow morning.

I’m sure I just overlooked something. It’s good to know @Jennifer_Fouquier got it to go through. It’s most likely something silly on my end.

Thanks & I’ll be in touch.

1 Like

How about the full command you ran?

Unfortunately, I don’t have access to that at the moment. I am using the Mac in the lab & I didn’t have time to save my editor to Google Drive before I had to leave. I’ll look it over tomorrow & post the full command in the morning when I get back to the lab.

Thanks/sorry!

1 Like

Sorry for the delayed response, but I got it to work. I’m not sure what I did any different from yesterday, but it saved from .nwk to .qza today. Maybe the Mac was just having a case of the Monday’s or something.

Either way, thank you so much for the quick responses & the help!! Much appreciated!

3 Likes

Hello @Jennifer_Fouquier,
Could you please provide the ghost_tree.nwk using the latest version of UNITE (ver8 from 02.02.2019)? I was not able to successfully install the q2-ghost-tree to be able to build them by myself.
I am interested in the dynamic version.

Thank you!

1 Like

@helenaax2r, hello!

Yes, I will work on that early next week. Please note that there is currently an issue with using these ghost trees in QIIME 2 that may or may not be (TBD) related to the underscores found in accession IDs that are in the recent UNITE databases (previous UNITE versions did not have these characters and were the files I used for testing). I'm not entirely sure, but it's the only difference I can see that has arisen with the new UNITE files and was not something I would expect to be an issue :slight_smile: . You can follow here: Ghost tree filtering? - #8 by Jennifer_Fouquier. Thanks!

Hi Jeniifer,
I could not find a tree for 2.2.19 UNITE data base, so im assuming its not ready yet?
I wanted to create one by my self but I could not understand the tutorial properly.

  1. in the first step im creating an otu map? but im using ASVs, is that ok? should i increase the simiilarity from 90% ?

  2. i could not figure how to get this input files:
    –i-extension-taxonomy
    –i-extension-sequences extension
    –i-foundation-tree foundation_
    –i-foundation-taxonomy

Your help would be much appreciated :))

1 Like

Hi there Arnon.

I forgot to update this thread as well. I had a family emergency and am behind. Sorry everyone! cc @helenaax2r

I do have time to work on making new trees today and tomorrow if you would like to do other things in the meantime. It appears that there are quite a few questions related to ghost-tree at the moment, so I need to make sure things are running smoothly in QIIME 2 which likely involves clarifying things on the tutorial.

ghost-tree was developed before the ASV process, that's why the OTU picking exists. It also has a benefit though. It allows you to regroup your ASVs to lower percent identity threshold. Because ghost-tree partially works using nomenclature, this will allow some of the "unidentified" organisms to cluster with organisms that are "identified" taxonomically in the hopes that you will maintain some of their phylogenetic information rather than discarding them.

Example from the tutorial:

Group your rep seqs at 90% similarity.

This handles the abundant ‘unidentified’ organisms.

qiime ghost-tree extensions-cluster \
--i-extension-sequences extension_seqs.qza \
--p-similarity-threshold 0.90 \
--o-otu-map extensions_otu_map_90.qza

I'm not sure what your question is on point 2.

Hope that helps for now. Let me make some trees :slight_smile: -Jennifer

3 Likes

Hey Jennifer, thank you for replying :slight_smile:
I hope everything is OK with you and your family!

thank you for taking your time working on the new trees, apparently their high on demand now ^^
please let us know when they are ready :slight_smile:

my 2nd point related to the input files for the different command in the plugin.
specially the foundation files:
–i-foundation-tree foundation
–i-foundation-taxonomy

where they are coming from, how do i create them in the first place?

1 Like

Hi Arnon,

Still working on the trees. Will take me a few more days to get to the bottom of an issue that was recently identified. :upside_down_face:

Have you seen this tutorial? There are small files you can use as a demo, and it also mentions the "full size files"

I agree, the tutorial could use more description of what the files are, and not just the commands used. Thanks for bringing this up. I need to expand some of the documentation.

The paper might also help clarify things:

The foundation tree is a tree built from an alignment database for an evolutionarily conserved marker gene (like the 18S for the fungal example) that can be aligned across distant taxonomic lineages to build a “foundation” tree. It's the tree that you will use to graft your extension sequences onto. When you have sequences such as ITS that are good at taxonomic identification but that make terrible alignments across all lineages (but ITS does make good alignments for closely related organisms, and it has good taxonomic resolution/identification), you don't just want to make a tree from all of your ITS seqs....so ghost-tree grafts them onto the foundation to build a hybrid-gene tree that has the benefits of both ITS and 18S gene regions. Please note that the foundation can be either a tree or an alignment but you have to choose the right subcommand in ghost-tree.

Silva 18S is a good foundation tree for fungal ITS. For other organisms, I'm not sure. I'm guessing there are more applications but ghost-tree was designed with fungal diversity analysis in mind.

2 Likes

Hi @helenaax2r and @1119,

Here are the most recent UNITE database ghost trees. :grin:

There is an issue in ghost-tree that I will fix soon and you can find the details here if you are interested. Basically, the pre-built trees are good to go and formatted correctly, but the software needs to be modified so it adds quotes to the IDs in the Newick to allow underscores that are in the new UNITE accession IDs. I will do that next and improve docs.

If you have any questions, please let me know, especially while documentation is sparse!

2 Likes

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.