Generate Krona charts from Taxa Barplots

I have written a python script to generate Krona charts from taxa barplots, and I would like to share it.

Krona charts are interactive pie charts, and I think they are useful because it visualizes samples one by one very good.

But the problem is we need to change our OTU/ASV tables to a tsv file which Krona likes.

Krona tsv format is like this: krona-format.tsv (64 Bytes)

An example Krona tsv: O5.tsv (15.4 KB)

An example Krona chart (upload as zip since it is not allowed to upload html): Krona.zip (50.8 KB)

To get started we need to install Krona. Here is the github page : Home · marbl/Krona Wiki · GitHub

After the install I do not recomment to run update scripts in KronaTools because we won't use them and it will start a huge download.

We need to be able to call Krona from bash so make sure that ktImportText command works.

After that, we just need a taxa-bar-plots.qzv file to work with this Python script: generate-krona.py (2.1 KB)

Simply, the script extracts level-7.csv from taxa-bar-plots.qzv, creates tsv files for krona, generates krona file as html, removes anthing unnecessary in the process.

You can read the comments to make changes in the script. Script is written to work with python3.

Have fun.

(First time posting a tutorial, I need all the criticism and suggestions to improve it. )

11 Likes

Awesome work @the_dummy! I am looking forward to giving this a spin!

Looks like you are just a stone's-throw away from writing your own QIIME 2 plugin... interested in giving that a try? We can point you in the right direction!

#extract level-7.csv from taxa barplots

Why extract from the barplot.qzv? Why not accept a feature table and taxonomy file (or a feature table that has been collapsed to a specific level) as input? That would be the first step toward making this a little more plugin-like and efficient.

4 Likes

Well, now I will look into how the plugins work.

Why extract from the barplot.qzv? Why not accept a feature table and taxonomy file (or a feature table that has been collapsed to a specific level) as input? That would be the first step toward making this a little more plugin-like and efficient.

This is a very good idea. With this, you won't need to visualize everything if you want a limit to a level.

Now, with argparse, I have added some tricks to the script.

As @Nicholas_Bokulich suggested I tried to use collapsed table, and it extracts metadata.tsv from collapsed taxa visualization (a qzv file).

I didn't want to deal with a feature-table.biom because I don't know the nature of the file.

This script has input, output, exclude, regex options now!

krona-arg.py (2.3 KB)

Let's hear out how it can be improved much more..

3 Likes

Another good reason to write this as a QIIME 2 plugin — you can take a FeatureTable[Frequency] artifact as input, QIIME 2 will validate the input type and transform it into a pandas.DataFrame for manipulating and saving as a TSV for krona. No need to figure out the biom-format API (which while useful for many other things is not needed here since you are really just formatting the table for krona)

1 Like

Pneumothorax hits again!

Eh… I’m back again, but I need to look into this later. I was at hospital for like 10 days and everything is a mess.

Of course if anyone has problems with the script, I can help. Improvements have to wait :frowning:

1 Like

Ahhh! I hope you feel better soon and sort that out. Im glad you’re out of the hospital.

Hey, I’m glad you’re recovering!
This is a really nice work and I’am in complete agrement you should work on a plugin. People would use it.
Cheers

Hello there, I tried running your script with the taxa-bar-plots.qzv flie but I keep gettting a syntax error, it looks like this:

./generate-krona.py: line 6: syntax error near unexpected token (' ./generate-krona.py: line 6: def unzip(qzv_file):'

Well, first thing I thought was your python version. I have written it in python3 so you could check that.

Also, try this krona-arg.py. Check --help with this one.

And shout out to everyone, I’m back and will work on changing this into a plugin!

2 Likes

welcome back @the_dummy! Glad to hear you are better!

1 Like

(base) [mqekum@node0511 ~]$ python --version
Python 3.7.4
Is this version of python a problem?, the krona-arg.py script also gives a similar error:

(base) [mqekum@node0512 Healthy-bloem] ./krona-arg.py ./krona-arg.py: line 10: syntax error near unexpected token `(' ./krona-arg.py: line 10: `parser = argparse.ArgumentParser()' (base) [mqekum@node0512 Healthy-bloem] parser = argparse.ArgumentParser()’
bash: syntax error near unexpected token `(’

Then after trying --help i get this:

(base) [mqekum@node0512 Healthy-bloem]$ python krona-arg.py --help
File “krona-arg.py”, line 25
xif args.exclude:
^
SyntaxError: invalid syntax

Quick question does the script require the taxa-bar-plots.qzv and the level-7.csv files to be in a specific folder. Another thing I work with fungal data using the UNITE database, does this have to be considered?

Hello again, I’m having an issue running the script

Traceback (most recent call last):
File “krona-arg.py”, line 72, in
lines.pop(0)
IndexError: pop from empty list

Well, you are hitting an error in almost every single line of the script. I believe there is a common problem with all of these errors.

How did you solve the previous problem? This might lead me somewhere.

Hello again,

Just an update with the work.

THIS IS HARD!

Since my background is molecular biology I’m having serious trouble. Is there any material other than q2 plugin development which would help me understand the idea of developing a plugin, the perspective, whatever that is called. Maybe an outside video, paper etc. about this process.

I feel like I need to understand the logic first. I don’t believe that I lack the required programming skills.

I really want to make this into a plugin to give back to the community. Any help would be appreciated.

1 Like

Thanks for your hard work! It will all be so worth it in the end! :smile:

The tutorial you've found is unfortunately really the only tutorial we have that explains the process for a first-time plugin developer. The other main resource we have is what I would call the desk reference for QIIME 2 development:
https://dev.qiime2.org/latest/

But that is not really what you are looking for.

I have found that the best way to learn to write a QIIME 2 plugin is to look at the source code of other plugins and use that as a template... in your case, check out a plugin like q2-diversity and focus on a visualizer action like alpha-group-significance.

We know we need better resources to help new plugin developers... but in the meantime we are here to answer your questions! So ask away — if we get a good back-and-forth discussion we may split it off into a separate forum topic that we can point to as a new learning resource for plugin development :smile:

1 Like

I have checked taxa barplot. That gave me some ideas and moral support :smiley: .

  1. I think I can use extract function in this plugin.
  2. Seems like I need to make scripts as _version.py, _method.py etc. so that it is easier to upgrade at some point. I don’t know much about the reason of importing different scripts instead of making a single one, maybe some explanation could enlighten me.
  3. Extraction seems easy but I didn’t see a clear way to make .qzv file.
  4. There were some js files, but I don’t know anything about js. What does that part do with the plugin? Is it about .qzv generation, because as I told before, I couldn’t see something related.

Many more questions will come :face_with_monocle:

1 Like

Good questions! barplot may not be the best one to look at since it uses some javascript (that's what the .js files are all about). Maybe look at diversity beta-group-significance instead as a simpler example.

What for specifically? Is there a line in the q2-taxa source code that you are referring to? Or do you mean the qiime tools extract function? If you give me your use case I may be able to think of a solution.

Splitting up your code into separate modules just makes everything easier to navigate; it helps keep things neat and tidy for maintenance and testing. You don't need to, it's just good practice.

QIIME 2 does that for you! You would create an index template file as described here and set up the appropriate plugin registrations as described in that tutorial.

For example, see how it is done in beta-group-significance. Most of that command is working to run the tests and generate the plots/outputs. Then it is all tossed into an HTML template with these lines:

The first 2 lines are setting the path to the index.html template, then the following lines are loading that template and inserting different data (stats results, plots) into different sections of the template. For example, the result_html (a table of results rendered as html) is being inserted (and formatting) at this point in the template:

At no point in the code is the HTML being built "from scratch", though I suppose that is another valid option.

In your case, it looks like KRONA is already outputting an html... so maybe you don't even need to follow these steps. Instead, save that HTML file as output_dir/index.html... give it a spin and let's see!

2 Likes

Seems like krona pop up in the workshop too :slight_smile:

Ok, I will finish this thing faster now…

1 Like

Here is the plugin topic. Posting it just in case.

4 Likes