How to save the CSV / create a table from the barplot visualisation using Qiime2 API

[cosull]

Good afternoon,

I have been struggling on this for a few days.

I am able to generate the barplot with our taxonomy, that much works (see below). I am unable to extract the CSV from the barplot which has all the data combined using python or the qiime2 API.

from qiime2.plugins.taxa.visualizers import barplot
taxa_bar_plot = barplot(denoised.table, taxonomy.classification, metadata_table)
taxa_bar_plot.visualization

The only way I can generate the CSV I want (as shown in my screen capture below) is by dragging my visualization file (.qzv) into the online qiime2 viewer and manually using the download CSV option at https://view.qiime2.org. See here my screen capture.

See here the video capture of the file we want to get but via code not in the viewer as shown here:

http://recordit.co/p0QBZ0Hji8

Of course, we are just testing this in the online viewer tool but really we want to generate this same CSV but using the Qiime2 API in python in our Jupyter notebook. Does anyone know how we can do this in code?

That button in the online qiime2 viewer is getting me the EXACT CSV file we need to proceed beyond where we are stuck. We need to do this in python of course. We will use this table / data frame to do our next step which will be diversity analysis.

I would appreciate any help on this as it has been a few days trying to figure this out.

THANK YOU IN ADVANCE.

Col

[thermokarst (Matthew Ryan Dillon)]

Hi @cosull, this is a great question!

Luckily, all QIIME 2 outputs are zip files, and have a convenient API for getting at the contents of the zip file.

import pathlib
import shutil
import tempfile

# this function exports all CSV files from a QIIME 2 Visualization
# to the filepath specified in `dest`
def extract_csvs(viz, dest):
    # create a temp dir to work in, that way we don't have
    # to manually clean up, later
    with tempfile.TemporaryDirectory() as temp:
        # export the `data` directory from the visualization
        viz.export_data(temp)
        temp_pathlib = pathlib.Path(temp)
        # iterate through all of the files that we just extracted above
        for file in temp_pathlib.iterdir():
            # if the file is a csv file, copy it to the final dest
            if file.suffix == '.csv':
                shutil.copy(file, dest)

The interesting part is the viz.export_data() line - you can export anywhere - in this example I chose to export to a temp directory, that way I can pull out the files that I am interested in, later on in the function.

To string this together with what you have posted above:

from qiime2.plugins.taxa.visualizers import barplot
taxa_bar_plot = barplot(denoised.table, taxonomy.classification, metadata_table)
extract_csvs(taxa_bar_plot.visualization, '/path/to/save/csv')

You could also modify the function to load those CSVs as Pandas dataframes, for example, and return those, instead of copying the CSV files. If you would like, I could show you an example of that, too - just let me know!

[timanix (Timur Yergaliyev)]

Hi! I used this code to read the table at desired taxonomy level in Jupyter lab without Qiime API:

import pandas as pd
def bar_unzip(qza,lev):  
    a = !unzip $qza
    digest = a[1].split('/')[0].replace('  inflating: ','')
    inf = digest + '/data/level-%s.csv'%lev
    data = pd.read_csv(inf, sep=',',index_col=0)
    !rm -r $digest
    return data

data = bar_unzip('taxabarplot.qzv',2)

First argument - barplot and second - taxonomy level. This function deletes extracted files after reading, but you can modify it as you want.

[jwdebelius (Justine Debelius)]

Hi @cosull,

A third option (because there are many ways to go!) would be to generate that table yourself. So, I think qiime taxa collapse will get you a feature table at a specific taxonomic level (i.e. genus) without having to go through visualization.

Best,
Justine

[cosull]

Thanks @thermokarst.

This worked perfectly. Thank you.