importing feature tables as Pandas df with artifact API

Hello, I’ve done most of my contaminant removal and inspection in python and I’d like to re-import the new feature table as a qiime2 artifact. I have the feature table in a wide-format df with the index as sample_id. I was following the artifact api tutorial but when I used the following, the command seemed to run, there were no errors, but also no .qza file generated.

ASV_table = Artifact.import_data("FeatureTable[Frequency]", feature_wide)

When I ran the same command using a table that had a numerical index I received an error to the effect of ‘first column must be character string.’ So I was hoping the re index would solve that issue.

Thank you

2 Likes

Hi @hsapers! If you would like to save a file to disk you can call ASV_table.save('my-filename.qza') - replace my-filename.qza with the path to the file you would like to save to. Hope that helps!

2 Likes

Thanks @thermokarst

I think I still must be missing something and still have trouble moving things from python to q2 to R and back. Right now I have my feature table, tax table, and metadata as three panda’s dataframes. To save the metadata and import into q2 I can use

# save to disk in python
meta_sample_update.to_csv('meta_update.csv', index=False)

# read into q2 bash - or bring in with any function that needs metadata
!qiime metadata tabulate \
     --m-input-file metadata_mapping.tsv \
     --o-visualization tabulated-sample-metadata.qzv

But I seem to be getting stuck with the tax and feature tables. I’ve tried:

# save to disk bash
!ASV_df_control_ASV_removed.save('ASV_update.qza')

returns

/usr/bin/sh: -c: line 0: syntax error near unexpected token `'ASV_update.qza''
/usr/bin/sh: -c: line 0: `ASV_df_control_ASV_removed.save('ASV_update.qza')'

and

#save to disk python
ASV_df_control_ASV_removed.save('ASV_update.qza')

returns

AttributeError: 'DataFrame' object has no attribute 'save'

Is there a save module from q2 that I need to import?
Once I have the tables saved to disk as qza files, can I import them into q2 using:

ASV_file.qza = Artifact.import_data("FeatureTable[Frequency]", feature_wide)
tax_file.qza = Artifact.import_data("TaxonomyTable")

Thank you

Hi @hsapers! You appear to be mixing python commands and shell script commands in your script.

shell command:

note, in jupyter/ipython, prefixing the “command” with a ! automatically makes it a “shell command” - at that point this is no longer running in python, but instead it is running like a traditional shell (“terminal”) command.

So hopefully that makes this error clear:

you’re trying to run python code as a shell command - the two aren’t compatible.


Where are you getting ASV_df_control_ASV_removed from? According to the error message its a pandas Dataframe, which isn’t a QIIME 2 Artifact, or Results object.

If you’re still running into problems, please share a full set of commands (minimum working example) - we will need the full picture of what you’re doing to provide further assistance. Thanks!

I was thinking, maybe a pure python example might help:

import pandas as pd
import qiime2
from qiime2.plugins import feature_table as ft_plugin


# create a feature table, in pandas
table_df = pd.DataFrame([[1, 2, 3],
                         [4, 5, 6],
                         [7, 8, 9]],
                        index=['s1', 's2', 's3'],
                        columns=['f1', 'f2', 'f3'])

# import pandas feature table into QIIME 2
table_artifact = qiime2.Artifact.import_data('FeatureTable[Frequency]', table_df)

# save QIIME 2 Artifact
table_artifact.save('table1.qza')

# convert table to relative frequency, using a QIIME 2 plugin
relative_freq_results = ft_plugin.methods.relative_frequency(table=table_artifact)

# look at the results
print(relative_freq_results)
# Results (name = value)
# -----------------------------------------------------------------------------------------------------------------
# relative_frequency_table = <artifact: FeatureTable[RelativeFrequency] uuid: 6eeb8bd6-7cce-43af-a4e5-9fed6b6cec41>

# notice the name of the output: relative_frequency_table.
# If I want to get that Artifact from the results:
table_relative_artifact = relative_freq_results.relative_frequency_table

# save the new relative frequency table
table_relative_artifact.save('table2.qza')

:qiime2:

2 Likes

Brilliant - thanks @thermokarst!

I have been using SoS notebooks and flipping between language kernels in different cells. I was missing the link between importing the object and saving it as a qza object and I thought maybe the save command was a shell command.

Are there similar q2.Artifact.import_data('FeatureTable[Frequency]', df) commands for tax tables? Or is it easier to subset the tax.qza using the feature table?

Thank you

You’re welcome!

Makes sense.

Yes - Artifact.import_data! QIIME 2 uses a semantic type system to “label” all imported and generated data - the 'FeatureTable[Frequency]' bit is an example of a semantic type - you would read this as “a feature table of frequency”. You can import a 'FeatureData[Taxonomy]' using the same mechanism shown above.

I don’t follow, perhaps you can share a minimum working example?