Pairwise Problems with Re-imported Taxa

[jbethany (Julie Bethany Rakes)]

Hello,

I’m processing data to compare microbial community shifts and I’ve imported a file with my data filtered for cyanobacteria only and updated the taxonomy; however, when I try to run pairwise PERMANOVA I get an error (see below). I don’t receive this error when running pairwise PERMANOVA’s with the original, unfiltered file. All files are formatted the same with the exception of removing non-cyanobacteria taxa.

Plugin error from diversity:
All values in the grouping vector are unique. This method cannot
operate on a grouping vector with only unique values (e.g., there are
no ‘within’ distances because each group of objects contains only a
single object).
Debug info has been saved to /tmp/qiime2-q2cli-err-jb1gnjr1.log.

I would appreciate any help! Thanks!

Julie

[ebolyen (Evan Bolyen)]

Hey @jbethany,

Sorry for the delayed response.

@jbethany

Plugin error from diversity:

All values in the grouping vector are unique. This method cannot
operate on a grouping vector with only unique values (e.g., there are
no ‘within’ distances because each group of objects contains only a
single object).

This is basically saying that the metadata column you provided doesn't have any groups to compare, so there's nothing for PERMANOVA to do.

What is the exact command you are running? And what --m-metadata-category are you using? I'm thinking something went wrong in filtering, but I'd need more information about what groups are supposed to exist to really say for sure.

[jbethany (Julie Bethany Rakes)]

Hi ebolyen,

Thanks for getting back with me! I’m attempting to run the following script:

qiime diversity beta-group-significance --i-distance-matrix core-metrics-results-Cyano/bray_curtis_distance_matrix.qza --m-metadata-file RecyclingPlots-metadata.tsv --m-metadata-category Combo --o-visualization core-metrics-results/Bray_Curtis_Combo.qzv --p-pairwise

The metadata category is a combination of factors, all text based. An example would be HSiI2 and HSiF2. I have triplicates of each of these in the column. I can get the Bray Curtis results and p-values based on this column when all bacterial data are included but not when I re-import the cyanobacterial data only.

Thanks!
Julie

[ebolyen (Evan Bolyen)]

Would it be possible to send your metadata file (in a DM or otherwise)?

@jbethany

I can get the Bray Curtis results and p-values based on this column when all bacterial data are included but not when I re-import the cyanobacterial data only.

If you look at the metadata column "Combo", do all of the sample IDs that are cyanobacterial have the same value?

[jbethany (Julie Bethany Rakes)]

Hi Evan,

I've attached the metadata file. I've tried adjusting values in the Combo column (removing underscores, changing to numeric entry), with no luck.
I only have cyanobacterial information when trying to run the pairwise Permanova. The sample ids do not all have the same values in the Combo column, they are in triplicates.

Recycling-metadata.tsv (1.5 KB)

Thanks!

[thermokarst]

Thanks for sharing your metadata! Unfortunately there wasn't a smoking gun there...

Can you provide a little more detail regarding "I’ve imported a file with my data filtered for cyanobacteria only and updated the taxonomy," in your original post above? Are you filtering your feature table to remove Cyanobacteria? If so, can you run qiime feature-table summarize on the filtered table, and send the "Frequency per sample detail" CSV?

My hypothesis right now is that your filtering might also be removing samples, making it so that your metadata is actually a superset of the samples represented in your filtered feature table, which might be confounding what is happening here.

If you are able to share your filtered feature table, the provenance would be really useful for helping us piece together your actions!

Thanks!!

[jbethany (Julie Bethany Rakes)]

Definitely, so I'm using Qiime1 with the following script to remove any non-cyanobacteria. I'm interested in only the cyanobacteria community.

filter_taxa_from_otu_table.py -i table.w_omd.biom -o otu_table_cyano_only.biom -p p__Cyanobacteria

Originally, it looked like sample removal might be the problem because there was a difference in the number of samples between the metadata file and the feature-table summary. I had to remove some of my samples and didn't update the metadata file.

However, I just tried updating my metadata file with the appropriate number of samples, sorted based on the Site column and still get the same error. I'm attaching my new metadata file and the Frequency per sample detail qzv (I can't upload csv files to this site).

New-metadata.tsv (1.4 KB)

FeatureTableSummary.qzv (340.2 KB)

Thank you!

[thermokarst]

Thanks for all the great info @jbethany!

@jbethany

filter_taxa_from_otu_table.py -i table.w_omd.biom -o otu_table_cyano_only.biom -p p__Cyanobacteria

Not really related to your issue here, but we actually support this kind of filtering directly in QIIME 2! There are a few examples floating around the forum, but this one is a good starting point! Apologies if you already knew about this, but I wanted to toss it out there --- I had actually been assuming you performed your filtering in QIIME 2, which was why I asked for the filtered feature table's provenance --- I would've been able to see exactly how filtering took place. No big deal, but provenance is certainly one of my favorite features of QIIME 2!

@jbethany

However, I just tried updating my metadata file with the appropriate number of samples, sorted based on the Site column and still get the same error.

The metadata file you attached doesn't have a Combo column , can you please share the exact command you ran (since it doesn't look like your old command would've made sense here), as well as the exact error message with it?

Also just for grins, does it work for any metadata column?

Thanks for your patience!

PS - There are no "required columns" in metadata - you don't need to include things like BarcodeSequence or LinkerPrimerSequence if they aren't applicable to your study! Also, you can name the ID column whatever you want, so if you would rather it say Sample (instead of #SampleID, which is a QIIME 1 compatibility thing), you are more than welcome to change. No worries either way, just wanted to let you know!

[jbethany (Julie Bethany Rakes)]

Thanks for the info about filtering in QIIME 2! I’ll definitely move to that.

Here’s the new command (without the Combo column):

qiime diversity beta-group-significance \
  --i-distance-matrix core-metrics-results-CyanoOnly/bray_curtis_distance_matrix.qza \
  --m-metadata-file New-metadata.tsv \
  --m-metadata-category Site \
  --o-visualization core-metrics-results/Bray_Curtis_Combo.qzv \
  --p-pairwise

Also, here’s the exact error:

Plugin error from diversity:

All values in the grouping vector are unique. This method cannot
_ operate on a grouping vector with only unique values (e.g., there are
_ no ‘within’ distances because each group of objects contains only a_
_ single object)._

Debug info has been saved to /tmp/qiime2-q2cli-err-6m2v3t8l.log.

Metadata columns with only two options work, so if the Site column only has “Sandy” or “Silty” it will work but not with multiple different values (even though the different values each have more than one instance).

Thanks for the info about the metadata table setup!

[thermokarst]

Hi @jbethany - thanks for the followup - I am still pretty convinced this is related to some disparity with Sample IDs.

Would it be possible for you to provide some pieces of your data:

1. Your feature table Artifact, before filtering in QIIME 1
2. Your feature table Artifact, after filtering in QIIME 1
3. The Bray Curtis Distance matrix you are using in your call to beta-group-significance

You could send those via a direct message to me if you don’t want to share them publicly on the forum.

Sorry this has caused you so much trouble - we will get to the bottom of it!!

[jbethany (Julie Bethany Rakes)]

Hi @thermokarst,

Table.qza is my original feature table.

table.qza (332.0 KB)

Table_Cyano_only_notaxa.qza is the updated feature table.

Table_Cyano_only_notaxa.qza (42.1 KB)

This is the Bray Curtis Distance matrix I'm calling.

bray_curtis_pcoa_results.qza (20.4 KB)

Thank you very much for all of your help!

[thermokarst]

@jbethany Ah ha! I think we are onto something here! I used provenance to figure out what even sampling depth you used when running core-metrics:

[note: this screenshot came from loading bray_curtis_pcoa_results.qza at https://view.qiime2.org and navigating to the "Provenance Tab"]

Then I generated a summarize viz for your filtered table, and plugged in that even sampling depth (32150):

[note: this screenshot came from running feature-table summarize]

As you can see, at this depth, you filtered out all but 4 of your samples: TI3, TI5, FF21, and TI4, which means the rarefied table used to compute the distance matrix only had 4 samples in it, instead of 36, like you expected.

This means that, effectively, the only metadata values that are actually being used are:

[note: this screenshot came from running metadata tabulate]

So the error you saw before makes sense, because you are attempting to perform a pairwise comparison (--pairwise) to all of the "groups" within the category Site, but two of the three "groups" (coldsiltfinal1 & hotsiltyinitial) only have a single value within them, so there is no way to compute the "within" distances (there is nothing to compare to!)

So specifically, you can drop the --pairwise parameter, and then it should work for you, but, I suggest that you take a step back and rethink your rarefaction depth, since that is some pretty aggressive filtering, and was responsible for some unexpected behavior here.

Hope that helps! Thanks!

[jbethany (Julie Bethany Rakes)]

Thank you very much!!! I’ll adjust my rarefaction depth and rerun!