" Feature IDs found in the table are missing in the metadata" but I have those features in metadata...

[delibasanta (Delia)]

Hi everyone,

I am trying to group samples by metadata category by creating an average community with the next two lines:

qiime feature-table group --i-table rar5000_Taxafilt_table_Merged_200.qza --m-metadata-file Metadata_Hyliolas4.txt --m-metadata-column SiteName --p-axis sample --p-mode mean-ceiling --o-grouped-table SiteName_mean_rarefied_table_Merged_200.qza

qiime taxa barplot --i-table SiteName_mean_rarefied_table_Merged_200.qza --i-taxonomy Taxonomy_Merged_200.qza --m-metadata-file Metadata_Hyliolas4.txt --output-dir SiteName_Taxonomy_Plots_200 --verbose

But in this lat I had an error:
File "/home/delibasanta/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/q2cli/commands.py", line 327, in call
results = action(**arguments)
File "</home/delibasanta/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/decorator.py:decorator-gen-144>", line 2, in barplot
File "/home/delibasanta/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/sdk/action.py", line 240, in bound_callable
output_types, provenance)
File "/home/delibasanta/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/sdk/action.py", line 445, in callable_executor
ret_val = self._callable(output_dir=temp_dir, **view_args)
File "/home/delibasanta/miniconda/envs/qiime2-2019.7/lib/python3.6/site-packages/q2_taxa/_visualizer.py", line 29, in barplot
raise ValueError('Feature IDs found in the table are missing in the '
ValueError: Feature IDs found in the table are missing in the metadata: {'Venado', 'Sauzal', 'Grulla', 'Rosario', 'Catavina'}.
Plugin error from taxa:
Feature IDs found in the table are missing in the metadata: {'Venado', 'Sauzal', 'Grulla', 'Rosario', 'Catavina'}.

I checked my metadata, and the column "SiteName" has the features Venado, Sauzal, Grulla, Rosario, Catavina that mentioned the error, also it doesn't have empty cells. I don't know what is the problem. Could you help me please?

Thank you very much,

Delia

[thermokarst]

Hey there @delibasanta!

@delibasanta

Could you help me please?

Of course!

By "grouping" your feature table, you are changing your sample IDs to be something new (‘Venado’, ‘Sauzal’, ‘Grulla’, ‘Rosario’, ‘Catavina’). You will need to create a new sample metadata file, because technically these are new "samples". I suggest running feature-table summarize on the grouped table to see what happened to your sample IDs.

[SetaPark]

Hello
I am stuck exactly here. I understand what you are saying, but not sure where would the sample ID be changed? i.e. in which column?
Thanks.

[SetaPark]

Hello again

I have added a bit more detail. I first did this.
qiime feature-table group
--i-table $FILTER/table-dada2.qza
--p-axis sample
--m-metadata-file $WORKING/sample_metadata.tsv
--m-metadata-column Substrate
--p-mode mean-ceiling
--o-grouped-table $FILTER/grouped-mean-table-dada2.qza ;
qiime feature-table summarize
--i-table $FILTER/grouped-mean-table-dada2.qza
--o-visualization $TAXONOMY/All-tax/grouped-mean-table-dada2.qzv

Then did this. using metadatasample_metadata.tsv (1.3 KB)
qiime taxa barplot
--i-table $FILTER/grouped-mean-table-dada2.qza
--i-taxonomy $TAXONOMY/All-tax/taxonomy_gg0.7.qza
--m-metadata-file $WORKING/sample_metadata.tsv
--o-visualization $TAXONOMY/All-tax/grouped-taxa-bar-plots_gg0.7.qzv

Got the error:
Feature IDs found in the table are missing in the metadata: {'Water', 'Inulin GR', 'Hayward', 'FS Pooled', 'Gold3'}

So regrouped the metadata to make a mock one like so gp_sample_metadata.tsv (1.3 KB)

then got the error
Metadata IDs must be unique. The following IDs are duplicated: 'FS Pooled', 'Gold3', 'Hayward', 'Inulin GR', 'Water'

I know this is simple. What am I not doing right in the metadata? Thanks very much.
Regards
SetaPark

[thermokarst]

Hey @SetaPark! Thanks for sharing your revised Metadata file. That won't work, because you are grouping multiple samples into new "pooled" samples. Once grouped, you will most likely lose any identifying information, since, each sample must have a unique ID (which means it is not possible to disambiguate more than one sample in the group).

Maybe instead of "grouping" you should focus on "sorting" the taxa barplots, which will make "psuedo-groups" for you.

[SetaPark]

Hi @thermokarst
Thanks for your reply.
I am able to group the barplots by multiple categories

. What I want to do is create a plot of the average community for each grouping, as mentioned in the topic of this email thread.
So I want the plot to look like drawing.

Regards
S

[Nicholas_Bokulich]

@SetaPark,
You are doing the right thing with feature-table group but you are not grouping your sample metadata file correctly. As the error indicates, each sample ID must be unique, but all you have done is relabel the sample IDs of the original samples to match the group that they are being lumped into.

You need to trim out duplicated lines from your sample metadata file so that each group is represented by a single line, and only group-wide metadata values are represented. So instead of this:

#SampleID	ExpR10677 ID	Project Code	DNA Ex ID	Experimental Sample ID	Substrate	Material Type	Time Point h	Sample Name
FS Pooled	S203	P-300801-20	KFSL_D001	KFSL063	FS Pooled	Faecal Slurry	0	FS Pooled
FS Pooled	S204	P-300801-20	KFSL_D002	KFSL063	FS Pooled	Faecal Slurry	0	FS Pooled
FS Pooled	S205	P-300801-20	KFSL_D003	KFSL063	FS Pooled	Faecal Slurry	0	FS Pooled

You probably want something like this:

#SampleID	Project Code	Experimental Sample ID	Substrate	Material Type	Time Point h	Sample Name
FS Pooled	P-300801-20	KFSL063	FS Pooled	Faecal Slurry	0	FS Pooled

and likewise for the remaining duplicated lines in your modified sample metadata file.

[SetaPark]

Thanks very much @Nicholas_Bokulich. That worked. I get it now. So you condense your metadata essentially.
For people who get stuck here, just to let u know the metadata now should look like this

Thanks very much for your help.
Regards
S