ANCOMBC difficulty with column headings in metadata

I'm trying to run ANCOMBC using Qiime version 2023.7 in a conda environment, but get error messages that suggest it is only reading the first word in my metadata column headings. It does work with single-word column headings like "batch", but not with multiple word headings like "stem-height-cm". As far as I can tell a heading like this doesn't violate metadata requirements, so I'm not sure what is causing the problem.
Here is the command I used, plus the error message:

qiime composition ancombc
--i-table /home/aaron/TrapCultures/2022/NGS/Trap2022-16S-filtered-table.qza
--m-metadata-file /home/aaron/TrapCultures/2022/NGS/Trap2022-metadata.csv
--p-formula stem-height-cm
--o-differentials /home/aaron/TrapCultures/2022/NGS/Trap2022-16S-ancombc-stem-height-cm.qza

Plugin error from composition:
'stem' is not a column in the metadata. Available columns: 'batch', 'plate', 'row', 'column', '#plate-row-column', 'sample-name', 'informative-name', 'lyophilized-days', 'age-of-stand-years', 'years-since-dieback', 'material', 'depth-upper-cm', 'depth-lower-cm', 'material-amount-mg', 'shipped-volume-ul', 'nanodrop-ng-ul', 'nanodrop-260-280', 'qubit3.0-ng-ul', 'DNA-concentration-ng-mg', 'qubit2.0-ng-ul', 'percent-water', 'plant-genotype', 'plant-health', '#stand-notes', 'plant-collection-date', 'plant-GPS-source', 'soil-health', 'soil-treatment', 'soil-GPS-source', 'soil-ratio', 'soil-collection-date', 'potting-date', '0month-date', 'NDVI-0month', '0.5month-date', 'NDVI-0.5month', '1month-date', 'NDVI-1month', '1.5month-date', 'NDVI-1.5month', '2month-date', 'NDVI-2month', '6month-date', 'NDVI-6month', 'stem-height-cm', 'stem-number', 'stem-nodes-total', 'stem-green-nodes', 'stem-branches', 'stem-DWB', 'root-DWB-inside', 'root-DWB-outside', 'root-DWB-total', 'rhizome-DWB-inside', 'heat-units-degree.days', 'growing-days', 'time-months', 'replicate'
Debug info has been saved to /tmp/qiime2-q2cli-err-1kqmeugu.log

I also ran it with the "--verbose" flag:

Traceback (most recent call last):
File "/home/aaron/miniconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3802, in get_loc
return self._engine.get_loc(casted_key)
File "pandas/_libs/index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 165, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 5745, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 5753, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'stem'

The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/aaron/miniconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/qiime2/metadata/metadata.py", line 637, in get_column
series = self._dataframe[name]
File "/home/aaron/miniconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/pandas/core/frame.py", line 3807, in getitem
indexer = self.columns.get_loc(key)
File "/home/aaron/miniconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3804, in get_loc
raise KeyError(key) from err
KeyError: 'stem'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/aaron/miniconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/q2cli/commands.py", line 478, in call
results = self._execute_action(
File "/home/aaron/miniconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/q2cli/commands.py", line 539, in _execute_action
results = action(**arguments)
File "", line 2, in ancombc
File "/home/aaron/miniconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/qiime2/sdk/action.py", line 342, in bound_callable
outputs = self.callable_executor(
File "/home/aaron/miniconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/qiime2/sdk/action.py", line 566, in callable_executor
output_views = self._callable(**view_args)
File "/home/aaron/miniconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/q2_composition/_ancombc.py", line 41, in ancombc
return _ancombc(
File "/home/aaron/miniconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/q2_composition/_ancombc.py", line 121, in _ancombc
metadata.get_column(term)
File "/home/aaron/miniconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/qiime2/metadata/metadata.py", line 640, in get_column
raise ValueError(
ValueError: 'stem' is not a column in the metadata. Available columns: 'batch', 'plate', 'row', 'column', '#plate-row-column', 'sample-name', 'informative-name', 'lyophilized-days', 'age-of-stand-years', 'years-since-dieback', 'material', 'depth-upper-cm', 'depth-lower-cm', 'material-amount-mg', 'shipped-volume-ul', 'nanodrop-ng-ul', 'nanodrop-260-280', 'qubit3.0-ng-ul', 'DNA-concentration-ng-mg', 'qubit2.0-ng-ul', 'percent-water', 'plant-genotype', 'plant-health', '#stand-notes', 'plant-collection-date', 'plant-GPS-source', 'soil-health', 'soil-treatment', 'soil-GPS-source', 'soil-ratio', 'soil-collection-date', 'potting-date', '0month-date', 'NDVI-0month', '0.5month-date', 'NDVI-0.5month', '1month-date', 'NDVI-1month', '1.5month-date', 'NDVI-1.5month', '2month-date', 'NDVI-2month', '6month-date', 'NDVI-6month', 'stem-height-cm', 'stem-number', 'stem-nodes-total', 'stem-green-nodes', 'stem-branches', 'stem-DWB', 'root-DWB-inside', 'root-DWB-outside', 'root-DWB-total', 'rhizome-DWB-inside', 'heat-units-degree.days', 'growing-days', 'time-months', 'replicate'

Hello @Aaron_DeVries,

What happens when you put stem-height-cm in quotes: 'stem-height-cm'?

Hello @Aaron_DeVries,

After reading a little it looks like this may actually be a bug, and the hyphens are being interpreted as part of the formula. Removing the hyphens from the metadata columns should be a quick fix but I understand that that'll be a hassle. We'll get back to you once we've looked into this further.

Hello @Aaron_DeVries,

Turns out this is a known issue and there is little we can do about it at the moment because changes are needed in the underlying ancombc software which we are only borrowing. Your best bet is going to be to remove the hyphens or replace them with an underscore.

All right, thanks for the update. I'll try the underscores instead.

1 Like