ANCOM-BC plugin potential bug

Hello Q2 folks!

I have been attempting to use the ANCOM-BC plugin for differential analysis tutorial (here) and it appears there is a bug that I'm not sure how to fix. I am getting the same error as this post here. Here is the command I ran and the error:

qiime ancombc ancombc
--i-table ../04_tables/table_samplefilt_freq10filt_nochlomito_min11160_nopos.qza
--m-metadata-file ../../02_metadata/map_ancombc.txt
--p-formula "ADD_0"
--o-differentials differentials.qza
--verbose

Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_ancombc.R ./input.biom.tsv ./input.map.txt ADD_0 holm 0.9 1000 ADD_0 TRUE TRUE 1e-05 100 TRUE 0.05 FALSE ./output.summary.txt

R version 4.0.5 (2021-03-31)
── Attaching packages ─────────────────────────────────────── tidyverse 1.3.1 ──
:heavy_check_mark: ggplot2 3.3.5 :heavy_check_mark: purrr 0.3.4
:heavy_check_mark: tibble 3.1.6 :heavy_check_mark: dplyr 1.0.7
:heavy_check_mark: tidyr 1.1.4 :heavy_check_mark: stringr 1.4.0
:heavy_check_mark: readr 2.1.0 :heavy_check_mark: forcats 0.5.1
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
:heavy_multiplication_x: dplyr::filter() masks stats::filter()
:heavy_multiplication_x: dplyr::lag() masks stats::lag()
Traceback (most recent call last):
File "/Users/heatherdeel/opt/miniconda3/envs/qiime2-2021.8/lib/python3.8/site-packages/q2cli/commands.py", line 329, in call
results = action(**arguments)
File "", line 2, in ancombc
File "/Users/heatherdeel/opt/miniconda3/envs/qiime2-2021.8/lib/python3.8/site-packages/qiime2/sdk/action.py", line 245, in bound_callable
outputs = self.callable_executor(scope, callable_args,
File "/Users/heatherdeel/opt/miniconda3/envs/qiime2-2021.8/lib/python3.8/site-packages/qiime2/sdk/action.py", line 418, in callable_executor
artifact = qiime2.sdk.Artifact._from_view(
File "/Users/heatherdeel/opt/miniconda3/envs/qiime2-2021.8/lib/python3.8/site-packages/qiime2/sdk/result.py", line 305, in _from_view
result = transformation(view, validate_level)
File "/Users/heatherdeel/opt/miniconda3/envs/qiime2-2021.8/lib/python3.8/site-packages/qiime2/core/transform.py", line 70, in transformation
new_view = transformer(view)
File "/Users/heatherdeel/opt/miniconda3/envs/qiime2-2021.8/lib/python3.8/site-packages/qiime2/core/transform.py", line 221, in wrapped
file_view = transformer(view)
File "/Users/heatherdeel/opt/miniconda3/envs/qiime2-2021.8/lib/python3.8/site-packages/q2_types/feature_data/_transformer.py", line 645, in _224
qiime2.Metadata(data).save(str(ff))
File "/Users/heatherdeel/opt/miniconda3/envs/qiime2-2021.8/lib/python3.8/site-packages/qiime2/metadata/metadata.py", line 405, in init
super().init(dataframe.index)
File "/Users/heatherdeel/opt/miniconda3/envs/qiime2-2021.8/lib/python3.8/site-packages/qiime2/metadata/metadata.py", line 96, in init
self._assert_valid_id_header(id_header)
File "/Users/heatherdeel/opt/miniconda3/envs/qiime2-2021.8/lib/python3.8/site-packages/qiime2/metadata/metadata.py", line 134, in _assert_valid_id_header
raise ValueError(

ValueError: pandas index name (Index.name) must be one of the following values, not None:

Case-insensitive: 'feature id', 'feature-id', 'featureid', 'id', 'sample id', 'sample-id', 'sampleid'
Case-sensitive: '#OTU ID', '#OTUID', '#Sample ID', '#SampleID', 'sample_name'

Plugin error from ancombc:
pandas index name (Index.name) must be one of the following values, not None:
Case-insensitive: 'feature id', 'feature-id', 'featureid', 'id', 'sample id', 'sample-id', 'sampleid'
Case-sensitive: '#OTU ID', '#OTUID', '#Sample ID', '#SampleID', 'sample_name'
See above for debug info.

Here's what I think is happening:

  1. When I run the command, ancombc automatically generates an input.biom.tsv file
  2. The input.biom.tsv file, when generated, has nothing in the top left cell (screenshot shown here)
  3. This is causing the "None" error, and as far as I know there is not a way to change this because each time the ancombc command is run, a new input.biom.tsv file is generated and then automatically used in another command (e.g., Command: run_ancombc.R ./input.biom.tsv ./input.map.txt ADD_0 holm 0.9 1000 ADD_0 TRUE TRUE 1e-05 100 TRUE 0.05 FALSE ./output.summary.txt) that I can not manually change.

If there is a workaround for this that would be great, otherwise I could also give this a shot in R. Just wanted to point this out!

Thanks Q2 team for all you do,

Heather Deel

1 Like

I second Heather's sentiments - very thankful and would find the ANCOM-BC plugin very useful if we can figure out why that A1 cell becomes empty.

Thank you,
Nate

Hello @hdeel and @nathaniel_hubert ,

I do not have a solution — I am only writing to highlight the information in this post from @jwdebelius :

and to say that

  1. q2-ancombc appears to be still in development/incomplete, and ANCOM-BC is not developed or maintained by members of the Q2 team (otherwise we would be happy to help!)
  2. I do not believe that the ANCOM-BC developers monitor this forum, and there is already a similar issue reported in the q2-ancombc github repository.
  3. I do not think that the missing index in the biom table is actually the issue, because the error message appears to occur further downstream (i.e., after reading that table into R). I think the error might actually be indicating that the output file (list of features) is not formatted correctly.

You could try modifying the source code to uncomment this line:

A bit of a shot in the dark, but that's where I would begin.

Sorry that's not really an answer, just hoping to point you in the right direction and make it clear that I am not sure that this plugin is actually officially released yet :man_shrugging:

5 Likes

Thank you very much for the insight, Nicholas!
I can see from their github page that they are actively working on this plugin. I am hoping to use it for publication purposes, so probably best to wait until it is officially released. (Plus I wouldn't really know how to modify the script appropriately and then use it... yet).
Thanks again,
Nate

1 Like

Hello again, Nicholas,
I tried uncommenting (removing the "#") from within Line 116 of q2_ancombc/method.py, but it did not work...
Do I need to initiate the script somehow?
Thank you again,
Nate

After uncommenting line 116 and deleting the taxonomy command, I ran this command and successfully generated a differentials.qza file:

qiime ancombc ancombc
--i-table pct.qza
--m-metadata-file QIIME_map.txt
--p-formula "geno_diet_week"
--o-differentials differentials.qza

I then tried running the following, with and without the taxonomy input to no avail:

qiime metadata tabulate
--m-input-file differentials.qza \
--i-input-file taxonomy.qza
--o-visualization differentials.qzv

I get the following error:

There was an issue with loading the file as metadata:

Metadata file path doesn't exist, or the path points to something other than a file. Please check that the path exists, has read permissions, and points to a regular file (not a directory):

There may be more errors present in the metadata file. To get a full report, sample/feature metadata files can be validated with Keemei: https://keemei.qiime2.org

Find details on QIIME 2 metadata requirements here: Metadata in QIIME 2 — QIIME 2 2021.11.0 documentation

It looks like the tabulate command is looking for a mapping or metadata file, but I am unsure how to import the differentials.qza for this command.

I exported the differentials.qza file and there are results. I am not sure if they are reliable, and not 100% how to interpret. For example, I understand that q-values are FDR corrected p-values, just not sure which group is enriched for a given taxon. And I am wondering what the visualization output would look like.

I am very excited to use q2-ancombc and appreciate the effort to get it working. This will be an excellent addition to the QIIME workflow. Thank you!
Nate

1 Like