I receive error with command: qiime composition ancom

User Support
1

All the analysis in the previous steps as per the moving picture tutorial was perfect. But qiime composition ancom gave errors all the time.

--i-table comp-area-table.qza \

--m-metadata-file sample-metadata-21-june-2023.tsv \

--m-metadata-column area \

--o-visualization ancom-area-table.qzv

--verbose

Plugin error from composition:

ids_to_keep must contain at least one ID.

Debug info has been saved to /var/folders/d2/68pmzfrx1szc6n15w73bfjjh0000gn/T/qiime2-q2cli-err-03epk47h.log

zsh: command not found: --verbose

3

Hi @Bhagwan,

It looks like you're missing a back slash after your --o-visualization line, which is why the --verbose command wasn't recognized. Can you please re-run your command with --verbose once more so we can take a look at the error log? Additionally, it would be helpful to look at your input table if you don't mind sharing that (either in this thread or in a DM is fine). Thanks!

4

Hi Lizgehret,
I am getting similar errors. I also have a back slash and a forward slash too. DM sent with more details.

--i-table comp-vegetation-table.qza \

--m-metadata-file sample-metadata-21-june-2023.tsv \

--m-metadata-column vegetation \

--o-visualization \ ancom-vegetation.qzv

--verbose

Plugin error from composition:

ids_to_keep must contain at least one ID.

Debug info has been saved to /var/folders/d2/68pmzfrx1szc6n15w73bfjjh0000gn/T/qiime2-q2cli-err-cl4g050m.log

5

Hi @Bhagwan,

The \ should be at the end of this line:

--o-visualization ancom-vegetation.qzv \
1 Like
6

Dear Greg and Liz,
I tried after changing backward slash and id's too. selected columns are categorial. Please go through the following: Error is continued ###

###1

qiime composition ancom
--i-table comp-vegetation-table.qza
--m-metadata-file sample-metadata-21-june-2023.tsv
--m-metadata-column vegetation
--o-visualization ancom-vegetation.qzv
--verbose
Traceback (most recent call last):
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/commands.py", line 468, in call
results = action(**arguments)
File "", line 2, in ancom
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/sdk/action.py", line 274, in bound_callable
outputs = self.callable_executor(
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/sdk/action.py", line 558, in callable_executor
ret_val = self._callable(output_dir=temp_dir, **view_args)
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/q2_composition/_ancom.py", line 47, in ancom
metadata = metadata.filter_ids(table.index)
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/metadata/metadata.py", line 1244, in filter_ids
filtered_series = self._filter_ids_helper(self._series, self.get_ids(),
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/metadata/metadata.py", line 188, in _filter_ids_helper
raise ValueError("ids_to_keep must contain at least one ID.")
ValueError: ids_to_keep must contain at least one ID.

Plugin error from composition:

ids_to_keep must contain at least one ID.

See above for debug info.

####2

qiime composition ancom
--i-table comp-vegetation-table.qza
--m-metadata-file sample-metadata-21-june-2023.tsv
--m-metadata-column vegetation
--o-visualization ancom-vegetation.qzv
Plugin error from composition:

ids_to_keep must contain at least one ID.

Debug info has been saved to /var/folders/d2/68pmzfrx1szc6n15w73bfjjh0000gn/T/qiime2-q2cli-err-8q9azkvk.log
(qiime2-2023.5) bhagwanrekadwad@Bhagwans-MacBook-Pro ANCOM-26-june-2023 % qiime composition ancom --i-table comp-vegetation-table.qza --m-metadata-file sample-metadata-21-june-2023.tsv --m-metadata-column vegetation --o-visualization ancom-vegetation.qzv --verbose
Traceback (most recent call last):
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/commands.py", line 468, in call
results = action(**arguments)
File "", line 2, in ancom
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/sdk/action.py", line 274, in bound_callable
outputs = self.callable_executor(
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/sdk/action.py", line 558, in callable_executor
ret_val = self._callable(output_dir=temp_dir, **view_args)
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/q2_composition/_ancom.py", line 47, in ancom
metadata = metadata.filter_ids(table.index)
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/metadata/metadata.py", line 1244, in filter_ids
filtered_series = self._filter_ids_helper(self._series, self.get_ids(),
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/metadata/metadata.py", line 188, in _filter_ids_helper
raise ValueError("ids_to_keep must contain at least one ID.")
ValueError: ids_to_keep must contain at least one ID.

Plugin error from composition:

ids_to_keep must contain at least one ID.

See above for debug info.

###3

qiime composition ancom --i-table comp-vegetation-table.qza --m-metadata-file sample-metadata-21-june-2023.tsv --m-metadata-column vegetation --o-visualization ancom-vegetation.qzv
Plugin error from composition:

ids_to_keep must contain at least one ID.

Debug info has been saved to /var/folders/d2/68pmzfrx1szc6n15w73bfjjh0000gn/T/qiime2-q2cli-err-xrhz3vy9.log

8

Hi @Bhagwan,

Thanks for sharing that output! Here is where I think the error is actually coming from:

In ancom, the metadata is filtered from the associated IDs found in your feature table. What raise ValueError("ids_to_keep must contain at least one ID.") tells me is that when your metadata is being filtered from the feature table, there are no remaining IDs (i.e. ids_to_keep). I suspect there may be either mismatched data in your table vs. your metadata file, or your table may be empty. I would take a look at your feature table alongside your metadata and ensure that both have overlapping IDs, and that your table isn't empty for any reason.

Cheers :lizard:

10

@lizgehret

Hi Liz,
I've shared some files via DM. Please go through it and suggest needful.

Bhagwan

11

@gregcaporaso @lizgehret

Dear Greg,
I tried to re-run the data successfully analyzed in the QIIM2 2019 version with every possible combination of commands. A sample metadata file is revalidated by using Keemai for QIIME2. I think all inputs are alright from the end, if I am not missing anything.
The error still persists.

(qiime2-2023.5) bhagwanrekadwad@Bhagwans-MacBook-Pro postdoc_data_ana % qiime composition ancom
--i-table comp-area-table.qza
--m-metadata-file sample-metadata-28-june-2023.tsv
--m-metadata-column area
--o-visualization ancom-area-table.qzv
Plugin error from composition:

ids_to_keep must contain at least one ID.

Debug info has been saved to /var/folders/d2/68pmzfrx1szc6n15w73bfjjh0000gn/T/qiime2-q2cli-err-dkpua2i1.log

(qiime2-2023.5) bhagwanrekadwad@Bhagwans-MacBook-Pro postdoc_data_ana % qiime composition ancom
--i-table comp-area-table.qza
--m-metadata-file sample-metadata-28-june-2023.tsv
--m-metadata-column area
--o-visualization ancom-area-table.qzv \

Plugin error from composition:

ids_to_keep must contain at least one ID.

Debug info has been saved to /var/folders/d2/68pmzfrx1szc6n15w73bfjjh0000gn/T/qiime2-q2cli-err-i8gm_r0i.log

(qiime2-2023.5) bhagwanrekadwad@Bhagwans-MacBook-Pro postdoc_data_ana % qiime composition ancom
--i-table comp-area-table.qza
--m-metadata-file sample-metadata-28-june-2023.tsv
--m-metadata-column area
--o-visualization \ ancom-area-table.qzv \

Plugin error from composition:

ids_to_keep must contain at least one ID.

Debug info has been saved to /var/folders/d2/68pmzfrx1szc6n15w73bfjjh0000gn/T/qiime2-q2cli-err-eykaz_nt.log

(qiime2-2023.5) bhagwanrekadwad@Bhagwans-MacBook-Pro postdoc_data_ana % qiime composition ancom
--i-table comp-area-table.qza
--m-metadata-file sample-metadata-28-june-2023.tsv
--m-metadata-column area
--o-visualization \ ancom-area-table.qzv
--verbose
Traceback (most recent call last):
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/commands.py", line 468, in call
results = action(**arguments)
File "", line 2, in ancom
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/sdk/action.py", line 274, in bound_callable
outputs = self.callable_executor(
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/sdk/action.py", line 558, in callable_executor
ret_val = self._callable(output_dir=temp_dir, **view_args)
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/q2_composition/_ancom.py", line 47, in ancom
metadata = metadata.filter_ids(table.index)
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/metadata/metadata.py", line 1244, in filter_ids
filtered_series = self._filter_ids_helper(self._series, self.get_ids(),
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/metadata/metadata.py", line 188, in _filter_ids_helper
raise ValueError("ids_to_keep must contain at least one ID.")
ValueError: ids_to_keep must contain at least one ID.

Plugin error from composition:

ids_to_keep must contain at least one ID.

See above for debug info.

(qiime2-2023.5) bhagwanrekadwad@Bhagwans-MacBook-Pro postdoc_data_ana % qiime composition ancom
--i-table comp-area-table.qza
--m-metadata-file sample-metadata-28-june-2023.tsv
--m-metadata-column area
--o-visualization ancom-area-table.qzv \
--verbose
Traceback (most recent call last):
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/commands.py", line 468, in call
results = action(**arguments)
File "", line 2, in ancom
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/sdk/action.py", line 274, in bound_callable
outputs = self.callable_executor(
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/sdk/action.py", line 558, in callable_executor
ret_val = self._callable(output_dir=temp_dir, **view_args)
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/q2_composition/_ancom.py", line 47, in ancom
metadata = metadata.filter_ids(table.index)
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/metadata/metadata.py", line 1244, in filter_ids
filtered_series = self._filter_ids_helper(self._series, self.get_ids(),
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/metadata/metadata.py", line 188, in _filter_ids_helper
raise ValueError("ids_to_keep must contain at least one ID.")
ValueError: ids_to_keep must contain at least one ID.

Plugin error from composition:

ids_to_keep must contain at least one ID.

See above for debug info.

(qiime2-2023.5) bhagwanrekadwad@Bhagwans-MacBook-Pro postdoc_data_ana % qiime composition ancom
--i-table comp-area-table.qza
--m-metadata-file sample-metadata-28-june-2023.tsv
--m-metadata-column area
--o-visualization ancom-area-table.qzv
Plugin error from composition:

ids_to_keep must contain at least one ID.

Debug info has been saved to /var/folders/d2/68pmzfrx1szc6n15w73bfjjh0000gn/T/qiime2-q2cli-err-7u2847j7.log
(qiime2-2023.5) bhagwanrekadwad@Bhagwans-MacBook-Pro postdoc_data_ana % qiime composition ancom
--i-table comp-area-table.qza
--m-metadata-file sample-metadata-28-june-2023.tsv
--m-metadata-column area
--o-visualization \ ancom-area-table.qzv
Plugin error from composition:

ids_to_keep must contain at least one ID.

Debug info has been saved to /var/folders/d2/68pmzfrx1szc6n15w73bfjjh0000gn/T/qiime2-q2cli-err-oqfjiiqu.log

(qiime2-2023.5) bhagwanrekadwad@Bhagwans-MacBook-Pro postdoc_data_ana % qiime composition ancom
--i-table comp-area-table.qza
--m-metadata-file sample-metadata-28-june-2023.tsv
--m-metadata-column area
--o-visualization ancom-area-table.qzv \

Plugin error from composition:

ids_to_keep must contain at least one ID.

Debug info has been saved to /var/folders/d2/68pmzfrx1szc6n15w73bfjjh0000gn/T/qiime2-q2cli-err-8t_7uesb.log
(qiime2-2023.5) bhagwanrekadwad@Bhagwans-MacBook-Pro postdoc_data_ana % qiime composition ancom --i-table comp-area-table.qza --m-metadata-file sample-metadata-28-june-2023.tsv --m-metadata-column area --o-visualization ancom-area-table.qzv
Plugin error from composition:

ids_to_keep must contain at least one ID.

Debug info has been saved to /var/folders/d2/68pmzfrx1szc6n15w73bfjjh0000gn/T/qiime2-q2cli-err-zyb4odid.log
(qiime2-2023.5) bhagwanrekadwad@Bhagwans-MacBook-Pro postdoc_data_ana % qiime composition ancom --i-table comp-area-table.qza --m-metadata-file sample-metadata-28-june-2023.tsv --m-metadata-column area --o-visualization ancom-area-table.qzv \

Plugin error from composition:

ids_to_keep must contain at least one ID.

Debug info has been saved to /var/folders/d2/68pmzfrx1szc6n15w73bfjjh0000gn/T/qiime2-q2cli-err-6k7s16br.log

(qiime2-2023.5) bhagwanrekadwad@Bhagwans-MacBook-Pro postdoc_data_ana % qiime composition ancom --i-table comp-area-table.qza --m-metadata-file sample-metadata-28-june-2023.tsv --m-metadata-column area --o-visualization ancom-area-table.qzv --verbose
Traceback (most recent call last):
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/commands.py", line 468, in call
results = action(**arguments)
File "", line 2, in ancom
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/sdk/action.py", line 274, in bound_callable
outputs = self.callable_executor(
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/sdk/action.py", line 558, in callable_executor
ret_val = self._callable(output_dir=temp_dir, **view_args)
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/q2_composition/_ancom.py", line 47, in ancom
metadata = metadata.filter_ids(table.index)
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/metadata/metadata.py", line 1244, in filter_ids
filtered_series = self._filter_ids_helper(self._series, self.get_ids(),
File "/opt/homebrew/Caskroom/miniconda/base/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/metadata/metadata.py", line 188, in _filter_ids_helper
raise ValueError("ids_to_keep must contain at least one ID.")
ValueError: ids_to_keep must contain at least one ID.

Plugin error from composition:

ids_to_keep must contain at least one ID.

See above for debug info.
(qiime2-2023.5) bhagwanrekadwad@Bhagwans-MacBook-Pro postdoc_data_ana %

15

Hi @Bhagwan,

Thanks for sending those files over - I took a look at both of the tables you shared, and they are both empty. I took a quick look at the provenance for both tables in QIIME 2 View (i.e. what commands you ran on these artifacts) and I am fairly certain the issue is coming from this parameter within filter-samples:

where:"[sampleID]='vegetation'"

What you're saying with this command is to only retain samples where the sampleID column is equal to vegetation - which will never be true, since your sampleID column contains the IDs for all of your samples. You'll want to re-run this with something like '[vegetation]="yes"' which will filter your samples based on the metadata column 'vegetation' on all samples that have 'yes' in that column.

Hope this helps! Cheers :lizard:

1 Like
17

@lizgehret Dear Liz,
I tried to re-run these commands after redoing the QIIME2 analysis with the suggested changes.
'[vegetation]="yes"'

After trying a couple of times. The computing power used for analysis is a Macbook Pro M2 Pro with 16 GB of RAM.
I didn't get any output, and the command was running for more than 33 hours. Later, I terminated it. What will the reason be for it? How much time it will take for 66 samples having initial size of fastq file 4 gb in total.

19

@Bhagwan,

This is a reasonable amount of RAM.

There isn't a hard and fast answer for how long it should take, but that's much longer than I would expect. Even though your initial FASTQ size is a bit large, this command is running on the feature table - which should be much smaller than your initial sequencing file.

Do you mind sharing your feature table with me (either here or DM is fine) that you're using as input for filter-samples? I'd like to examine it and also try to replicate the issue on my machine. Thanks! :lizard:

22

Hi @Bhagwan,

Thanks for sharing your table and metadata file with me - I've run filter-samples and add-pseudocount on your table, and am currently running ancom. You do have quite a large table, and something to remember is that ancom is making comparisons across every entry in your table, so I do expect that it will take a while to run. I'll let you know if it completes within a reasonable amount of time, otherwise I might recommend running ancombc for differential abundance analysis - the compute time is much lower than with ancom, and with a larger table like yours, this might be a better way forward. Cheers :lizard: