q2-metnet plugin error

Community Plugin Support
1

Dear All,

I am trying out the q2-metnet plugin and facing with the following error.

(qiime2-amplicon-2024.5) annalex@ANNPC:~/q2-metnet$ qiime metnet generateFeatures --i-frequency GMIA_table.qza --i-taxa GMIA_taxonomy.qza --p-selection AGORAv103 --p-level s --o-reactions ../output_reactions.qza --o-subsystems ../output_subsystems.qza --o-xmatrix ./output_X_matrix.qza --verbose
Traceback (most recent call last):
File "/home/annalex/miniconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/q2cli/commands.py", line 520, in call
results = self._execute_action(
File "/home/annalex/miniconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/q2cli/commands.py", line 581, in _execute_action
results = action(**arguments)
File "", line 2, in generateFeatures
File "/home/annalex/miniconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/sdk/action.py", line 342, in bound_callable
outputs = self.callable_executor(
File "/home/annalex/miniconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/sdk/action.py", line 576, in callable_executor
output_views = self._callable(**view_args)
File "/home/annalex/miniconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/metnet-2023.0.1-py3.9.egg/q2_metnet/_generateFeatures.py", line 97, in generateFeatures
PresentTaxa, newFrequency, Samples = _extractTaxaPresentAGREDA(frequency, taxa, stream, level)
File "/home/annalex/miniconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/metnet-2023.0.1-py3.9.egg/q2_metnet/_inputFiles.py", line 132, in _extractTaxaPresentAGREDA
return _contextTaxa(frequency, taxa, reference, level)
File "/home/annalex/miniconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/metnet-2023.0.1-py3.9.egg/q2_metnet/_inputFiles.py", line 36, in _contextTaxa
df_frequency['LINEAGE'] = [dictionary_otu_taxa for x in df_frequency.index]
File "/home/annalex/miniconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/metnet-2023.0.1-py3.9.egg/q2_metnet/_inputFiles.py", line 36, in
df_frequency['LINEAGE'] = [dictionary_otu_taxa for x in df_frequency.index]
KeyError: '34'

Plugin error from metnet:

'34'

Kindly advise on this.

Thanks.

3

Could you show us the input data you are using to launch q2-metnet?

1 Like
4

Hi @tblasco,

I am also testing this tool and I obtain the same error output. I am running this via qiime2-amplicon-2024.5.

This issue is that the code is written to process an ASV/OTU table that is already transposed. That is, a standard QIIME 2 table should have the samples as rows and ASVs as columns. Whereas the example test file and code expects the samples as columns and ASVs as rows. For example compare the outputs of any table from the QIIME 2 tutorials to your example test table by running the following command on your asv_table.qza example table from here and the dada2_table.qza from here:

qiime metadata tabulate \
    --m-input-file asv_table.qza \
   --o-visualization asv_table_tab.qzv

qiime metadata tabulate \
    --m-input-file dada2_table.qza \
   --o-visualization dada2_table_tab.qzv

That is the .transpose command is not necessary for a standard table. Otherwise the user must run the following prior to running the analysis:

 qiime feature-table transpose \
    --i-table ./dada2_table.qz  \
    --o-transposed-feature-table ./dada2_table_transposed.qza

Upon running this command I was able to get past the error that @annalex observed, and moved on to a new error:

Traceback (most recent call last):
  File "/home/SE/BMIG-6202-MSR/qiime2-amplicon-2024.5/lib/python3.9/site-packages/q2cli/commands.py", line 520, in __call__
    results = self._execute_action(
  File "/home/SE/BMIG-6202-MSR/qiime2-amplicon-2024.5/lib/python3.9/site-packages/q2cli/commands.py", line 581, in _execute_action
    results = action(**arguments)
  File "<decorator-gen-908>", line 2, in generateFeatures
  File "/home/SE/BMIG-6202-MSR/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/sdk/action.py", line 342, in bound_callable
    outputs = self._callable_executor_(
  File "/home/SE/BMIG-6202-MSR/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/sdk/action.py", line 576, in _callable_executor_
    output_views = self._callable(**view_args)
  File "/home/SE/BMIG-6202-MSR/qiime2-amplicon-2024.5/lib/python3.9/site-packages/metnet-2023.0.1-py3.9.egg/q2_metnet/_generateFeatures.py", line 82, in generateFeatures
    PresentTaxa, newFrequency, Samples = _extractTaxaPresentAGREDA(frequency, taxa, stream, level)
  File "/home/SE/BMIG-6202-MSR/qiime2-amplicon-2024.5/lib/python3.9/site-packages/metnet-2023.0.1-py3.9.egg/q2_metnet/_inputFiles.py", line 132, in _extractTaxaPresentAGREDA
    return _contextTaxa(frequency, taxa, reference, level)
  File "/home/SE/BMIG-6202-MSR/qiime2-amplicon-2024.5/lib/python3.9/site-packages/metnet-2023.0.1-py3.9.egg/q2_metnet/_inputFiles.py", line 46, in _contextTaxa
    if not "" == tmp[depth_level[level]]:
IndexError: list index out of range

Plugin error from metnet:

  list index out of range

See above for debug info.

I think this has to do with the fact that most users will have input taxonomy files that do not have a full taxonomy as does your test file. You can use RESCRIPt's edit-taxonomy command to help with this, see below. Or add code to q2-metnet to handle these cases.

I suggest that the q2-metnet test cases use files similar to those of the QIIME 2 tutorials for code testing, as this will be the typical format of most users. Or be very explicit, with detailed instructions, on how to format the input files for use in q2-metnet.

Also, looking at the q2-metnet code, I noticed that correspondent_level and depth_level dictionaries do not allow domain. This will be problematic for those that use GTDB, and SILVA. These databases use domain and not kingdom, so you might want to allow d as a possible entry, or convert d to k behind the scenes. Or suggest that the users use RESCRIPt to change d__ to k__ like so:

qiime rescript edit-taxonomy \
    --i-taxonomy ./taxonomy.qza \
    --p-search-strings 'd__' \
    --p-replacement-strings 'k__' \
    --o-edited-taxonomy ./taxonomy-edit.qza

You can apply --p-use-regex (and/or use an input file) for more complicated edits, see here.

I hope this helps @tblasco . :slight_smile:

2 Likes
5

@tblasco My input file is the table.qza output after the DADA2 pipeline without any transformations or changes. I think @SoilRotifer has explained it best.

1 Like
6

Hi,

Thank you very much for your comments @SoilRotifer @annalex . I will try to update the code as soon as possible. I will let you know through this post

2 Likes
7

Hi,

We have made the following changes based on your comments to avoid the errors of the 'generateFeatures' method.

  • Apply only the .transpose command when is necessary.
  • Edit the FeatureData to contain all the taxonomic levels.
  • For the domain option. We describe at the README.md that the parameter 'level' of the 'generateFeatures' method can be defined as 'k' for the analysis at both kingdom or domain level. We don´t find necessary to make changes in the code.

With all these changes I was able to avoid the errors using as input data the 'dada2_table.qza' and 'taxonomy.qza' from here.

3 Likes
8

Thank you for doing this @tblasco! I'm looking forward to trying this out. :raised_hands:

I'll let you know if I run into any issues. :computer:

1 Like
9

Hi @tblasco,

Some updates....

Using your test files I am unable to run this command as outlined on your github README:

First error:

qiime metnet plotBoxplot \
	--i-table ./subs_scores.qza \
	--i-differentialresults ./diff_subs.qza \
	--m-sample-metadata-file ./test/meta.tsv \
	--m-sample-metadata-column Condition \
	--p-condition-name Celiac \
	--p-control-name Lean \
	--p-namefeature "S142 | Bile acid metabolism" \
	--o-visualization ./sub_boxplot.qzv

I obtain this error:

Plugin error from metnet:

  The ID for reaction/subsystem or for the samples are not present in the data

Debug info has been saved to /var/folders/g3/nx9hvncx72bbf5j8505qt6qjwfqn63/T/qiime2-q2cli-err-h_g7mloc.log

So, I tabulated the sub_scores.qza file and made a sub_scores.qzv file. There was no "S142 | Bile acid metabolism", but there was a "S168 | Bile acid metabolism" feature. I tried to rerun the above command with this feature, and still obtained the same error.

I also noticed that the tutorial does not provide an example to run a command like this:

qiime metadata tabulate \
    --m-input-file diff_subs.qza \
    --o-visualization diff_subs.qzv

So, that the user can figure out what to provide for --p-namefeature .... Or provide something similar.

Second error:

On my own data I was able to run qiime metnet generateFeatures ... but when I run the commands:

qiime metnet plotPCA \
    --i-table ./metnet-reactions.qza \
    --m-sample-metadata-file metadata.tsv \
    --m-sample-metadata-column Diet \
    --o-visualization ./metnet-rxns-diet-pca.qzv

qiime metnet plotClusteMap \
    --i-table ./metnet-reactions.qza \
    --m-sample-metadata-file $metadata \
    --m-sample-metadata-column Diet \
    --o-visualization ./metnet-rxns-dietduration-hclust.qzv

and I get this error:

Plugin error from metnet:

  "['RH-8b', 'RH-20', 'RH-21', 'RH-22', 'RH-23', 'RH-24', 'RH-74b', 'RH-80b'] not in index"

Debug info has been saved to /home/mrobeson/tmp/qiime2-q2cli-err-umdaisrf.log

I realized that this is because I had removed some samples earlier in my pipeline, but the information for these sample were still in my metadata file. Once I removed these entries from my metadata file these commands worked. :fireworks:

I'd recommend handling this within your code. That is, the code should ignore any extra entries in the metadata file. Or at least let users know that the metadata file must match the data being analyzed. In QIIME 2, it is typical for most data types to be a subset of the metadata file. This way users are not creating a separate metadata file for every analysis they plan to do. I think this is what users will tend to expect.

-Cheers!

1 Like
10

Hi @SoilRotifer,

We have made several changes in the code and README file.

First, functions plotBoxplot, plotClusteMap and 'plotPCA` do not consider the extra rows in the metadata.
We have solved the problems related to the "S142 | Bile acid metabolism" feature. It has been changed by "S144 | Bile acid metabolism" and now it works. This information has been also updated in the README.
Finally, we have added to the README the indicated instructions to generate visualization objects here.

Thanks for your helpful comments.

2 Likes
11

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.