Aldex2 Error: Not Recognizing Condition

I am really happy you are getting Aldex2 up and running @dgiguer

The results indicate that I don't have a condition defined and the --help vs. the tutorial files are not lining up. There is a divergence between the --help commands vs. the library.qiime2.org tutorial. Here is my code and error (not recognizing the category)

(qiime2-2019.7) pmrubine-u01dx:Gneiss kristinaadmin$ qiime aldex2 aldex2 --i-table filteredtable.qza --m-metadata-file DVSMetadata.txt --m-metadata-column Combined --p-test glm --p-denom iqlr --output-dir FeedAldex2 --verbose

Traceback (most recent call last):

File "/Users/kristinaadmin/miniconda2/envs/qiime2-2019.7/lib/python3.6/site-packages/q2cli/commands.py", line 327, in call

results = action(**arguments)

TypeError: aldex2() missing 1 required positional argument: 'condition'

Plugin error from aldex2:

aldex2() missing 1 required positional argument: 'condition'

In the --help file, I did just those directions. Using the tutorial...I get...(feed=combined)

See above for debug info.
(qiime2-2019.7) pmrubine-u01dx:Gneiss kristinaadmin$ qiime aldex2 aldex2 --i-table filteredtable.qza --m-metadata-file DVSMetadata.txt --p-condition Feed --output-dir FeedAldex2 --verbose
Usage: qiime aldex2 aldex2 [OPTIONS]

Performs log-ratio transformation and statistical testing

Inputs:
--i-table ARTIFACT FeatureTable[Frequency]
The feature table of abundances [required]
Parameters:
--m-metadata-file METADATA
--m-metadata-column COLUMN MetadataColumn[Categorical]
The "condition": this column will be used as an
experimental descriptor to group samples [required]
--p-mc-samples INTEGER The number of monte carlo samples to be used
[default: 128]
--p-test TEXT The statistical test to run
Choices('t', 'glm') [default: 't']
--p-denom TEXT Choices('all', 'iqlr')
The features used to decide a reference frame
[default: 'all']
Outputs:
--o-differentials ARTIFACT FeatureData[Differential]
The estimated per-feature differentials [required]
Miscellaneous:
--output-dir PATH Output unspecified results to a directory
--verbose / --quiet Display verbose output to stdout and/or stderr
during execution of this action. Or silence output
if execution is successful (silence is golden).
--citations Show citations and exit.
--help Show this message and exit.

                There was a problem with the command:                     

(1/1?) no such option: --p-condition
(qiime2-2019.7) pmrubine-u01dx:Gneiss kristinaadmin$

See above for debug info.

So, there is a disparity and I don't know how to resolve this as the github documentation is not great either (same error). I think this is an amazing plugin and I look forward to your feedback.

1 Like

Hi @Kristina_Feye! Not the developer of q2-aldex2, but just curious, what version of q2-aldex2 are you running?

conda list q2-aldex2

On my my machine, I have q2-aldex2 0.1.0. The tutorial fails for me, but in basically the opposite manner:

qiime aldex2 aldex2 \
    --i-table gut-table.qza \
    --m-metadata-file sample-metadata.tsv \
    --m-metadata-column subject \
    --output-dir gut-test

Usage: qiime aldex2 aldex2 [OPTIONS]

  Performs log-ratio transformation and statistical testing

Inputs:
  --i-table ARTIFACT FeatureTable[Frequency]
                          The feature table of abundances.          [required]
Parameters:
  --m-metadata-file METADATA...
    (multiple arguments   Sample metadata
     will be merged)                                                [required]
  --p-condition TEXT      Experimental descriptors to group samples [required]
  --p-mc-samples INTEGER  The number of monte carlo samples to be used
                                                                [default: 128]
  --p-test TEXT           The statistical test to run, options include `t`,
                          or `glm`                              [default: 't']
  --p-denom TEXT          The features used to decide a reference frame.
                                                              [default: 'all']
Outputs:
  --o-differentials ARTIFACT FeatureData[Differential]
                          The estimated per-feature differentials.  [required]
Miscellaneous:
  --output-dir PATH       Output unspecified results to a directory
  --verbose / --quiet     Display verbose output to stdout and/or stderr
                          during execution of this action. Or silence output
                          if execution is successful (silence is golden).
  --citations             Show citations and exit.
  --help                  Show this message and exit.

                    There was a problem with the command:                     
 (1/1?) no such option: --m-metadata-column

Either way, it looks like @dgiguer might need a few days to get things sorted out so that the QIIME 2 plugin matches the aldex2 API, and that the Library Tutorial uses the right version, too. I suspect that @dgiguer will need to release a new conda build for q2-aldex2, but not sure about that.

3 Likes

I just downloaded it today. Everything is up to date. It checked out in r too

1 Like

And thanks for the reply :slight_smile:

Unfortunately download date doesn't necessarily matter, sometimes conda will grab different versions of things. Please provide the information I requested above, that way we can assist you!

Hi @Kristina_Feye,

Thanks for posting, you are the first to catch a bug!

The parameter --p-condition was recently changed to --m-metadata-column (thanks @fedarko !) and I did not update the tutorial (or apparently the conda release) to reflect that. This has been corrected. If you remove the plugin and reinstall, it should now work using the correct --m-metadata-column parameter. I have also updated the tutorial to reflect this. The updated q2-aldex2 version for this bug fix on conda is 0.1.1.

Thanks for catching this and let me know if you encounter anymore issues! I will also be working on the to improve consistency with the Github wiki.

Cheers,

Dan

6 Likes

How do I directly remove a plugin? Also, I downloaded from your github site last week, I should probably remove that build as well.

Understood. I also tried to download Aldex2 from github a while back.

(qiime2-2019.7) pmrubine-u01dx:~ kristinaadmin$ qiime aldex2 --help
Usage: qiime aldex2 [OPTIONS] COMMAND [ARGS]...

Description: Analysis Of Differential Abundance Taking Sample Variation
Into Account

Plugin website: GitHub - ggloor/q2-aldex2: qiime2 plugin for aldex2 for differential abundance analysis

Getting user support: Please post to the QIIME 2 forum for help with this
plugin: https://forum.qiime2.org

Options:
--version Show the version and exit.
--citations Show citations and exit.
--help Show this message and exit.

Commands:
aldex2 Analysis Of Differential Abundance
effect-plot Effect plots
extract-differences Extract differentially expressed features
(qiime2-2019.7) pmrubine-u01dx:~ kristinaadmin$ qiime aldex2 --version
QIIME 2 Plugin 'aldex2' version 1.14.1 (from package 'q2-aldex2' version 0+untagged.31.g6a06bdc)
(qiime2-2019.7) pmrubine-u01dx:~ kristinaadmin$

It looks like it is version 1.14.1 that I was running. I have not updated the plugin.

1 Like

Awesome, thank you! I will try this during the weekend. I am really excited ya'll have this ready to rock! I've appreciated ALDEX2 for a while.

1 Like

Hi @Kristina_Feye,

You can remove it using conda remove -n qiime2-2019.7 q2-aldex2 (assuming your environment name is qiime2-2019.7), then reinstall using conda install -c dgiguere q2-aldex2. This should work!

Cheers,

Dan

1 Like

I don't need to update this in R as well, do I?

Thank you!

1 Like

It doesn't require the R package to be updated, but it is always a good idea to make sure you have the most recent version installed!

Cheers,

Dan

Hello, we have a new error...sorry to report...

(qiime2-2019.7) pmrubine-u01dx:DVDataKMF kristinaadmin$ qiime aldex2 effect-plot --i-table differentials.qza --o-visualization plotdifferencs.qzv
Plugin error from aldex2:

'diff.win'

Debug info has been saved to /var/folders/q2/2g0__ts95_qffw_fphpj19fc0000gr/T/qiime2-q2cli-err-tht75bpy.log
(qiime2-2019.7) pmrubine-u01dx:DVDataKMF kristinaadmin$ qiime aldex2 effect-plot --i-table differentials.qza --o-visualization plotdifferencs.qzv --verbose
Traceback (most recent call last):
File "/Users/kristinaadmin/miniconda2/envs/qiime2-2019.7/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 2657, in get_loc
return self._engine.get_loc(key)
File "pandas/_libs/index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 132, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1601, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1608, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'diff.win'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/kristinaadmin/miniconda2/envs/qiime2-2019.7/lib/python3.6/site-packages/q2cli/commands.py", line 327, in call
results = action(**arguments)
File "</Users/kristinaadmin/miniconda2/envs/qiime2-2019.7/lib/python3.6/site-packages/decorator.py:decorator-gen-498>", line 2, in effect_plot
File "/Users/kristinaadmin/miniconda2/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/sdk/action.py", line 240, in bound_callable
output_types, provenance)
File "/Users/kristinaadmin/miniconda2/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/sdk/action.py", line 445, in callable_executor
ret_val = self._callable(output_dir=temp_dir, **view_args)
File "/Users/kristinaadmin/miniconda2/envs/qiime2-2019.7/lib/python3.6/site-packages/q2_aldex2/_visualizer.py", line 32, in effect_plot
btw_max = table['diff.win'].max()
File "/Users/kristinaadmin/miniconda2/envs/qiime2-2019.7/lib/python3.6/site-packages/pandas/core/frame.py", line 2927, in getitem
indexer = self.columns.get_loc(key)
File "/Users/kristinaadmin/miniconda2/envs/qiime2-2019.7/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 2659, in get_loc
return self._engine.get_loc(self._maybe_cast_indexer(key))
File "pandas/_libs/index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 132, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1601, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1608, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'diff.win'

Plugin error from aldex2:

'diff.win'

See above for debug info.
(qiime2-2019.7) pmrubine-u01dx:DVDataKMF kristinaadmin$

Additionally, this error occurs when we are looking for significant differences.

(qiime2-2019.7) pmrubine-u01dx:DVDataKMF kristinaadmin$ qiime aldex2 extract-differences --i-table differentials.qza --o-differentials signdiff.qza --p-sig-threshold 0.1 --p-effect-threshold 0 --p-difference-threshold 0 --verbose
Traceback (most recent call last):
File "/Users/kristinaadmin/miniconda2/envs/qiime2-2019.7/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 2657, in get_loc
return self._engine.get_loc(key)
File "pandas/_libs/index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 132, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1601, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1608, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'we.eBH'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/kristinaadmin/miniconda2/envs/qiime2-2019.7/lib/python3.6/site-packages/q2cli/commands.py", line 327, in call
results = action(**arguments)
File "</Users/kristinaadmin/miniconda2/envs/qiime2-2019.7/lib/python3.6/site-packages/decorator.py:decorator-gen-495>", line 2, in extract_differences
File "/Users/kristinaadmin/miniconda2/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/sdk/action.py", line 240, in bound_callable
output_types, provenance)
File "/Users/kristinaadmin/miniconda2/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/sdk/action.py", line 383, in callable_executor
output_views = self._callable(**view_args)
File "/Users/kristinaadmin/miniconda2/envs/qiime2-2019.7/lib/python3.6/site-packages/q2_aldex2/_method.py", line 80, in extract_differences
if sig_threshold < table[effect_statistic_function].min():
File "/Users/kristinaadmin/miniconda2/envs/qiime2-2019.7/lib/python3.6/site-packages/pandas/core/frame.py", line 2927, in getitem
indexer = self.columns.get_loc(key)
File "/Users/kristinaadmin/miniconda2/envs/qiime2-2019.7/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 2659, in get_loc
return self._engine.get_loc(self._maybe_cast_indexer(key))
File "pandas/_libs/index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 132, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1601, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1608, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'we.eBH'

Plugin error from aldex2:

'we.eBH'

See above for debug info.
(qiime2-2019.7) pmrubine-u01dx:DVDataKMF kristinaadmin$

Thank you for sticking with it and letting me there are issues!

The differentials.qza file must not have the "diff.win" column. If you export the differentials artifact using qiime tools export --input-path differentials.qza --output-path exported-table, you should have a file under export-tabled/differentials.tsv that looks something like this (with differental featureid names of course):

featureid rab.all rab.win.subject.1 rab.win.subject.2 diff.btw diff.win effect overlap we.ep we.eBH wi.ep wi.eBH
#q2:types numeric numeric numeric numeric numeric numeric numeric numeric numeric numeric numeric
4b5eeb300368260019c1fbc7a3c718fc 6.65490542551418 11.6553625594995 -2.73742589600386 -14.511868370871 2.79215666864643 -5.21255557436981 0.00027407495720991 0.00263222492744773 0.0608728827871078 0.0285714285714286 0.19562437140331
fe30ff0f71a38a39cf1717ec2be3a2fc -1.75892279830972 -0.661063928331792 -2.5666349104301 -1.82770378943793 4.67032225404692 -0.3760155548118 0.284047027552365 0.422546378745444 0.650532905554991 0.4625 0.669093530638667

Can you check to see what the differentials.tsv file looks like?

Thanks,

Dan

**edit: grammar, is -> are

1 Like

The TSV file has the following heading....

image featureid kw.ep kw.eBH glm.ep glm.eBH
#q2:types numeric numeric numeric numeric
7c0b0ee0c96842999b3d54c4caefc5ed 0.018139988 0.15678026 0.101365735 0.380899772
a2225b79fc55173571f6fc19eb59a515 0.16833002 0.503426541 0.122922072 0.392867047
89a738e1a681203098ed00dee5da95cc 0.011707153 0.121511019 0.350661179 0.652735144

1 Like

Sorry for it being sloppy, but yes that is my only set of headings. I ran glm. =(

1 Like

The reason for this bug is because the glm function currently does not have a visualizer (yet). I realize there is an option to allow glm as the test parameter, it should have been noted that it is currently for future use only.

For now, the best way to plot would be to export the differentials artifact and manually plot with python or R. I will work on getting this into the package, and post when it has been included! I'm sorry this wasn't noted!

Thank you for reporting these bugs,

Dan

1 Like

No worries. =)

Will it work for the t-test then?

Yes, the aldex-effect-plot visualizer will produce output!