GNEISS. Aitchison basis is not orthonormal

timanix · May 6, 2019, 12:34pm

Dear all!

I successfully performed GNEISS analysis on my data using Qiime2 2019.1.
Now, I tried to rerun the same notebook with the same commands but in Qiime2-2019.4 environment and received following error:

Plugin error from gneiss:

  Aitchison basis is not orthonormal

Debug info has been saved to /tmp/qiime2-q2cli-err-xegrcup1.log

Log:

/home/bio/anaconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/gneiss/util.py:245: FutureWarning: '.reindex_axis' is deprecated and will be removed in a future version. Use '.reindex' instead.
  _table = _table.reindex_axis(sorted_features, axis=1)
Traceback (most recent call last):
  File "/home/bio/anaconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/q2cli/commands.py", line 311, in __call__
    results = action(**arguments)
  File "</home/bio/anaconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/decorator.py:decorator-gen-273>", line 2, in ilr_hierarchical
  File "/home/bio/anaconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/qiime2/sdk/action.py", line 231, in bound_callable
    output_types, provenance)
  File "/home/bio/anaconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/qiime2/sdk/action.py", line 365, in _callable_executor_
    output_views = self._callable(**view_args)
  File "/home/bio/anaconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/q2_gneiss/composition/_method.py", line 20, in ilr_hierarchical
    return ilr_transform(add_pseudocount(table, pseudocount), tree)
  File "/home/bio/anaconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/gneiss/composition/_composition.py", line 43, in ilr_transform
    balances = ilr(_table.values, basis)
  File "/home/bio/anaconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/skbio/stats/composition.py", line 573, in ilr
    _check_orthogonality(basis)
  File "/home/bio/anaconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/skbio/stats/composition.py", line 1326, in _check_orthogonality
    raise ValueError("Aitchison basis is not orthonormal")
ValueError: Aitchison basis is not orthonormal

I wasn’t able to find anything about this error. Any ideas?

Nicholas_Bokulich · May 6, 2019, 2:15pm

@timanix,
Could you please post the full command you used? This will help the gneiss developer trace your problem.

For now, my recommendation is to load your notebook in a 2019.1 environment and run with that. Some of the underlying packages that QIIME 2 uses for statistical analysis have been updated since the 2019.1 release, so there may be some backward incompatibility that needs to be addressed.

Good luck!

timanix · May 6, 2019, 3:37pm

Thank you for your answer!

Of course. Here it is:

folders = ['All_Samples', 'No_Control', 'Control']
for folder in folders:
    Niche_list = ['Root', 'Gall', 'J2', 'Soil']
    for Niche in Niche_list:
        outdir    = '%s/By_Niche/%s/GNEISS'            % (folder, Niche)
        !mkdir $outdir
        
        table     = '%s/By_Niche/%s/%s_%s_table.qza'   % (folder, Niche, folder, Niche)
        hierarchy = outdir + '/%s_hierarchy.qza'       % Niche
        balances  = outdir + '/%s_balances.qza'        % Niche
        !qiime gneiss correlation-clustering \
            --i-table $table \
            --o-clustering $hierarchy
        !qiime gneiss ilr-hierarchical \
            --i-table $table \
            --i-tree $hierarchy \
            --o-balances $balances

It was first I wanted to check.
I had Qiime2-2019.1 installed in anaconda2. Everything worked fine, I successfully performed GNEISS as described above. Then new release came out, I changed some filtration steps in the very beginning of the analysis so I decided to rerun my notebooks. But I wasn't able to install new release, constantly getting error as described here QIIME2 2018.4 Installation Error. Since I wanted to switch to anaconda 3 anyway I just deleted anaconda2, installed anaconda3 and new Q2 release. And got a mistake in GNEISS. As you suggested, I decided to repeat this step in Q2-2019.1, but again encountered an issue with installing Q2-2019.1 with the same error as here QIIME2 2018.4 Installation Error.
After running command from this answer QIIME2 2018.4 Installation Error - #9 by ebolyen I managed to install 2019.1 version.

Right now I am running the same command with GNEISS and it is working without errors. So maybe I found a bug.

Thanks again

mortonjt · May 6, 2019, 3:52pm

weird … the only reason why something like that would pop up is because there is something wrong with the tree.

Definitely do keep us posted if this pops up again.

timanix · May 6, 2019, 3:56pm

Yes, it is strange. In Q2-2019.4 I am receiving an error, in Q2-2019.1 it’s working fine. The same data.

mortonjt · May 6, 2019, 3:59pm

Hi @timanix - do you mind if you could provide the files that were used to generate this error?

timanix · May 6, 2019, 4:03pm

Here is the table I used. Do you need some more files/information?
All_Samples_Root_table.qza (391.8 KB)

thermokarst · May 19, 2019, 10:13pm

Hi @timanix - have you updated to the latest 2019.4, with the openblas fix? If so, please re-run these commands in that env. If not, please update.

timanix · May 19, 2019, 10:28pm

Hi!
Yes, I updated Qiime2 to the latest version with fixes and encountered the same problem. Moreover, I redid cutadapt trimming and Dada2 (BTW, it took three hours against 24 with previous one with the same data, thx guys for the update) with different parameters (due to other issues that were solved by this) and still got an error. I’ll check tomorrow if the error message changed. It’s only happens in GNEISS, so I am performing it in 2019.1

timanix · May 20, 2019, 8:31am

Hi!
The error message changed with the last update.
Actually, now this two commands are working fine:

!qiime gneiss correlation-clustering \
    --i-table $tabin \
    --o-clustering $hier
!qiime gneiss ilr-hierarchical \
    --i-table $tabin \
    --i-tree $hier \
    --o-balances $balan

So, the previous issue was fixed with the last update, I suppose.
New error popping up on this command:

!qiime gneiss ols-regression \
    --p-formula TP \
    --i-table $balan \
    --i-tree $hier \
    --m-metadata-file Metadata/metadata.tsv \
    --o-visualization $regress

Text of the error:

Plugin error from gneiss:
  Detected zero variance balances - double check your table for unobserved features.
Debug info has been saved to /tmp/qiime2-q2cli-err-hz09nj4b.log

Log:

Traceback (most recent call last):
  File "/home/bio/anaconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/q2cli/commands.py", line 311, in __call__
    results = action(**arguments)
  File "</home/bio/anaconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/decorator.py:decorator-gen-278>", line 2, in ols_regression
  File "/home/bio/anaconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/qiime2/sdk/action.py", line 231, in bound_callable
    output_types, provenance)
  File "/home/bio/anaconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/qiime2/sdk/action.py", line 427, in _callable_executor_
    ret_val = self._callable(output_dir=temp_dir, **view_args)
  File "/home/bio/anaconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/q2_gneiss/regression/_regression.py", line 28, in ols_regression
    raise UserWarning(message)
UserWarning: Detected zero variance balances - double check your table for unobserved features.

The same command with the same data worked fine in Qiime2-2019.1

mortonjt · May 20, 2019, 3:27pm

@timanix, could you provide all of the files used to reproduce this error?

I can't comment on the different between qiime2-2019.1 and qiime-2019.4, but I will say that there are a lot of features that are only observed in one sample

This will cause problems for most statistical analyses, so those should be filtered out beforehand.

timanix · May 20, 2019, 4:58pm

My bad, I did a filtration step to get rid of low abundant features or features presented only in 5 samples with my main table and forgot to check it again after dividing my table. So, the problem definitely was fixed with the latest update.
Thank you for your patience and help!