Integer division or modulo by zero Error

Alex_14262 · April 1, 2018, 5:22pm

Hello,

After running gneiss I identified two outliers by looking at the residual plot and also at the proportion plot. I wanted to remove them from the analysis because they are quite different values from the rest so I filtered those two samples from the table with:

qiime feature-table filter-samples --i-table f-table.qza --m-metadata-file Mapping-file.tsv --p-where "SampleID IN ('1157B_FR900_T4', '1162B_FR900_T2')" --o-filtered-table gf-table.qza

and then rerun gneiss, but when I run the regression:

qiime gneiss ols-regression --p-formula "Paint+Paint_type+Timepoint" --i-table balances2.qza --i-tree hierarchy.qza --o-visualization regression_summary2.qza --m-metadata-file Mapping-file2.tsv

I get this error:

/opt/qiime2/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/gneiss-0.4.2-py3.5.egg/gneiss/regression/_ols.py:193: RuntimeWarning: invalid value encountered in true_divide
/opt/qiime2/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/scipy/stats/_distn_infrastructure.py:879: RuntimeWarning: invalid value encountered in greater
return (self.a < x) & (x < self.b)
/opt/qiime2/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/scipy/stats/_distn_infrastructure.py:879: RuntimeWarning: invalid value encountered in less
return (self.a < x) & (x < self.b)
/opt/qiime2/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/scipy/stats/_distn_infrastructure.py:1818: RuntimeWarning: invalid value encountered in less_equal
cond2 = cond0 & (x <= self.a)
/opt/qiime2/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/gneiss-0.4.2-py3.5.egg/gneiss/regression/_ols.py:206: RuntimeWarning: invalid value encountered in double_scalars
Traceback (most recent call last):
File "/opt/qiime2/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2cli/commands.py", line 224, in call
results = action(**arguments)
File "", line 2, in ols_regression
File "/opt/qiime2/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/qiime2/sdk/action.py", line 228, in bound_callable
output_types, provenance)
File "/opt/qiime2/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/qiime2/sdk/action.py", line 424, in callable_executor
ret_val = self._callable(output_dir=temp_dir, **view_args)
File "/opt/qiime2/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2_gneiss/regression/_regression.py", line 34, in ols_regression
ols_summary(output_dir, res, tree)
File "/opt/qiime2/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/gneiss-0.4.2-py3.5.egg/gneiss/plot/_regression_plot.py", line 277, in ols_summary
_k, _l = model.kfold(), model.lovo()
File "/opt/qiime2/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/gneiss-0.4.2-py3.5.egg/gneiss/regression/_ols.py", line 345, in kfold
folds = [np.arange(is, ((is)+s) % nobs) for i in range(num_folds)]
File "/opt/qiime2/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/gneiss-0.4.2-py3.5.egg/gneiss/regression/_ols.py", line 345, in
folds = [np.arange(is, ((is)+s) % nobs) for i in range(num_folds)]
ZeroDivisionError: integer division or modulo by zero

Plugin error from gneiss:

integer division or modulo by zero

See above for debug info.

I removed the samples from the metadata file (not sure if it was actually necessary), because prior to this I was getting an error about converting float NaN to integer. However, the error then changed to the I posted above.

This is the data:
gf-table.qza (80.2 KB)
balances2.qza (93.6 KB)
hierarchy.qza (77.3 KB)
Mapping-file2.tsv (2.1 KB)

Thanks

mortonjt · April 4, 2018, 5:13am

Glad that the residual plots are useful.

It looks like there are only two samples in your table, namely 1157B_FR900_T4, and 1162B_FR900_T2. Are these the samples that you want to filter out? If so, the filtering criteria may need to be inverted.

Alex_14262 · April 4, 2018, 9:04am

Hi,

Yes, precisely! I didn’t realise I had to use --p-exclude instead of --p-where. Thanks!

Calliope_Holingue · April 12, 2018, 1:41pm

Hi there,

I am having a similar issue that I'm hoping someone can help me out with. I'm using American Gut Project data. I have imported an OTU table (.biom), and created the "FeatureTable[Frequency]" and "FeatureData[Taxonomy]" file, collapsed the taxa at the p2 level, created a composition table, ran hierarchical clustering, and finally created the balances file which I'd like to use to run regression on.

However, when I run the ols-regression I receive the following error:

Plugin error from gneiss:

  integer division or modulo by zero

Debug info has been saved to /sge_scratch/temp/4820787.1.shared.q/qiime2-q2cli-err-qr221vxz.log

I've included by code below and put the data files here:

I'm running Qiime 2 from the command line on a cluster.

I'm new to Qiime 2 and would appreciate any help. I've viewed the files in Qiime 2 View but can't identify where the problem is.

Thanks in advance!

Calliope

Code:

#summarize biom table
biom summarize-table -i '/users/cholingu/agp_data/03-otus/100nt/gg-13_8-97-percent/otu_table.biom' --qualitative -o '/users/cholingu/agp_data/projects/sleep_duration/otu_table.biom_summarytable.txt'

#import sample data
qiime metadata tabulate --m-input-file '/users/cholingu/agp_data/projects/sleep_duration/sample.data_complete_subset.tsv' --o-visualization '/users/cholingu/agp_data/projects/sleep_duration/tabulated-sample-metadata.qzv'

#create feature table
qiime tools import --input-path '/users/cholingu/agp_data/03-otus/100nt/gg-13_8-97-percent/otu_table.biom' --output-path '/users/cholingu/agp_data/projects/sleep_duration/feature-table.qza' --type "FeatureTable[Frequency]"

#create taxa file
qiime tools import --input-path '/users/cholingu/agp_data/03-otus/100nt/gg-13_8-97-percent/otu_table.biom' --output-path '/users/cholingu/agp_data/projects/sleep_duration/feature-taxonomy.qza' --source-format BIOMV210Format --type "FeatureData[Taxonomy]"

#collapse taxa at p2 level
qiime taxa collapse --i-table /users/cholingu/agp_data/projects/sleep_duration/feature-table.qza --i-taxonomy /users/cholingu/agp_data/projects/sleep_duration/feature-taxonomy.qza --p-level 2 --o-collapsed-table /users/cholingu/agp_data/projects/sleep_duration/feature-table-collapsedp2.qza

#create composition table
qiime gneiss add-pseudocount --i-table /users/cholingu/agp_data/projects/sleep_duration/feature-table-collapsedp2.qza --p-pseudocount 1 --o-composition-table /users/cholingu/agp_data/projects/sleep_duration/composition.collapsedp2.qza

#run hierarchical clustering
qiime gneiss correlation-clustering --i-table /users/cholingu/agp_data/projects/sleep_duration/composition.collapsedp2.qza --o-clustering /users/cholingu/agp_data/projects/sleep_duration/hierarchy.collapsedp2.qza

#output balances
qiime gneiss ilr-transform --i-table /users/cholingu/agp_data/projects/sleep_duration/composition.collapsedp2.qza --i-tree /users/cholingu/agp_data/projects/sleep_duration/hierarchy.collapsedp2.qza --o-balances /users/cholingu/agp_data/projects/sleep_duration/balances.collapsedp2.qza

thermokarst · April 13, 2018, 3:46am

Hey there @Calliope_Holingue!

Thanks for providing all those commands above, but, I didn't actually see this command listed in your script - can you please copy-and-paste that here? Otherwise we don't know exactly what command(s) you ran to produce that error, make sense?

Thanks!

Calliope_Holingue · April 13, 2018, 3:27pm

Sorry about that! Here is what I ran:

qiime gneiss ols-regression --p-formula “sample_name_full + sleep_duration_r” --i-table /users/cholingu/agp_data/projects/sleep_duration/balances.collapsedp2.qza --i-tree /users/cholingu/agp_data/projects/sleep_duration/hierarchy.collapsedp2.qza --m-metadata-file /users/cholingu/agp_data/projects/sleep_duration/sample.data_complete_subset.tsv --o-visualization /users/cholingu/agp_data/projects/sleep_duration/regression_summary_p2_sleep.qzv

mortonjt · April 16, 2018, 3:49am

@Calliope_Holingue just as a sanity check, you were able to sanity check the tables (i.e. no extremely low abundance / variance features)?

Calliope_Holingue · April 17, 2018, 3:23am

Hi @mortonjt, this was a good suggestion. I further filtered the features and samples (–p-min-frequency 1500 for samples, --p-min-frequency 10 for features, --p-min-samples 2 for features) and the regression ran successfully.

Can you clarify how to check for low abundance / variance features? Is this something done in Qiime View? I wasn’t able to see any of this sort of exploratory info when looking at that.

Thanks!

thermokarst · April 17, 2018, 1:06pm

@mortonjt might have some other suggestions here, but the first thing that comes to my mind is running feature-table summarize, then looking at the "Feature Detail" tab (see the MP tutorial for an example). In the linked example, there are a bunch of features that are only observed once (you will most likely need to scroll down to see that info, the table is sorted by decreasing frequency.

Does that get at what you were asking about? If not, please let us know. Good luck!

Calliope_Holingue · April 17, 2018, 1:31pm

yes, thank you, this is very helpful! thanks for your help with this.

system · May 18, 2018, 7:31pm

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.