About q2-SCNIC installation

Hello there!

I'm trying to figure out how to install and run SCNIC but facing my inhability to do it, I feel like a Gordon Ramsay's :sandwich:.

Lately I've reproduced errors ranging from the ones previously reported in q2-SCNIC Correlations calculation #error plugin DataFrame to_dense and q2 scnic fails in calculating correlations which apparently are due to the q2 environment version (mainly 2021.4, 2020.8, 2020.11); to other errors associated with the conda channel, pip installation, missing libraries (networkx and/or r-pillar=1.2.2 in q2-2018.8), no longer supported and conflicted envs (q2-2018.11), etc (I've lost count of all the errors so far).

My question: is this plugin still functional and ready to use? and if it is under what conditions needs to be installed in order to run properly?

Cheers,
Luis Alfonso.

PD, I know this topic kinda resurfaces every two months so if it's redundant please don't accept it.

Hey Luis, did you have any issues installing the plugin using these commands: QIIME 2 Library?

Hey Kumar,

Following the link's commands, installing with pip, in qiime 2021.2 (and other recent qiime2 versions), python 3.6.13, I get the following error:
(qiime2-20212) [[email protected] workdir-feb2021] pip install --user scnic
Requirement already satisfied: scnic in /home/larteag7/.conda/envs/qiime2-20212/lib/python3.6/site-packages (0.6.2)
Requirement already satisfied: networkx>=2 in /home/larteag7/.conda/envs/qiime2-20212/lib/python3.6/site-packages (from scnic) (2.5)
(qiime2-20212) [[email protected] workdir-feb2021]$ pip install git+https://github.com/lozuponelab/q2-SCNIC.git
Collecting git+https://github.com/lozuponelab/q2-SCNIC.git
Cloning https://github.com/lozuponelab/q2-SCNIC.git to /tmp/pip-req-build-sjg3ewpk
Running command git clone -q https://github.com/lozuponelab/q2-SCNIC.git /tmp/pip-req-build-sjg3ewpk
Building wheels for collected packages: q2-SCNIC
Building wheel for q2-SCNIC (setup.py) ... done
Created wheel for q2-SCNIC: filename=q2_SCNIC-2020.10+4.gf8566e7-py3-none-any.whl size=13659 sha256=fc8d940bcab83bfe52d67cefaeffd6b5b0528c84f9f02a27f21ba67c861c3447
Stored in directory: /tmp/pip-ephem-wheel-cache-_sk_h2f1/wheels/64/7b/3c/5bcf3c4eb19280ce7539e810efd388ff14d07e9c5283f87c24
Successfully built q2-SCNIC
Installing collected packages: q2-SCNIC
Attempting uninstall: q2-SCNIC
Found existing installation: q2-SCNIC 2018.8.0+20.g07ffa4a
Uninstalling q2-SCNIC-2018.8.0+20.g07ffa4a:
Successfully uninstalled q2-SCNIC-2018.8.0+20.g07ffa4a
Successfully installed q2-SCNIC-2020.10+4.gf8566e7

...

(qiime2-20212) [[email protected] workdir-feb2021]$ qiime SCNIC calculate-correlations --i-table fake-data-filtered.qza --p-method sparcc --o-correlation-table fake-correls.qza
Plugin error from SCNIC:

'DataFrame' object has no attribute 'to_dense'

Debug info has been saved to /tmp/qiime2-q2cli-err-0dzoplyr.log
(qiime2-20212) [[email protected] workdir-feb2021]$ cat /tmp/qiime2-q2cli-err-0dzoplyr.log
Correlating with sparcc
Traceback (most recent call last):
File "/home/larteag7/.conda/envs/qiime2-20212/lib/python3.6/site-packages/q2cli/commands.py", line 329, in call
results = action(**arguments)
File "", line 2, in calculate_correlations
File "/home/larteag7/.conda/envs/qiime2-20212/lib/python3.6/site-packages/qiime2/sdk/action.py", line 245, in bound_callable
output_types, provenance)
File "/home/larteag7/.conda/envs/qiime2-20212/lib/python3.6/site-packages/qiime2/sdk/action.py", line 390, in callable_executor
output_views = self._callable(**view_args)
File "/home/larteag7/.conda/envs/qiime2-20212/lib/python3.6/site-packages/q2_SCNIC/_SCNIC_methods.py", line 29, in calculate_correlations
correls = ca.fastspar_correlation(table, verbose=True, nprocs=n_procs)
File "/home/larteag7/.conda/envs/qiime2-20212/lib/python3.6/site-packages/SCNIC/correlation_analysis.py", line 77, in fastspar_correlation
table.to_dataframe().to_dense().to_csv(path.join(temp, 'otu_table.tsv'), sep='\t', index_label='#OTU ID')
File "/home/larteag7/.conda/envs/qiime2-20212/lib/python3.6/site-packages/pandas/core/generic.py", line 5141, in getattr
return object.getattribute(self, name)
AttributeError: 'DataFrame' object has no attribute 'to_dense'

As someone mentioned before, in the same step if you use a different method, it works.

(qiime2-20212) [[email protected] workdir-feb2021]$ qiime SCNIC calculate-correlations --i-table fake-data-filtered.qza --p-method pearson --o-correlation-table fake-correls.qza
Saved PairwiseFeatureData to: fake-correls.qza

I read somewhere than in an 2018 version of qiime it might work so, following link's commands installing with pip, in qiime 2018.8, python 3.5.5, I get the following error:

(qiime2-20188) [[email protected] workdir-feb2021] pip install --user scnic
Collecting scnic
Collecting networkx>=2 (from scnic)
Using cached https://files.pythonhosted.org/packages/41/8f/dd6a8e85946def36e4f2c69c84219af0fa5e832b018c970e92f2ad337e45/networkx-2.4-py3-none-any.whl
Installing collected packages: networkx, scnic
Successfully installed networkx-2.4 scnic-0.6.2
(qiime2-20188) [[email protected] workdir-feb2021] pip install git+https://github.com/lozuponelab/q2-SCNIC.git
Collecting git+https://github.com/lozuponelab/q2-SCNIC.git
Cloning https://github.com/lozuponelab/q2-SCNIC.git to /tmp/pip-req-build-ed9vwtrj
Requirement already satisfied (use --upgrade to upgrade): q2-SCNIC==2020.10+4.gf8566e7 from git+https://github.com/lozuponelab/q2-SCNIC.git in /home/larteag7/.conda/envs/qiime2-20188/lib/python3.5/site-packages
Building wheels for collected packages: q2-SCNIC
Running setup.py bdist_wheel for q2-SCNIC ... done
Stored in directory: /tmp/pip-ephem-wheel-cache-4ufgz68z/wheels/ad/4f/90/efd772f34ca8f61d2990b841f051aaf9ecdb816e01512ae3a1
Successfully built q2-SCNIC
(qiime2-20188) [[email protected] workdir-feb2021]$ qiime dev refresh-cache

QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
Traceback (most recent call last):
File "/home/larteag7/.conda/envs/qiime2-20188/bin/qiime", line 11, in
sys.exit(qiime())
File "/home/larteag7/.conda/envs/qiime2-20188/lib/python3.5/site-packages/click/core.py", line 722, in call
return self.main(*args, **kwargs)
File "/home/larteag7/.conda/envs/qiime2-20188/lib/python3.5/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/home/larteag7/.conda/envs/qiime2-20188/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/larteag7/.conda/envs/qiime2-20188/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/larteag7/.conda/envs/qiime2-20188/lib/python3.5/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/larteag7/.conda/envs/qiime2-20188/lib/python3.5/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/home/larteag7/.conda/envs/qiime2-20188/lib/python3.5/site-packages/q2cli/dev.py", line 27, in refresh_cache
import q2cli.cache
File "/home/larteag7/.conda/envs/qiime2-20188/lib/python3.5/site-packages/q2cli/cache.py", line 301, in
CACHE = DeploymentCache()
File "/home/larteag7/.conda/envs/qiime2-20188/lib/python3.5/site-packages/q2cli/cache.py", line 61, in init
self._state = self._get_cached_state(refresh=refresh)
File "/home/larteag7/.conda/envs/qiime2-20188/lib/python3.5/site-packages/q2cli/cache.py", line 107, in _get_cached_state
self._cache_current_state(current_requirements)
File "/home/larteag7/.conda/envs/qiime2-20188/lib/python3.5/site-packages/q2cli/cache.py", line 200, in _cache_current_state
state = self._get_current_state()
File "/home/larteag7/.conda/envs/qiime2-20188/lib/python3.5/site-packages/q2cli/cache.py", line 238, in _get_current_state
plugin_manager = qiime2.sdk.PluginManager()
File "/home/larteag7/.conda/envs/qiime2-20188/lib/python3.5/site-packages/qiime2/sdk/plugin_manager.py", line 44, in new
self._init()
File "/home/larteag7/.conda/envs/qiime2-20188/lib/python3.5/site-packages/qiime2/sdk/plugin_manager.py", line 59, in _init
plugin = entry_point.load()
File "/home/larteag7/.conda/envs/qiime2-20188/lib/python3.5/site-packages/pkg_resources/init.py", line 2325, in load
return self.resolve()
File "/home/larteag7/.conda/envs/qiime2-20188/lib/python3.5/site-packages/pkg_resources/init.py", line 2331, in resolve
module = import(self.module_name, fromlist=['name'], level=0)
File "/home/larteag7/.conda/envs/qiime2-20188/lib/python3.5/site-packages/q2_SCNIC/plugin_setup.py", line 7, in
from q2_SCNIC._format import GraphModelingLanguageFormat, GraphModelingLanguageDirectoryFormat, PairwiseFeatureDataFormat,
File "/home/larteag7/.conda/envs/qiime2-20188/lib/python3.5/site-packages/q2_SCNIC/_format.py", line 3, in
import networkx as nx
ImportError: No module named 'networkx'

Let me know what you think,
Luis.

1 Like

Hi!
I collected all above mentioned errors while trying to work with SCNIC or q2-SCNIC.
The problem is that 'to_dense' function was deprecated in version 1.0, so you need to downgrade to pandas 0.25

Do not use conda to downgrade since it will mess with dependencies.
For me worked:
pip install pandas==0.25

PS. Do not forget to upgrade pandas back to previous version since you may encounter some errors in other plugins,or install a separate qiime2 version that will be used only for SCNIC

2 Likes

Hi Timur,

Sorry for the late answer!
It worked, then it failed due to fastspar but after a conda installation (following cmd)

conda install -c bioconda -c conda-forge --override-channels fastspar

I was finally able to run scnic,
Thank you so much,
Cheers,