Loading Qiime Environment TypeError- got type bytes

Rex · August 2, 2023, 8:32pm

Hi there,
I am running Qiime2-2023.5 in a conda environment through an ssh accessed Linux server. I am using these instructions.
I kept running into this error, followed the rabbit hole here, and followed 'Magic-Oak' instructions to run
pip config set global.index-url https://mirrors.aliyun.com/pypi/simple/
before running pip install scikit-learn==0.20.3

Now, when running quiime2 --help, or activating the environment, I encounter this error:

QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
Traceback (most recent call last):
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/util.py", line 266, in get_plugin_manager
    return qiime2.sdk.PluginManager.reuse_existing()
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/sdk/plugin_manager.py", line 58, in reuse_existing
    raise UninitializedPluginManagerError
qiime2.sdk.plugin_manager.UninitializedPluginManagerError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/bin/qiime", line 11, in <module>
    sys.exit(qiime())
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/click/core.py", line 1054, in main
    with self.make_context(prog_name, args, **extra) as ctx:
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/click/core.py", line 920, in make_context
    self.parse_args(ctx, args)
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/click/command.py", line 43, in parse_args
    return super().parse_args(ctx, args)
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/click/core.py", line 1613, in parse_args
    rest = super().parse_args(ctx, args)
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/click/core.py", line 1378, in parse_args
    value, args = param.handle_parse_result(ctx, opts, args)
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/click/core.py", line 2360, in handle_parse_result
    value = self.process_value(ctx, value)
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/click/core.py", line 2322, in process_value
    value = self.callback(ctx, self, value)
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/click/core.py", line 1273, in show_help
    echo(ctx.get_help(), color=ctx.color)
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/click/core.py", line 699, in get_help
    return self.command.get_help(self)
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/click/core.py", line 1298, in get_help
    self.format_help(ctx, formatter)
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/click/core.py", line 1331, in format_help
    self.format_options(ctx, formatter)
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/click/command.py", line 157, in format_options
    for subcommand in self.list_commands(ctx):
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/commands.py", line 92, in list_commands
    plugins = sorted(self._plugin_lookup)
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/commands.py", line 76, in _plugin_lookup
    import q2cli.core.cache
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/core/cache.py", line 285, in <module>
    CACHE = DeploymentCache()
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/core/cache.py", line 61, in __init__
    self._state = self._get_cached_state(refresh=refresh)
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/core/cache.py", line 107, in _get_cached_state
    self._cache_current_state(current_requirements)
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/core/cache.py", line 205, in _cache_current_state
    state = self._get_current_state()
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/core/cache.py", line 253, in _get_current_state
    plugin_manager = q2cli.util.get_plugin_manager()
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/util.py", line 278, in get_plugin_manager
    return qiime2.sdk.PluginManager()
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/sdk/plugin_manager.py", line 67, in __new__
    self._init(add_plugins=add_plugins)
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/sdk/plugin_manager.py", line 105, in _init
    plugin = entry_point.load()
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2517, in load
    return self.resolve()
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2523, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2_alignment/__init__.py", line 9, in <module>
    from ._mafft import mafft, mafft_add
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2_alignment/_mafft.py", line 11, in <module>
    import skbio
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/skbio/__init__.py", line 19, in <module>
    import skbio.diversity  # noqa
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/skbio/diversity/__init__.py", line 402, in <module>
    from ._driver import (alpha_diversity, beta_diversity, partial_beta_diversity,
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/skbio/diversity/_driver.py", line 13, in <module>
    import sklearn.metrics
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/sklearn/__init__.py", line 64, in <module>
    from .base import clone
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/sklearn/base.py", line 14, in <module>
    from .utils.fixes import signature
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/sklearn/utils/__init__.py", line 14, in <module>
    from . import _joblib
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/sklearn/utils/_joblib.py", line 22, in <module>
    from ..externals import joblib
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/sklearn/externals/joblib/__init__.py", line 119, in <module>
    from .parallel import Parallel
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/sklearn/externals/joblib/parallel.py", line 28, in <module>
    from ._parallel_backends import (FallbackToBackend, MultiprocessingBackend,
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/sklearn/externals/joblib/_parallel_backends.py", line 22, in <module>
    from .executor import get_memmapping_executor
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/sklearn/externals/joblib/executor.py", line 14, in <module>
    from .externals.loky.reusable_executor import get_reusable_executor
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/sklearn/externals/joblib/externals/loky/__init__.py", line 12, in <module>
    from .backend.reduction import set_loky_pickler
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/sklearn/externals/joblib/externals/loky/backend/reduction.py", line 125, in <module>
    from sklearn.externals.joblib.externals import cloudpickle  # noqa: F401
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/sklearn/externals/joblib/externals/cloudpickle/__init__.py", line 3, in <module>
    from .cloudpickle import *
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py", line 167, in <module>
    _cell_set_template_code = _make_cell_set_template_code()
  File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py", line 148, in _make_cell_set_template_code
    return types.CodeType(
TypeError: an integer is required (got type bytes)

Is this at all related to this thread's error? ImportError for miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/scipy/interpolate/dfitpack.cpython-38- x86_64-linux-gnu.so: failed to map segment from shared object after native install. - #7 by Mirte_Kuijpers

Is it possible I simply need to run a different version of python on the envrionment?

Any help is appreciated. Thank you.

Rex · August 3, 2023, 6:31pm

I went back to my original error which is similar to what is here, but the solution did not work for me.

I removed the environment, ran conda clean --all, started from scratch to reproduce the first error (repeated below).

I then tried to update scikit-learn and received:
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. umap-learn 0.5.3 requires scikit-learn>=0.22, but you have scikit-learn 0.20.3 which is incompatible.

When I check the glibc version with ldd --version I see
ldd (Ubuntu EGLIBC 2.19-0ubuntu6.14) 2.19 Copyright (C) 2014 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Written by Roland McGrath and Ulrich Drepper.

I'm pretty lost.

first error:

qiime2.sdk.plugin_manager.UninitializedPluginManagerError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/bin/qiime", line 11, in
sys.exit(qiime())
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/click/core.py", line 1130, in call
return self.main(*args, **kwargs)
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/click/core.py", line 1054, in main
with self.make_context(prog_name, args, **extra) as ctx:
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/click/core.py", line 920, in make_context
self.parse_args(ctx, args)
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/click/command.py", line 43, in parse_args
return super().parse_args(ctx, args)
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/click/core.py", line 1610, in parse_args
echo(ctx.get_help(), color=ctx.color)
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/click/core.py", line 699, in get_help
return self.command.get_help(self)
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/click/core.py", line 1298, in get_help
self.format_help(ctx, formatter)
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/click/core.py", line 1331, in format_help
self.format_options(ctx, formatter)
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/click/command.py", line 157, in format_options
for subcommand in self.list_commands(ctx):
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/commands.py", line 92, in list_commands
plugins = sorted(self._plugin_lookup)
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/commands.py", line 76, in _plugin_lookup
import q2cli.core.cache
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/core/cache.py", line 285, in
CACHE = DeploymentCache()
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/core/cache.py", line 61, in init
self._state = self._get_cached_state(refresh=refresh)
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/core/cache.py", line 107, in _get_cached_state
self._cache_current_state(current_requirements)
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/core/cache.py", line 205, in _cache_current_state
state = self._get_current_state()
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/core/cache.py", line 253, in _get_current_state
plugin_manager = q2cli.util.get_plugin_manager()
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/util.py", line 278, in get_plugin_manager
return qiime2.sdk.PluginManager()
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/sdk/plugin_manager.py", line 67, in new
self._init(add_plugins=add_plugins)
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/sdk/plugin_manager.py", line 105, in _init
plugin = entry_point.load()
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/pkg_resources/init.py", line 2517, in load
return self.resolve()
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/pkg_resources/init.py", line 2523, in resolve
module = import(self.module_name, fromlist=['name'], level=0)
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2_alignment/init.py", line 9, in
from ._mafft import mafft, mafft_add
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2_alignment/_mafft.py", line 11, in
import skbio
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/skbio/init.py", line 19, in
import skbio.diversity # noqa
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/skbio/diversity/init.py", line 402, in
from ._driver import (alpha_diversity, beta_diversity, partial_beta_diversity,
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/skbio/diversity/_driver.py", line 13, in
import sklearn.metrics
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/sklearn/init.py", line 83, in
from .utils._show_versions import show_versions
File "/home/guestuser/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/sklearn/utils/_show_versions.py", line 12, in
from ._openmp_helpers import _openmp_parallelism_enabled
ImportError: dlopen: cannot load any more object with static TLS`

lizgehret · August 3, 2023, 8:11pm

Hi @Rex,

Welcome to the :qiime2: forum!

This is certainly not something that should be required when installing QIIME 2. I suspect that something may have gone wrong in your installation, or an existing configuration on this Linux server is contributing to this issue.

It's hard to say - but if this machine is a Linux server that you're accessing via ssh (vs. your personal machine), it may have something to do with the way this server was configured.

From the traceback you provided, it looks like you are using Python 3.8 - which is what we are currently using for QIIME 2. So that shouldn't be the source of the issue.

Can you share a bit more information on the Linux server you are using? Is this owned by an institution or company? Do you have full user permissions on this server?

system · September 4, 2023, 2:11am

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.