Fatal Python error: Py_Initialize: unable to load the file system codec

ecg · 2019-02-13 20:13:12 UTC

I am trying to activate qiime2 on my Mac terminal again after a while, so when I run:

source activate qiime2-2018.11

I get the following error:
Fatal Python error: Py_Initialize: unable to load the file system codec
Traceback (most recent call last):

File “/Users/user1/miniconda3/lib/python3.5/encodings/init.py”, line 31, in <module>
File “/Users/user1/miniconda3/lib/python3.5/codecs.py”, line 322, in <module>

NameError: name ‘open’ is not defined

Is this due to a photon version/upgrade? What do I need to do to fix this? Do I need to modify the scripts, or downgrade photon?

Let me know,

Thanks

E

thermokarst · 2019-02-11 17:48:18 UTC

Hey there @ecg — I wonder if something is up with your environment — can you run the following and paste the results here:

env

Thanks!

ecg · 2019-02-11 18:08:45 UTC

Hi, here it is:

Thanks,

TERM_PROGRAM=Apple_Terminal
SHELL=/bin/bash
TERM=xterm-256color
TMPDIR=/var/folders/cw/f68f84_d41sbtj8flvsls8mxljrs1s/T/
CONDA_SHLVL=0
Apple_PubSub_Socket_Render=/private/tmp/com.apple.launchd.N2TkKsd4lG/Render
TERM_PROGRAM_VERSION=361.2
TERM_SESSION_ID=902543AD-C050-461C-8E45-C56E6459ADBA
USER=user1
SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.nXMc9tDKQU/Listeners
__CF_USER_TEXT_ENCODING=0x671C6439:0x0:0x2
PATH=/opt/local/bin:/opt/local/sbin:/Users/user1/miniconda3/bin:/opt/local/bin:/opt/local/sbin:/Users/ecg1g15/miniconda3/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin
PWD=/Users/user1
LANG=en_GB.UTF-8
XPC_FLAGS=0x0
XPC_SERVICE_NAME=0
SHLVL=1
HOME=/Users/user1
LOGNAME=user1
DISPLAY=/private/tmp/com.apple.launchd.el8AL41zM2/org.macosforge.xquartz:0
_=/usr/bin/env

thermokarst · 2019-02-11 18:10:12 UTC

Hey there @ecg - it looks like you have a few miniconda deployments in your PATH — any idea what is going on there? I’m not sure if that is the problem, but, that can probably cause some issues.

Can you try installing 2019.1? I wonder if your env just got corrupted…

ecg · 2019-02-11 18:24:19 UTC

Hi @thermokarst ,
I have recently been updating other software, and had other errors when using phyton. I wonder if I have been updating/changing environments… Maybe best to do is uninstall all environments and photon and re-install the newest qiime 2019?

interesting, when trying to install the new qiime2 version as stated on https://docs.qiime2.org/2019.1/install/native/ it lets me install the file qiime2-2019.1-py36-osx-conda.yml

but when running the following command I get this same error:

conda env create -n qiime2-2019.1 --file qiime2-2019.1-py36-osx-conda.yml

Fatal Python error: Py_Initialize: unable to load the file system codec
Traceback (most recent call last):

File “/Users/user1/miniconda3/lib/python3.5/encodings/init.py”, line 31, in <module>
File “/Users/user1/miniconda3/lib/python3.5/codecs.py”, line 322, in <module>

NameError: name ‘open’ is not defined

Abort trap: 6

ebolyen · 2019-02-11 22:09:59 UTC

I think @thermokarst is on to something with the PATH.

Apparently problems like this stem from inconsistent Python installations.

I suspect there is another Python in :

/opt/local/bin

which is at the start of your PATH.

Try running this command before the conda command and see if that helps:

export PATH=/Users/user1/miniconda3/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin

That should set your PATH to something mostly sane, if source activate or conda start working after that, then there’s something we should be correcting in your .bash_profile/.bashrc to make sure things work.

ecg · 2019-02-12 18:08:45 UTC

Thanks @thermokarst and @ebolyen but I had no luck sorry, after running the export command and trying the conda command, still gives me the same “fatal python error”.

I wonder if I should uninstall all pythons from my device and re-install it from scratch?

thermokarst · 2019-02-12 23:39:47 UTC

Before uninstalling everything, what do you get back from conda info?

ecg · 2019-02-13 15:23:20 UTC

@thermokarst Same thing:

Fatal Python error: Py_Initialize: unable to load the file system codec
Traceback (most recent call last):
File “/Users/user1/miniconda3/lib/python3.5/encodings/init.py”, line 31, in <module>
File “/Users/user1/miniconda3/lib/python3.5/codecs.py”, line 322, in <module>
NameError: name ‘open’ is not defined
Abort trap: 6

thermokarst · 2019-02-13 15:41:47 UTC

Yeah, I strongly suspect that your PATH (and maybe PYTHONPATH) env vars are implicated here — this looks like the two different miniconda deployments you have are conflicting (one is at /Users/user1/miniconda3 and the other at /Users/ecg1g15/miniconda3, plus, who knows what other pythons are lurking in there). I think if I were in your shoes I would tidy up my workspace a bit and make sure that everything is squared away first (e.g. uninstall your extra python deployments, clean up your env vars, consider reducing your two users accounts to one, etc.). Sorry, wish I could be more help here, but please keep us posted if you get stuck!

ecg · 2019-02-13 16:10:42 UTC

Thanks @thermokarst will do and will post the results! Thanks a lot for your help to a very beginner in bioinformatics

ecg · 2019-02-13 19:52:51 UTC

Hi @thermokarst I got it sorted.

Turns out that I had about 7 different versions of python installed, I have installed them using brew and others so I had python on system, mac port and miniconda.

The version of miniconda that I had was also old, so deleted miniconda, re-installed the last version from https://conda.io/en/latest/miniconda.html and then re-installed qiime2 as stated on https://docs.qiime2.org/2019.1/install/native/#install-miniconda

Since qiime is using miniconda, I have moved miniconda to be top of the list to be the priority one when using python from the miniconda3 bin. Does this make sense? A couple of IT friends helped me out here. Thanks a lot for helping me anyways, I need to be more careful with how I install apps/packages… etc.

All the best,

E