Hello All - new user here trying to install Qiime2 on a native installation of Linux using Linux Mint.
Anaconda3 installed with Conda updated to 23.1.1 and instructions followed for Qiime2 installation:
conda info --envs
# conda environments:
#
base * /home/hannah/anaconda3
qiime2-2023.2 /home/hannah/anaconda3/envs/qiime2-2023.2
When I try to activate Qiime I receive the following repeat error.
$ conda activate qiime2-2023.2
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
Illegal instruction (core dumped)
$ qiime --help
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
Illegal instruction (core dumped)
I managed to find a mention of this in the forum but no resolutions as the OP didn't reply.
Could someone please help me understand what may be the problem?
Many thanks.
Sorry you're seeing this issue. The CPU is definitely on the older end of things, but in principle it should work. I have this script for debugging this part of the loading cycle:
If you could download and run that in your environment:
python qiime_more_info.py
we should see it load each plugin and print each import performed in the process. I still expect a core dump, but the last thing printed is the likely culprit.
I've tried to run the script and I'm not sure it's run correctly but I've listed the output below. Thanks for trying to help me sort this out.
hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$ conda activate
(base) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$ conda info --envs
# conda environments:
#
base * /home/hannah/anaconda3
python /home/hannah/anaconda3/envs/python
qiime2-2023.2 /home/hannah/anaconda3/envs/qiime2-2023.2
(base) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$ cd qiime2-2023.2
bash: cd: qiime2-2023.2: No such file or directory
(base) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$ activate qiime2-2023.2
/home/hannah/anaconda3/bin/activate: 5: /home/hannah/anaconda3/envs/qiime2-2023.2/etc/conda/activate.d/activate-r-base.sh: [[: not found
(base) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$ activate qiime2-2023.2
/home/hannah/anaconda3/bin/activate: 5: /home/hannah/anaconda3/envs/qiime2-2023.2/etc/conda/activate.d/activate-r-base.sh: [[: not found
(base) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$ conda activate qiime2-2023.2
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
#!/usr/bin/env python
import sys
from qiime2.sdk import PluginManager
class SnoopImports:
def find_spec(self, name, path=None, target=None):
print(name, path)
return None
def main():
for entry in PluginManager.iter_entry_points():
print(f"=> LOADING {entry.module_name}")
entry.load()
sys.meta_path = [SnoopImports()] + sys.meta_path
if __name__ == '__main__':
main()
Illegal instruction (core dumped)
(qiime2-2023.2) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$ #!/usr/bin/env python
(qiime2-2023.2) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$ import sys
Command 'import' not found, but can be installed with:
sudo apt install graphicsmagick-imagemagick-compat # version 1.4+really1.3.38-1ubuntu0.1, or
sudo apt install imagemagick-6.q16 # version 8:6.9.11.60+dfsg-1.3ubuntu0.22.04.3
sudo apt install imagemagick-6.q16hdri # version 8:6.9.11.60+dfsg-1.3ubuntu0.22.04.3
(qiime2-2023.2) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$ from qiime2.sdk import PluginManager
Command 'from' not found, but can be installed with:
sudo apt install mailutils
(qiime2-2023.2) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$
(qiime2-2023.2) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$ class SnoopImports:
Command 'class' not found, did you mean:
command 'clasp' from deb clasp (3.3.5-4ubuntu1)
command 'iclass' from deb ivtools-bin (2.0.11d.a1-1build1)
Try: sudo apt install <deb name>
(qiime2-2023.2) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$ def find_spec(self, name, path=None, target=None):
bash: syntax error near unexpected token `('
(qiime2-2023.2) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$ print(name, path)
bash: syntax error near unexpected token `name,'
(qiime2-2023.2) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$ return None
bash: return: None: numeric argument required
bash: return: can only `return' from a function or sourced script
(qiime2-2023.2) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$
(qiime2-2023.2) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$ def main():
bash: syntax error near unexpected token `('
(qiime2-2023.2) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$ for entry in PluginManager.iter_entry_points():
bash: syntax error near unexpected token `('
(qiime2-2023.2) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$ print(f"=> LOADING {entry.module_name}")
bash: syntax error near unexpected token `f"=> LOADING {entry.module_name}"'
(qiime2-2023.2) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$ entry.load()
>
>
> sys.meta_path = [SnoopImports()] + sys.meta_path
bash: syntax error near unexpected token `sys.meta_path'
(qiime2-2023.2) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$
(qiime2-2023.2) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$ if __name__ == '__main__':
> main()
> python qiime_more_info.py
bash: syntax error near unexpected token `python'
(qiime2-2023.2) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$ python qiime_more_info.py
python: can't open file 'qiime_more_info.py': [Errno 2] No such file or directory
(qiime2-2023.2) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$ conda activate python
(python) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$ python qiime_more_info.py
Command 'python' not found, did you mean:
command 'python3' from deb python3
command 'python' from deb python-is-python3
(python) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$
Hi @ebolyen - sorry for being dopey.
I have run the script and there is a lot of output code - so much so it exceeds the character limit so I have uploaded it as a .txt file attached to this message.
Looks like the troublesome plugin to load is q2-diversity, and more specifically its the chained import from q2-diversity-lib which then imports to unifrac:
Could you please give us the results of this command (inside the same QIIME 2 environment), which will tell us the version and build of unifrac you have installed:
conda list unifrac
as well as this command which will tell us what CPU instructions you have available:
iime2-2023.2) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$ conda list unifrac
# packages in environment at /home/hannah/anaconda3/envs/qiime2-2023.2:
#
# Name Version Build Channel
unifrac 1.1.1 py38h17adfb0_1 bioconda
unifrac-binaries 1.1.1 h15a0faf_4 bioconda
I'd like to try something I haven't really done before, we're going to try to capture the core-dump so that I can inspect it in the debugger and hopefully find the offending instruction. I should be able to work backwards from that to figure out why exactly unifrac(-binaries) is causing this issue.
There's apparently a million ways to do this, but since you are on linux mint, it's probably in one of two places:
it happens to be right in the same directory that you ran the command. If so you'll see some file or directory named core or something suggestive of that.
systemd is in charge of it via coredumpctl. I've never used this controller, but it should be as easy as just running the program coredumpctl (worst case add --help to learn more). If it isn't installed, then it's not systemd managing these and hopefully it's case 1.
Let me know if either of those seem the be the case for you. I appreciate you taking the time to work this out.
Ultimately, we're not really going to be able to fix this for the current release, since the compiled binaries are just not compatible with your CPU, but we may be able to fix future builds to avoid this as you aren't the only one who's run across this issue. It's just been really difficult to track down exactly what's wrong.
Actually it may be possible to compile these libraries yourself which may be a good approach for you and others with very old hardward, but I still need more information as to the actual cause before we go that route.
Right now, the best I can do is guess it's one of the popcount operations since those tend to be inter-cpu-generation instructions and there's like a dozen of them, I see
249dcd: f3 48 0f bd d0 lzcnt %rax,%rdx
in libssu.so disassembly for example, which theoretically predates the sandy-bridge architecture, but apparently these are technically the same opcode as a mostly backwards-compatible instruction, so I'm kind of out of good ways to look at this.
That said you seem to have SSE 4.2, so you probably do have that instruction.
Hi @ebolyen - thanks for spending the time on this. I'm guessing the easiest solution is going to be for me to get my hands on a PC and turn that over the Linux OS?
That said, I've tried to see what I can find and ran coredumpctl which came up with following output which, I guess is showing my multiple attempts to run this.
Going to that path I've looked for anything coredump related and can only identify dumpsexp and corelist.
I did a bit of searching though and found the core dumps are stored in /var/lib/systemd/coredump and have listed below what is in there. If any of that is any use and you want it please let me know.
Great job sleuthing, I think any of those should be fine, although the core.python.100 one is probably our script which should be a little bit smaller?
You should be able to just attach one of those, but if the forum gives you a fuss about the extension, just throw it in a .zip file and I can go from there.
I appear to be having a similar issue. However I don't have the (core dumped). I am using Ubuntu 22.04 LTS and installed miniconda and then natively installed qiime2. When using any qiime command I get this illegal instruction. Interestingly it does look like the environment activates. Below is what happens when I activate the environment
(base) dominic@LAPTOP-D67E3C7N:~ conda activate qiime2-2023.2
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
Illegal instruction
(qiime2-2023.2) dominic@LAPTOP-D67E3C7N:~
Running the python script included above leads to the illegal instruction under unifrac._api as above also
Thanks @dnfarsi, your CPU flag list will probably come in handy as you seem to have more.
I also notice that neither you nor @Cosmic have AVX instructions, which have become pretty common at this point, so it's probably going to be related to that. Once I have a core-dump I will report back with more info on the illegal instruction in @Cosmic's case.
As for what to do in the meanwhile, it may be that Intel's MKL library is causing the issue as it almost certainly expects AVX instructions. There is a conda package which will cause an environment to "resolve" with BLAS instead of MKL. Let me know if installing this happens to force a bunch of packages to change (I wouldn't mind a copy-paste of that output as well):
It seems that the nomkl doesn't do anything in one of my environments, and using these instructions on switching BLAS implementation, the q2 env is already not using MKL generally speaking:
Hello @ebolyen, thanks for the help. Below is the output from the provided code. Let me know if it is of any use!
(qiime2-2023.2) dominic@LAPTOP-D67E3C7N:~$ conda install -c conda-forge -c bioconda -c qiime2 nomkl
Collecting package metadata (current_repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /home/dominic/miniconda3/envs/qiime2-2023.2
added / updated specs:
- nomkl
The following packages will be downloaded:
package | build
---------------------------|-----------------
nomkl-1.0 | h5ca1d4c_0 4 KB conda-forge
------------------------------------------------------------
Total: 4 KB
The following NEW packages will be INSTALLED:
nomkl conda-forge/noarch::nomkl-1.0-h5ca1d4c_0
Proceed ([y]/n)? y
Downloading and Extracting Packages
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
Illegal instruction
When I went back to the coredump folder it had cleared out the core.python.1000 so I have compressed one of the qiime dumps and attached to this msg.
I ran conda install -c conda-forge -c bioconda -c qiime2 nomkl and said yes to the packages but then stupidly shut the terminal by mistake without grabbing the output.
I ran it again and the following output came up which probably isn't any use but at least we know it didn't change anything about the core dump I guess...sorry.
hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$ conda activate
(base) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$ conda activate qiime2-2023.2
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
Illegal instruction (core dumped)
(qiime2-2023.2) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$ conda install -c conda-forge -c bioconda -c qiime2 nomkl
Collecting package metadata (current_repodata.json): done
Solving environment: done
# All requested packages already installed.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
Illegal instruction (core dumped)
(qiime2-2023.2) hannah@hannah-HP-Compaq-Pro-4300-SFF-PC:~$
Hope that coredump output is helpful.
I'm out in the field this week so won't have Linux access but will pick up again next weekend when I'm back.
Thanks for your efforts!!
