Dear colleagues,
We just bought a new computer basically to do a lot of analysis with qiime2 and other bioinformatic software. The processor is an Intel (R) Core (TM) i9-9820X CPU (20 cores) with 128 GB RAM. I have installed today VIrtual Box v.6.0.8 and the last image of qiime2 (2019.4). This configuration works perfectly in other (older/simpler) PCs. The host operative systems is Windows 10 Pro. We are using the computer as a headless server.
The installation of q2 image works apparently very well, but every time that I try to execute a qiime2 command (importing data, visualization, etc) I get the same error, for example:
Yes, it is the official one, last release (yesterday). Exactly the same configuration works in a computer with a i7 processor, 32 GB RAM, also used as a headless server.
The installation works. But when I try to activate the qiime env, I get the same weard error:
(qiime2-2019.4) qiime2@qiime2core2019-4:~$ source activate qiime2-2019.4
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
Illegal instruction (core dumped)
Hmm, I suspect there is some kind of issue related to the host installation of virtualbox, perhaps some instruction set isn't present, or there is a setting misconfigured? Not really sure what else we can do on this end, but please keep us posted. Personally, my next step would be to figure out what is different between the two host machines you described, and also, I would get my sysadmin involved in that discussion.
Thanks. Please dont´close this topic because i will keep udating how it goes. I guess this can happen to other people. I also tried to install a previous version of Oracle VM (5. insteadd 6.) but with same results.
Related to this: do you think that the best option might be installing Linux as a host OS? (and then getting rith of VMs). We might experience some problems with third programs, but if we manage with qiime2 and R, 95% of the work could be done.
In that case, which Ubuntu version would you recommend?
thanks!
What’s really strange to me is that the core dump happens from the q2cli cache… That step really only imports a bunch of Python (which is the slow part), it shouldn’t be doing anything that uses a weird CPU instruction or edge case (although imports can do almost anything). The rest of the VM seems to work fine?
Do other terminal programs work, such as top (or htop), ls, and find?
I definitely support the notion of using Linux as the primary OS however!
Yes, it surprised me as well, because VMs are supposed to be clones. All goes right in the VM, with the exception of q2 commands. Htop, ls, Firefox etc. I suspect then that it has to be something very specific of python vs the new Intel i9 CPUs (of course, beyond my knowledge). And I hope that it only happens within VMs. FYI, I found a thread with sombody complaning for the same error, using numpy in a VM Ubuntu 16.04 (but with Intel i3 CPU): https://github.com/numpy/numpy/issues/9532
I will check if we can install Linux as OS and then report the result. Thanks!
Hello again!
After several trial/error tests and some work together our sysadmin I would like to shear what we got, just in case it is is useful for others. The problem was caused by an Oracle-VB incompatibility with Windows 10 Pro -what is not easy to detect because some things work, others do not, but you can actually install an Oracle VB in Windows 10 Pro; in our case, qiime2 env did not work-.
We moved then to Hyper-V, that is the virtualization system of Windows and it is optimized for this OS. It runs faster than Oracle VB. We tried two options that worked fine:
Installing a VB but using Hyper-V, with a clean install of Ubuntu 18 LTS. Then, proceed with a native install of qiime2 via Miniconda.
Using the image of qiime2 available for Oracle Virtual Box. This is a bit tricker because you have first to convert the .vmdk file to a .vhdk, but is very easy using the program StartWind v2v Converter (https://www.starwindsoftware.com/starwind-v2v-converter). After this, the q2 image works perfectly under Hyper-V.
Probably we will finally go for the installation of Linux as the OS of the machine, but Hyper-V seems a better option than Oracle in case of the need of VB under Windows10.
That’s great to hear! Sorry about the troubles with VirtualBox, but Hyper-V is certainly an amazing piece of technology, I’m a huge fan of that as well.
I’ll throw in a 3rd good option for Hyper-V:
We have Docker images for QIIME 2, which when combined with Docker on Windows and the Kitematic UI, is really very seamless to use.
Hi,
I’m having kind of the same issue. I’m trying to natively install Qiime2 Core 2019.4 on a linux virtual machine (Ubuntu 16.04.6 LTS) that’s running on our university’s server system. The installation runs smoothly without getting back any error. But when I’m trying to activate the environment I’m getting the same error ‘‘Illegal instruction (core dumped)’’. Might this be a problem due to natively installing qiime2 within a VM framework?
I’m thankfull for any help!
Cheers,
Sascha
Maybe... It is also possible that you installed the Mac environment on accident in your VM. Do you still have the .yml file downloaded as part of the installation? If so, can you double-check the filename?
Is there any chance to narrow down the issue? I've now upgraded to Ubuntu 18.04 and miniconda3, but the same issue occurs again when activating the environment:
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
Illegal instruction (core dumped)
If that is the case, I am not sure if there is anything we can do on this end. Maybe you could try @ebolyen’s recommendation of using docker? Or, @ja.morillo’s advice of using hyper-v? Sorry those aren’t very satisfying answers…
I’m having a similar issue with a native install of 2019.7 on a mac running High Sierra with 2x Xeon E5462.
I ran valgrind to get some details:
$ valgrind qiime dev refresh-cache
…
vex amd64->IR: unhandled instruction bytes: 0xF 0x1 0xD0 0x83 0xE0 0x76 0x83 0xF8 0x76 0xEB
vex amd64->IR: REX=0 REX.W=0 REX.R=0 REX.X=0 REX.B=0
vex amd64->IR: VEX=0 VEX.L=0 VEX.nVVVV=0x0 ESC=0F
vex amd64->IR: PFX.66=0 PFX.F2=0 PFX.F3=0
==12133== valgrind: Unrecognised instruction at address 0x103ddecf9.
==12133== at 0x103DDECF9: npy_cpu_supports (in /anaconda3/envs/qiime2-2019.7/lib/python3.6/site-packages/numpy/core/_multiarray_umath.cpython-36m-darwin.so)
==12133== by 0x103C5D53E: InitOperators (in /anaconda3/envs/qiime2-2019.7/lib/python3.6/site-packages/numpy/core/_multiarray_umath.cpython-36m-darwin.so)
==12133== by 0x103C5B15F: PyInit__multiarray_umath (in /anaconda3/envs/qiime2-2019.7/lib/python3.6/site-packages/numpy/core/_multiarray_umath.cpython-36m-darwin.so)
==12133== by 0x10018DB88: _PyImport_LoadDynamicModuleWithSpec (in /anaconda3/envs/qiime2-2019.7/bin/python)
==12133== by 0x10018D21B: _imp_create_dynamic (in /anaconda3/envs/qiime2-2019.7/bin/python)
==12133== by 0x10008ADD1: PyCFunction_Call (in /anaconda3/envs/qiime2-2019.7/bin/python)
==12133== by 0x1001602B1: _PyEval_EvalFrameDefault (in /anaconda3/envs/qiime2-2019.7/bin/python)
==12133== by 0x100153728: _PyEval_EvalCodeWithName (in /anaconda3/envs/qiime2-2019.7/bin/python)
==12133== by 0x100162899: fast_function (in /anaconda3/envs/qiime2-2019.7/bin/python)
==12133== by 0x10016248B: call_function (in /anaconda3/envs/qiime2-2019.7/bin/python)
==12133== by 0x10015FFC1: _PyEval_EvalFrameDefault (in /anaconda3/envs/qiime2-2019.7/bin/python)
==12133== by 0x1001627EB: fast_function (in /anaconda3/envs/qiime2-2019.7/bin/python)
==12133== Your program just tried to execute an instruction that Valgrind
==12133== did not recognise. There are two possible reasons for this.
==12133== 1. Your program has a bug and erroneously jumped to a non-code
==12133== location. If you are running Memcheck and you just saw a
==12133== warning about a bad jump, it’s probably your program’s fault.
==12133== 2. The instruction is legitimate but Valgrind doesn’t handle it,
==12133== i.e. it’s Valgrind’s fault. If you think this is the case or
==12133== you are not sure, please let us know and we’ll try to fix it.
==12133== Either way, Valgrind will now raise a SIGILL signal which will
==12133== probably kill your program.
==12133==
==12133== Process terminating with default action of signal 4 (SIGILL)
==12133== Illegal opcode at address 0x103DDECF9
==12133== at 0x103DDECF9: npy_cpu_supports (in /anaconda3/envs/qiime2-2019.7/lib/python3.6/site-packages/numpy/core/_multiarray_umath.cpython-36m-darwin.so)
==12133== by 0x103C5D53E: InitOperators (in /anaconda3/envs/qiime2-2019.7/lib/python3.6/site-packages/numpy/core/_multiarray_umath.cpython-36m-darwin.so)
==12133== by 0x103C5B15F: PyInit__multiarray_umath (in /anaconda3/envs/qiime2-2019.7/lib/python3.6/site-packages/numpy/core/_multiarray_umath.cpython-36m-darwin.so)
==12133== by 0x10018DB88: _PyImport_LoadDynamicModuleWithSpec (in /anaconda3/envs/qiime2-2019.7/bin/python)
==12133== by 0x10018D21B: _imp_create_dynamic (in /anaconda3/envs/qiime2-2019.7/bin/python)
==12133== by 0x10008ADD1: PyCFunction_Call (in /anaconda3/envs/qiime2-2019.7/bin/python)
==12133== by 0x1001602B1: _PyEval_EvalFrameDefault (in /anaconda3/envs/qiime2-2019.7/bin/python)
==12133== by 0x100153728: _PyEval_EvalCodeWithName (in /anaconda3/envs/qiime2-2019.7/bin/python)
==12133== by 0x100162899: fast_function (in /anaconda3/envs/qiime2-2019.7/bin/python)
==12133== by 0x10016248B: call_function (in /anaconda3/envs/qiime2-2019.7/bin/python)
==12133== by 0x10015FFC1: _PyEval_EvalFrameDefault (in /anaconda3/envs/qiime2-2019.7/bin/python)
==12133== by 0x1001627EB: fast_function (in /anaconda3/envs/qiime2-2019.7/bin/python)
==12133==
It looks like it was in numpy, (and you get the same error running python and trying to import numpy) so I edited the 2019.7 yml to pin an older version of numpy:
numpy=1.16
I then installed a new environment using that, and now refresh-cache seems to work.
for Linux