Pyinstaller Numpy "intel Mkl Fatal Error: Cannot Load Mkl_intel_thread.dll"

July 31, 2023 Post a Comment

I'm new with python apps. I'm trying to build my python GUI app with pyinstaller. My app depends on the following packages: PyQt4, numpy, pyqtgraph, h5py. I'm working with WinPytho

Solution 1:

I had the same issue using Pyinstaller and Numpy. By default pyinstaller seems to not take into account numpy binaries so you have to specify it manually. You can add the files editing the ".spec" file "binaries" variable, but that will only work for your current program. If you want it working for all the programs your freeze you should make a "hook" and save it in C:\Python3*\Lib\site-packages\PyInstaller\hooks.

I had to adapt LeonidR's code to make the numpy-hook working. I rewrited it using a more modern, pythonic approach using list comprehensions:

fromPyInstallerimport log as logging 
fromPyInstallerimport compat
from os import listdir

mkldir = compat.base_prefix + "/Lib/site-packages/numpy/core" 
logger = logging.getLogger(__name__)
logger.info("MKL installed as part of numpy, importing that!")
binaries = [(mkldir + "/" + mkl, '') for mkl inlistdir(mkldir) if mkl.startswith('mkl_')]

"Binaries" is a list of tuples. The second item of the tuple corresponds to the folder where you want to place the 'dlls'. In this case is empty so it copies them directly in the main folder where your '.exe' is.

Solution 2:

I just ran into the same problem. As a workaround I copied the DLL's manually, as described in https://stackoverflow.com/a/34893933/4089081

I'm trying to find a better solution though.

Solution 3:

I created a hook-numpy.py to deal with this problem:

from PyInstaller import log as logging 
from PyInstaller import compat
from os import listdir

libdir = compat.base_prefix + "/lib"
mkllib = filter(lambda x : x.startswith('libmkl_'), listdir(libdir))
if mkllib <> []: 
   logger = logging.getLogger(__name__)
   logger.info("MKL installed as part of numpy, importing that!")
   binaries = map(lambda l: (libdir + "/" + l, ''), mkllib)

In my case, conda is installing the mkl libraries to speed up numpy and scipy.

Solution 4:

j4n7's answer was very helpful, however, it may or may not be buggy. compat.base_prefix uses backslashes (at least for me) but they then concatenate with "/Lib/site-packages/numpy/core" (forward slashes).

>>>from PyInstaller import compat>>>compat.base_prefix
'C:\\Python34'
>>>mkldir = compat.base_prefix + "/Lib/site-packages/numpy/core">>>mkldir
'C:\\Python34/Lib/site-packages/numpy/core'

As you can see, it produces both forward and backward slashes in a path.

Here are my steps that allowed me to bundle the numpy mkl files into onefile. Note that my particular app uses matplotlib and the problem I was experiencing was everytime I clicked a button (tkinter) to execute the plot, the app crashed.

Steps

First: Make a build of your app using:

pyinstaller --onefile--windowed yourpythonappnamehere.py

Second: Open the .spec file and add this to it. Obviously make sure the below files actually exist first. You may not have Python34 so just a friendly warning not to copy blindly.

mkl_dlls = [('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_avx.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_avx2.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_avx512.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_core.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_def.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_intel_thread.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_mc.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_mc3.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_rt.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_sequential.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_tbb_thread.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_vml_avx.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_vml_avx2.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_vml_avx512.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_vml_cmpt.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_vml_def.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_vml_mc.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_vml_mc2.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_vml_mc3.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\libiomp5md.dll', '')]

Third: where it says binaries=None, change to binaries=mkl_dlls.

a = Analysis(['yourpythonappnamehere.py'],
             pathex=['C:\\Users\\...\\Documents\\...'],
             binaries=mkl_dlls,
             datas=None,
             ....

Fourth: Re-run the first step. When your app is built, go into the dist folder and launch your app. I hope it works for you!

UPDATE: If you get Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.dll but you can clearly see that mkl_intel_thread.dll IS IN your program directory, go to numpy/core and literally copy all the files with .dll extensions that you don't have and paste them into your program's directory and re-run. If it works, great, but you might want to delete one at a time to figure out which ones you need and which ones you don't.

Solution 5:

I just update numpy+mkl to the latest version, you can download numpy+mkl from here

Python Dummy