MATLAB Crashes when Using Conda Environment Other than Base

236 visualizzazioni (ultimi 30 giorni)
I am having trouble using a custom conda environment in MATLAB. Here's the code I use to bootstrap a conda environment with a few packages I need from from the web:
% bootstrap a miniconda installation & python 35 environment
% choose a miniconda version,
% !!! avoid version 4.5.12 due to openSSL DLL hell !!!
condaUrl = 'https://repo.anaconda.com/miniconda/';
s = webread(condaUrl, weboptions('ContentType','text'));
v = unique(strcat('Miniconda3',regexp(s,'(?<=>Miniconda3)[^<>]*x86_64\.exe','match')));
vn = regexp(v,'(?<=3-)(\d*\.\d*\.\d*)','match');
verMask = cellfun(@isempty, vn);
v(verMask) = [];
vn(verMask) = [];
whichVer = listdlg('ListString',v,...
'PromptString','Select Miniconda Version',...
'SelectionMode','single');
if isempty(whichVer)
% use the latest and greatest
urlString = 'https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe';
else
% use selected ver
urlString = strcat(condaUrl, v{whichVer});
end
fprintf('Downloading Miniconda3 Installer\n');
targetDir = websave('miniconda3.exe', urlString);
fprintf('Miniconda3 Download Complete\n');
fprintf('Installing Miniconda3\n');
disp('start /wait "" miniconda3.exe /InstallationType=JustMe /RegisterPython=0 /S /D=%UserProfile%\Miniconda')
!start /wait "" miniconda3.exe /InstallationType=JustMe /RegisterPython=0 /S /D=%UserProfile%\Miniconda
fprintf('Miniconda3 Installation Complete\n');
% build a custom conda environment for MATLAB
pyver = 3.5;
myenv = sprintf('myenv%0.0f', pyver*10);
condaCmdPath = fullfile(getenv('USERPROFILE'),'Miniconda\Scripts\activate.bat');
minicondaRoot = fullfile(getenv('USERPROFILE'),'Miniconda');
myenvRoot = fullfile(minicondaRoot, 'envs', myenv);
matlabPythonRoot = fullfile(matlabroot, 'extern', 'engines', 'python');
% write a bat file to run in condaprompt
fptr = fopen('condaSetup.bat','w');
fprintf(fptr, 'CALL %c%s%c %c%s%c\r\n', 34, condaCmdPath, 34, 34, minicondaRoot, 34);
% fprintf(fptr, '%ccomspec%c /k %c%c%s%c %c%s%c%c\r\n', 37, 37, 34, 34, condaCmdPath, 34, 34, minicondaRoot, 34, 34);
fprintf(fptr, 'CALL conda config --add channels conda-forge\r\n');
fprintf(fptr, 'CALL conda config --set proxy_servers.http http://np1prxy801:80\r\n');
fprintf(fptr, 'CALL conda config --set proxy_servers.https https://np1prxy801:80\r\n');
fprintf(fptr, 'CALL conda create -v -y -n %s python=%0.1f numpy scipy scikit-learn sklearn-contrib-py-earth\r\n', myenv, pyver);
fprintf(fptr, 'CALL conda activate %s\r\n', myenv);
fprintf(fptr, 'cd %s\r\n', matlabPythonRoot);
fprintf(fptr, 'python setup.py build -b %s install\r\n', myenvRoot);
fclose(fptr);
% run "makefile"
dos('condaSetup.bat','-echo');
% delete the bat file
delete 'condaSetup.bat'
% add this python environment to startup.m by appending these two lines:
% !%comspec% /c ""C:\Users\HB69954\Miniconda\Scripts\activate.bat" "C:\Users\HB69954\Miniconda\envs\myenv35""
% pyversion c:\users\hb69954\miniconda\envs\myenv35\python.exe
w = which('startup.m');
fptr = fopen(w,'r');
s = fread(fptr, inf, 'uint8=>char')';
fclose(fptr);
activatePyString = '!%comspec% /c ""C:\Users\HB69954\Miniconda\Scripts\activate.bat" "C:\Users\HB69954\Miniconda\envs\myenv35""';
swapPyVersionString = 'pyversion c:\users\hb69954\miniconda\envs\myenv35\python.exe';
fptr = fopen(w,'w');
fprintf(fptr,'%s\r\n%s\r\n%s\r\n', s, activatePyString, swapPyVersionString);
fclose(fptr);
This script downloads Miniconda3, installs it, and creates a conda virtual environment called myenv35.
By default, MATLAB chooses my base environment as the python version:
>> pyversion
version: '3.7'
executable: 'C:\Users\HB69954\Miniconda\python.exe'
library: 'C:\Users\HB69954\Miniconda\python37.dll'
home: 'C:\Users\HB69954\Miniconda'
isloaded: 1
I added these two lines to startup.m to load this python environment instead of the base environment:
!%comspec% /c ""C:\Users\HB69954\Miniconda\Scripts\activate.bat" "C:\Users\HB69954\Miniconda\envs\myenv35""
pyversion c:\users\hb69954\miniconda\envs\myenv35\python.exe
This apparently loads in MATLAB,
>> pyversion
version: '3.5'
executable: 'c:\users\hb69954\miniconda\envs\myenv35\python.exe'
library: 'c:\users\hb69954\miniconda\envs\myenv35\python35.dll'
home: 'c:\users\hb69954\miniconda\envs\myenv35'
isloaded: 0
I can use python standard library functions in this environment, i.e.
>> myList = py.list([1,2,3,0,8,9,7])
myList =
Python list with no properties.
[1.0, 2.0, 3.0, 0.0, 8.0, 9.0, 7.0]
>> myList.sort()
>> myList
myList =
Python list with no properties.
[0.0, 1.0, 2.0, 3.0, 7.0, 8.0, 9.0]
But MATLAB crashes whenever I try to use a library that installed in the virtual environment (i.e. numpy), MATLAB closes without issuing an error or warning.
x = py.numpy.array([1,2,3])
taskmgr doesn't show matlab.exe running in the background, and no crash dump files are created in %USERPROFILE%\AppData\Local\Temp, leading me to believe its not a real crash as much as some file/function is issuing an exit/close command to matlab.exe.
  1. Are python/conda environments supported in MATLAB? I can't think of a reason why they wouldn't be, but I haven't seen any official support documentation for them either.
  2. Could this be related to me installing MATLAB Engine for Python in the same virtual environment?
  3. Is there any other reason why swapping the python environment could be causing MATLAB to crash while running environment packages?

Risposte (3)

Julian Hapke
Julian Hapke il 19 Mar 2019
Modificato: Walter Roberson il 20 Dic 2021
Try this:
load your environment in a seperate promt, have a look at the path variable (assuming windows here)
echo %PATH%
unload the environment (back to base), look at the %PATH% again and find out the differences. There should be some paths from the custom evironment added to the path variable.
Add those path inside matlab, for example with:
z
pyExec = 'C:\software\miniconda3\envs\mlpy\python.exe';
pyRoot = fileparts(pyExec);
p = getenv('PATH');
p = strsplit(p, ';');
addToPath = {
pyRoot
fullfile(pyRoot, 'Library', 'mingw-w64', 'bin')
fullfile(pyRoot, 'Library', 'usr', 'bin')
fullfile(pyRoot, 'Library', 'bin')
fullfile(pyRoot, 'Scripts')
fullfile(pyRoot, 'bin')
};
p = [addToPath(:); p(:)];
p = unique(p, 'stable');
p = strjoin(p, ';');
setenv('PATH', p);
after that MATLAB should be able to use packages from that conda environment.
  4 Commenti
Fahad Raza
Fahad Raza il 16 Feb 2020
@Julian Thank you. It worked absolutely fine and you saved my day.
Seth Wagenman
Seth Wagenman il 24 Ago 2020
The only way to make sure this works is to run a Python script importing a library/module not in the base conda environment. But it definitely works!

Accedi per commentare.


Seth Wagenman
Seth Wagenman il 28 Ago 2020
Modificato: Seth Wagenman il 31 Ago 2020
For a conda environment named "useFromMATLAB", the following code works in Windows 10/Anaconda 3. Note that if you are debugging Python at the same time, and making changes to "some_awesome_python_module," you have to reload it every time (code below starting with clear classes is how to do that).
py_root_useFromMATLAB = fileparts(C:\anaconda_3\envs\useFromMATLAB\_conda.exe);
ENV = getenv('PATH');
ENV = strsplit(ENV, ';');
items_to_add_to_path = {
fullfile(py_root_useFromMATLAB, 'Library', 'mingw-w64', 'bin')
fullfile(py_root_useFromMATLAB, 'Library', 'usr', 'bin')
fullfile(py_root_useFromMATLAB, 'Library', 'bin')
fullfile(py_root_useFromMATLAB, 'Scripts')
};
ENV = [items_to_add_to_path(:); ENV(:)];
ENV = unique(ENV, 'stable');
ENV = strjoin(ENV, ';');
setenv('PATH', ENV);
clear classes
module_to_load = 'some_awesome_python_module';
python_module_to_use = py.importlib.import_module(module_to_load);
py.importlib.reload(python_module_to_use);
% Now you can use it like output = py.some_awesome_python_module.some_awesome_python_function(input)
  1 Commento
Seth Wagenman
Seth Wagenman il 28 Ago 2020
https://www.mathworks.com/help/matlab/matlab_external/call-modified-python-module.html

Accedi per commentare.


Tucker Downs
Tucker Downs il 20 Dic 2021
Modificato: Tucker Downs il 20 Dic 2021
The obvious feature / solution would be for Mathworks to just respect the user environment when people want to call a python script from matlab. Getting a simple script to run with my anaconda environment should not require such work arounds when I can open VSCode or terminal and run the exact same script in less than 10 secconds.
Extremely dissapointed in this right now, as much as I love Matlab it's shit like this that makes me want to leave the platform and go full time python. Even with the pain of losing MW support and documentation. The core features are just not keeping up with other development stratagies.
  5 Commenti
Tucker Downs
Tucker Downs il 20 Dic 2021
Hi @Walter Roberson thanks for your insightful comment.
It will be interesting to see how that affects vscode. I'm going to guess that I'll see the same functionality in code as I always have, in which case there won't really an excuse for a code development product like MATLAB desktop.
Majid Khalili
Majid Khalili il 17 Ott 2022
Modificato: Majid Khalili il 17 Ott 2022
@Tucker Downs did you manage to make it work for activating an envirioment from anaconda in Matlab?

Accedi per commentare.

Prodotti


Release

R2016b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by