Parallel for loop problem
1 view (last 30 days)
I'm running the following code in parallel. The variables of interes are w_b and alpha_b. The problem is that if I pause at a random time and then quit debugging the variables computed up to now are not saved. If i do that without the parallel cicle but with a simple for loop the variables are saved. For example, let's say I stop after 1 hour and I have 5 variables saved using a standard for loop, while with a parfor loop there are no saved variables (they are still equal to 0 as preallocated).
[LL, ~, P, obsmat, nrIterations]=dhmm_em(seq, prior, P_start_b, E,N_btstrp,Y_btstrp);
[w_b(k),alpha_b(k)]=parameters(P,ne,nn,N_btstrp,Y_btstrp,1); % VARIABLES OF INTEREST
Raymond Norris on 12 Aug 2022
@Simone Fiumi think of the code running in the parfor as running on some entirely different machine. There is no "debugging" per se. So pausing/stopping the parfor terminates the work entiring including the ability to checkpoint variables.
What you could consider doing is creating a data queue that sends data back to the MATLAB client. The client (outside of the parfor) could view the data queue. For instance:
D = parallel.pool.DataQueue;
parfor idx = 1:10
[wb, ab] = parameters(P,ne,nn,N_btstrp,Y_btstrp,1);
% Send data back to client
w_b(k) = wb;
alpha_b(k) = ab;
% Evaluate a snapshot of wb, ab
Not a perfect solution, but might help.
Benjamin Thompson on 12 Aug 2022
You may want to review this thread and use parfeval if the partial results option is something you want.