reproducible and independent random stream generation in parfor loop

We have a single program that has to be run on 60 independent (non-overlapping) random streams. We have 12 workers, and because each problem is to be solved independently of the others (no communication between workers), we have decided to use a parfor loop. the random streams have to be reproducible.

1- If we write the code as

stream = RandStream('mrg32k3a','Seed',seed);
parfor ii = 1:60
    set(stream,'Substream',ii);
    par(ii) = rand(stream);
end

will this create 60 reproducible non-overlapping random streams, where each seed is assigned to a single worker?

2- Within the code, we use normrnd and mvnrnd, which need rng to set the seed. How can we change the code above to be able to use normrnd and mvnrand? Will the use of rng(ii) above instead of substream solve the problem?

Thanks in advance.

Answer 1

[Edric Ellis]

This topic is covered here in the documentation. You should not mix setting 'Seed' with setting 'Substream'. (The 'Seed' value sets up the state of the random number generator in a different way, and does not give you the control you need in this situation). So, you should modify your code slightly to do this:

% Use parallel.pool.Constant to hold a RandStream on each worker
sc = parallel.pool.Constant(RandStream('mrg32k3a'));

parfor ii = 1:60
    % Get the stream value
    stream = sc.Value;
    % Set the Substream
    set(stream,'Substream',ii);
    % Make this stream the default (for normrnd etc.), and store
    % the old value for later.
    oldGlobalStream = RandStream.setGlobalStream(stream);

    par(ii) = rand(stream); % Or you could just call rand()
    
    % At the end, you could restore the old global stream
    RandStream.setGlobalStream(oldGlobalStream);
end

Here I've used RandStream.setGlobalStream to set up the stream for normrnd, and reverted at the end of the loop iteration.

[Ebru Angun]

Thanks.

Kind regards

Ebru

[Ebru Angun]

I was using

sc = parallel.pool.Constant(RandStream('mrg32k3a'));

without any problem But now, I received the following error message:

Error using parallel.pool.Constant/get.Value (line 191)

The value of a parallel.pool.Constant is only available on the workers.

stream = sc.Value;

What can be the reason for it?

Thanks in advance.

Ebru

[Edric Ellis]

This is because you're trying to access the .Value on the client, not the workers. Perhaps your pool has shut down and parfor is running on the client?

[Ebru Angun]

Thank you very much for the fast reply.

This is possible. I actually deleted the jobs by 'delete(mycluster.Jobs)'. Is it possible in such a case for the parfor still running on the client? How canı stop the parfor then and solve the problem?

Ebru

[Edric Ellis]

Deleting jobs in that way will shut down any active parpool - because a parpool needs a Job to manage the worker processes.

By default, when you encounter a parfor loop, a parpool will start if one is not available. But this doesn't happen if you set the "Auto create parallel pool" preference to false.

[Ebru Angun]

Thanks so much for all your help. I have realized that I have done a stupid mistake: I forgot to change the for loop to the parfor loop. That was the reason of the error message. After dealing with the code, ı have become blind and could not see such a mistake. Thanks.

Best regards

Ebru