what is the difference between running backgroundPool and parpool ('Threads') using a parfeval？

parpool('Threads')

delete(gcp('nocreate'))
pool=parpool('Threads')

f(1:8) = parallel.FevalFuture;
for i = 1:8
    f(i) = parfeval(pool, @task, 1, i, 'parpool'); 
end

afterEach(f, @disp, 0);
function output = task(taskID,poolType)
    pause(5);
    output = sprintf('Task %d is done in %s at %s', taskID, poolType,datetime('now'));
end

backgroundPool

b(1:8) = parallel.FevalFuture;
for i = 1:8
    b(i) = parfeval(backgroundPool, @task, 1, i, 'backgroundPool');
end

afterEach(b, @disp, 0);


function output = task(taskID,poolType)
    pause(5);
    output = sprintf('Task %d is done in %s at %s', taskID, poolType,datetime('now'));
end

I don't think there's any difference between them

Answer 1

[Walter Roberson]

"Pools created using parpool('Threads') and backgroundPool are both thread-based pools which utilize the same resources. It is possible that activity on one pool may block activity on the other and vice versa. Additionally, persistent data and random number generation stream state are shared in between these pools."

Unfortunately that talks more about the similarities than the differences. The differences are more difficult to find documentation on.

Background pools only support parfeval and parfevalOnAll but do not need the Parallel Computing Toolbox

Thread pools support additional constructs including parfor, but requires the Parallel Computing Toolbox

I do not know if there are any MATLAB functions that support one but not the other

[Edric Ellis]

Function support should be identical on both types of pool. The main other difference I think is that the backgroundPool always exists - it does not need to be created like a parpool pool.

It is possible, but not useful, to use this syntax to run parfor on backgroundPool:

parfor (i = 1:10, backgroundPool)
    out(i) = 7;
end

[Jixiong Su]

@Edric Ellis Thanks! However, I have another question.

My computer has 8 cores.

* The 8 process-based parallel pools and 8 background pools have all completed their tasks simultaneously.

delete(gcp('nocreate'))
pool=parpool('Processes')

% Asynchronously execute multiple tasks in the parallel pool
f(1:8) = parallel.FevalFuture;
for i = 1:8
    f(i) = parfeval(pool, @task, 1, i, 'parpool'); % Call the task function with parameter i and perform parallel computation
end

afterEach(f, @disp, 0);


b(1:8) = parallel.FevalFuture;
for i = 1:8
    b(i) = parfeval(backgroundPool, @task, 1, i, 'backgroundPool'); % Call the task function with parameter i and perform background computation
end

afterEach(b, @disp, 0);

function output = task(taskID,poolType)
    pause(5);
    output = sprintf('Task %d is done in %s at %s', taskID, poolType,datetime('now'));
end

Output:

Task 4 is done in parpool at 2023-11-25 11:41:46

Task 2 is done in parpool at 2023-11-25 11:41:46

Task 3 is done in parpool at 2023-11-25 11:41:46

Task 5 is done in parpool at 2023-11-25 11:41:46

Task 8 is done in parpool at 2023-11-25 11:41:46

Task 6 is done in parpool at 2023-11-25 11:41:46

Task 1 is done in parpool at 2023-11-25 11:41:46

Task 7 is done in parpool at 2023-11-25 11:41:46

Task 1 is done in backgroundPool at 2023-11-25 11:41:46

Task 7 is done in backgroundPool at 2023-11-25 11:41:46

Task 5 is done in backgroundPool at 2023-11-25 11:41:46

Task 2 is done in backgroundPool at 2023-11-25 11:41:46

Task 6 is done in backgroundPool at 2023-11-25 11:41:46

Task 3 is done in backgroundPool at 2023-11-25 11:41:46

Task 4 is done in backgroundPool at 2023-11-25 11:41:46

Task 8 is done in backgroundPool at 2023-11-25 11:41:46

The 8 thread-based parallel pools and 8 background pools are running simultaneously. The background pool can only grab one thread, and it can only use other threads to execute tasks after all tasks in the thread-based parallel pool are completed.

Why can backgroundPool still run in the background when eight threads are full?

delete(gcp('nocreate'))
pool=parpool('Threads')

% Asynchronously execute multiple tasks in the parallel pool
f(1:8) = parallel.FevalFuture;
for i = 1:8
    f(i) = parfeval(pool, @task, 1, i, 'parpool'); % Call the task function with parameter i and perform parallel computation
end

afterEach(f, @disp, 0);


b(1:8) = parallel.FevalFuture;
for i = 1:8
    b(i) = parfeval(backgroundPool, @task, 1, i, 'backgroundPool'); % Call the task function with parameter i and perform background computation
end

afterEach(b, @disp, 0);


function output = task(taskID,poolType)
    pause(5);
    output = sprintf('Task %d is done in %s at %s', taskID, poolType,datetime('now'));
end

Output:

Task 7 is done in parpool at 2023-11-25 11:48:19

Task 6 is done in parpool at 2023-11-25 11:48:19

Task 3 is done in parpool at 2023-11-25 11:48:19

Task 4 is done in parpool at 2023-11-25 11:48:19

Task 8 is done in parpool at 2023-11-25 11:48:19

Task 2 is done in parpool at 2023-11-25 11:48:19

Task 5 is done in parpool at 2023-11-25 11:48:19

Task 1 is done in backgroundPool at 2023-11-25 11:48:19

Task 1 is done in parpool at 2023-11-25 11:48:19

Task 2 is done in backgroundPool at 2023-11-25 11:48:24

Task 3 is done in backgroundPool at 2023-11-25 11:48:24

Task 7 is done in backgroundPool at 2023-11-25 11:48:24

Task 8 is done in backgroundPool at 2023-11-25 11:48:24

Task 5 is done in backgroundPool at 2023-11-25 11:48:24

Task 4 is done in backgroundPool at 2023-11-25 11:48:24

Task 6 is done in backgroundPool at 2023-11-25 11:48:24

[Edric Ellis]

Basically, although the pools share resources, they each have some exclusive resources to ensure that progress is never completely blocked by activity on the other pool.