Why more CPU cores makes parallel workers less efficient?

I use MATLAB 2023a on Linux server, and I can customize the number of logical CPU cores for a MATLAB session. But I found the time to run the same parfor loop codes using more logical cores is even longer than using less cores, which really confuses me. For example, 16 cores takes 61 second while 32 cores take 74 second, when other configurations are equivalent in terms of 2GB RAM per CPU core assigned and poolsize of parallel workers set to 4.

Answer 1

[John D'Errico]

I'll assume the process really can be perfectly parallelized, so that you can break the problem down into multiple fully parallel problems.

More logical cores are not always a gain. I recall the issue can be overhead. Suppose you had one real core, but the ability to split it into 1048576 logical cores? Now you can solve a million problems at once, but each one gets solved a million times more slowly. You are just slicing time. It seems like the same amount of work is being done, but you have overhead in that. Swapping in and out. Surely you agree this would be a bad idea, most of the time because the overhead will kill you at some point.

Having multiple logical cores is nice, because then your CPU can handle all of the many processes that are running concurrently in the background. I remember the old days, with one CPU, one core. Set something running, and then go read a book. A real book, not online, because you can't even answer your e-mail. Things are just frozen up until the job is done. So multiple real cores and multiple logical cores are great. But there are limits to the greatness.

The point in all this is that real cores are different from logical cores. You usually gain from the use of more real cores, though even there there can be limits. But time slicing a real core into too many logical cores often does not gain on these highly parallel problems, and may even slow things down.

[Yunxiao]

Hi, thank you for your answer. But I believe the logical cores in the server are just as real cores because the server distributes certain capability of computation to each core. So may I suspect it is because MATLAB could not distribute works to more cores then the efficiency is worser than less cores?

[Damian Pietrus]

As mentioned, you should only start up as many workers as there are physical cores on a CPU to avoid resource contention. Additionally, there is parallel overhead every time that you use a parfor, and you do want to match your iterations to your total number of workers in some way. If, as an example, you have 32 workers but only have 10 iterations, many of the workers are going to sit there idle.

Feel free to share a code example if you have any specific questions about your parfor loop