MATLAB loses track of directories/functions in the middle of a loop

Hi everyone, I'm running a global optimization using Simulated Annealing and I've been running into an issue where, when I run the program it'll get through most of one iteration, or even most of the second iteration (after a successful first iteration) and then randomly lose track of an available directory or function that it had no problem accessing before. It always breaks at a different point within the iteration, and I check the paths that it's struggling with and they work perfectly fine. I've tried the optimization with a smaller dataset, with different MaxIterations limits, and it doesn't have this problem. Only with the full dataset (it takes about 5 hours to run one iteration) does it struggle with this. Any help would be appreciated!

Answer 1

[Image Analyst]

It's hard to guess without any code. Are you sure you're providing the entire complete path to whatever function needs the folder or filename? Are you sure there is no call to cd() anywhere in your code?

See the FAQ:

https://matlab.fandom.com/wiki/FAQ#Where_did_my_file_go?_The_risks_of_using_the_cd_function.

[Yiling]

Thanks, yeah it's hard to post my code because it uses an impossible-to-navigate external toolbox and I have no idea what's going on in there. I guess my only question is, does MATLAB ever struggle with optimizing (here, I'm iterating through 25 sets of data to optimize a matrix of 1600 values for a reference value). I'm baffled that I could run the program successfully for smaller sets of data, or that the program could run successfully through the 25 sets once or twice but randomly fails on the 3rd try. Everything that worked before should still work :(

[Image Analyst]

How do you know it "loses track"? Is there an error message? Like saying file not found?

[Yiling]

Yep, either directory not found, or function doesn't exist/isn't defined, and each time it's a different directory/function that this happens at, but I know for a fact that these directories/functions are used in EVERY iteration of every loop so it's definitely been successful in previous iterations.

[Stephen23]

Do any of the functions (including any third-party toolboxes, etc) call CD or any other path-modifiying commands?

[Yiling]

From what I can tell, no 'cd' but there are addpath(genpath()) commands.

[Image Analyst]

I don't think addpath genpath should cause the problem. Show us what one of hte error messages says.

[Stephen23]

"there are addpath(genpath()) commands."

That is quite possibly related: if folders are being added to the path, there might be some race condition occurring.

It is unusual to have the need to add all folders under MATLABROOT/TOOLBOX. Something fishy there.

[Yiling]

Thank you both, I'll add an error message when it occurs again.