I did an experiment and it seemed to work.
- Construct a library model with a SubSystem block, mask it as 'LibA' and save it as "lib.slx" in folder .\FoldA
- Mask the SubSystem block as 'LibB' and save it as "lib.slx" in folder .\FoldB
- addpath('.\FoldA'), construct a model, drag and drop the 'LibA' block. save it and close the model.
- rmpath('.\FoldA'); addpath('.\FoldB'); open the model, it shows 'LibB'.
This is what I expected and it seemed that way. As long as you close the model and re-open the model, the library file used should be the first found in the path. Don't just use a simple block such as a Gain block, having gain of 1 in .\FoldA\lib.slx and gain of 2 in .\FoldB\lib.slx. The resulting model still points to the Gain block in the built-in Simulink library and the Gain value difference is regarded as different parameter values.
If the model is not closed and re-opened, the change of the library search path didn't seem to have effect, even if I tried to update the model.
A complicated situation could be that the library was used by multiple open models. In the above example, if a second model which also uses the block from lib.slx is kept open, then in step 4, the block still shows 'LibA'.
So the key point is whether the library has been cleared/removed from the memory. In a typical use case, close the model, switch the library search path, re-open the model should be able to switch the library. But to be robust, you really need to make sure to close all model and library files, or check if the library file is still loaded in the memory. Use these functions accordingly