How can I make data files available to my code running on MATLAB Distributed Computing Server?
8 visualizzazioni (ultimi 30 giorni)
Mostra commenti meno recenti
MathWorks Support Team
il 20 Lug 2017
Modificato: MathWorks Support Team
il 4 Gen 2022
I have a MATLAB program which uses some data files. These files are currently stored on my local computer. I can use these data files without issue when running parallel code on my local machine, but when I try to run on my MATLAB Distributed Computer Server cluster, I receive errors saying that the files cannot be found.
How can I make these data files available to my code running on the cluster?
Risposta accettata
MathWorks Support Team
il 4 Gen 2022
Modificato: MathWorks Support Team
il 4 Gen 2022
There are three ways to make local data files available to workers on a cluster:
1) Create a job for your computation, and attach files to the job. This option does not require infrastructure changes but will not scale well if you have many workers, large files, or a large number of files. The following example creates a job with attached files, adds a task, and submits the job. The code will need to be changed to refer to just the filename included in 'AttachedFiles', instead of the path to the file on the local machine.\n
c = parcluster('myRemoteClusterProfile');
j = batch(c, @myFunc, 1, {10,10}, 'AttachedFiles', {'myData.csv'}); % myFunc has 1 output argument and two inputs
Refer to the following documentation for more information about creating jobs for a cluster:
2) Start a parallel pool, and attach files to the job. This option is very similar to (1), but files will be attached to a parallel pool instead of a job. The files will remain on the workers while the pool is open. The same considerations apply to this approach as (1). Example:
c = parcluster('myRemoteClusterProfile');
poolobj = parpool(c);
addAttachedFiles(poolobj, {'file1.mat'});
Refer to the following documentation for more information about attaching files to a parallel pool:
3) Place the data in a networked file share which the worker machines can access
. This option may require some infrastructure changes depending on your network, however this option scales better for large files and many workers. Your code would need to use the path to the data at the network location instead of the path to the data on the local hard drive.
1 Commento
Raymond Norris
il 4 Gen 2022
You'll want to use the UNC path, not mapped drive. Permission issues makes it sound like there's a conflict with security levels of MJS. Contact MathWorks Technical Support (support@mathworks.com) for more help.
Più risposte (0)
Vedere anche
Categorie
Scopri di più su MATLAB Parallel Server in Help Center e File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!