fopen and fprintf do not work within parfor or smpd running on grid

17 visualizzazioni (ultimi 30 giorni)
Hello everyone,
I met a problem to use fopen and fprintf within parfor or smpd when I run my codes on grid. Here are my codes:
spmd
fid = fopen(sprintf( 'worker_%d.csv', labindex ),'w');
fprintf(fid,'%s\r\n',vector1);
fclose(fid);
...
end
OR
parfor i=1:100
t = getCurrentTask();
fid = fopen(sprintf( 'worker_%d.csv', t.ID ),'w');
fprintf(fid,'%s\r\n',vector1);
fclose(fid);
...
end
Neither one would work, and both of them will generated the similar error messages:
spmd:
??? Error using ==> spmd_feval at 8
Error detected on lab(s) 2 15 16
Error in ==> BruteForce2>(spmd) at 103
spmd
Error in ==> BruteForce2 at 103
spmd
Caused by:
Invalid file identifier. Use fopen to generate a valid file identifier.
Error stack:
BruteForce2>(spmd body) at 105
Invalid file identifier. Use fopen to generate a valid file identifier.
Error stack:
BruteForce2>(spmd body) at 105
Invalid file identifier. Use fopen to generate a valid file identifier.
Error stack:
BruteForce2>(spmd body) at 105
PARFOR:
??? Error using ==> parallel_function at 598
Error in ==> BruteForce2>(parfor body) at 146
Invalid file identifier. Use fopen to generate a valid file identifier.
Error in ==> BruteForce2 at 105
parfor i = 1:g
The problem is my fopen, fid returns -1. The codes are running ok with local workers. So would it be some writing permission issues? I would like to know what the problem is, and where our admin should change on his side. Could anyone give me any suggestion? Thanks a lot!
Rui

Risposta accettata

Walter Roberson
Walter Roberson il 3 Mag 2013
fname = sprintf( 'worker_%d.csv', t.ID );
[fid, message] = fopen(fname, 'w');
if fid < 0;
fprintf(2, 'failed to open "%s" because "%s"\n', fname, message);
%and here, get out gracefully
end
  9 Commenti
Rui
Rui il 6 Mag 2013
Walter, Just some of them. I have 16 workers. With help of our local unix expert, we find that if I login into Permission-denied machine manually using ssh, then I do not have this problem anymore. So it looks like a configuration problem on server side. We have sent them emails. Thanks a lot for your help!
Lab 1 to 12:
About to open "worker_6.csv"
Success opening "worker_6.csv"
Closed file "worker_6.csv"
Lab 13:
About to open "worker_13.csv"
failed to open "worker_13.csv" because "Permission denied"

Accedi per commentare.

Più risposte (2)

Jason Ross
Jason Ross il 6 Mag 2013
It's possible that the workers are running under a different user name than yours. You haven't provided the specifics of the operating systems and what schedulers you are using, but you can get this information using
spmd
system('whoami')
end
And then make sure that username also has access to the directories where you want to open the files.
  1 Commento
Rui
Rui il 6 Mag 2013
Jason, thanks for help! username is all right, we just figure out that this is possibly due to a configuration issue on server. Manually logging into machines using ssh could solve my problem. There may be something wrong with the interface between matlab and grid machines.

Accedi per commentare.


Jan
Jan il 4 Mag 2013
  • Do you have write permissions to the current directory?
  • Is the file open already from a former crashing call? See fopen all.
  • Is the file opened from another application?
Using absolute filenames is an important strategy especially for parallel processing and multi-threading. Checking the success of FOPEN is a good programming practize also.
  2 Commenti
Rui
Rui il 6 Mag 2013
Jan,
Just checked everything again. fopen all shows that nothing is open. I do have the permission, but not the grid workers.

Accedi per commentare.

Categorie

Scopri di più su Startup and Shutdown in Help Center e File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by