LU 5/6 are often stdio and I presume highly likely is for the compiler in question. LU 5 in that case will be input so the app is asking for input either from the keyboard or it appears from the error message to the default implicitly-opened file 'fort.5'.
The crystal ball tells me that the most likely problem is that when you run the code manually either you give a file name directly to the app that contains the input or if it is using the default name then when you start the shell session it is starting in a different directory location than where you're running it manually.
I suspect there's a good chance I was the one who suggested the mex route; I seem to recall a question not too long ago in that vein. :)
The advantages are that you have control over what's going on and can return i/o directly into the Matlab workspace w/o the shell session. The disadvantage is, of course, you have to write/debug the glue code--how difficult that is depends on the complexity of the code and your level of proficiency in Fortran and the mex interfaces.
