Running a Windows Matlab script using Hudson/Jenkins

I'm trying to figure out how to create a Windows batch job to run a MATLAB function using Hudson/Jenkins. I haven't seen much information about this so I was wondering if anyone else has tried it.

I have a batch file that mounts the drive where my MATLAB script lives, creates a command line for MATLAB, and starts my script. Right now my script is very basic, it creates a figure, saves it and then exits MATLAB. The command line looks something like this:

   matlab -noFigureWindow -nosplash -minimize -wait -r "cd C:\myDirectory; startup; myScript; exit"

I start it in Jenkins like this:

start /B /WAIT C:\myDirectory\run_matlab.bat

I can see my script running on the Jenkins console, see that the drive has been mounted as requested and see from the Task manager that MATLAB is running, but nothing else happens. The function should take less than a minute to run, but after several minutes nothing happens and I need to kill the job.

Does anyone have any experience with running MATLAB from Jenkins on a Windows slave? I need to run this function on Windows so UNIX is not an option.

Thanks in advance.

Answer 1

[Jason Ross]

I don't have experience with this exact environment, but in terms of running MATLAB on Windows in some sort of headless mode, I've run into the following things you might want to take a look at:

• Who is running the actual executable is very important, especially if you require network access. For example, the LocalSystem account does not have access to network resources by default, so to access a network resource you would need to provide real user credentials to access network resources. This may mean that your password is stored in clear text somewhere (yuck)
• Drive letters and mappings are unique for each user. So if you mount that drive with LocalSystem and then are expecting that drive letter to show up as another user, it's not going to happen (UNC paths would be safer -- but LocalSystem would need to have that set up)
• If a program is running under the context of a "service" (which would not surprise me in this case), the interactions of the program with the "desktop" are limited and you can get weird results if you have things that might need access to the screen / desktop -- like a figure window.

To debug this, I'd try the following:

• Find out who (meaning Windows username) is running the Jenkins service (I assume there is a service running to handle the inbound requests from the remainder of the Jenkins system). Try and execute your script as that user.
• If possible, try it out using only local resources. Don't mount external resources, move the data onto the slave host first and access it from there. See if it can find it.
• See if you use redirection of your script if you can get any more useful output, e.g.

start /B /WAIT C:\myDirectory\run_matlab.bat 1 > c:\temp\mystdout.txt 2> c:\temp\mystderr.txt

The console may be doing this redirection for you already. * Get the code down to the most basic thing that you can. Like "Hello World" basic. See if that will work and then add the other elements from your script and see where it hangs up.

Other errata:

• You mention that you have a script that mounts things, but use C:\myDirectory in the example above. Are you mounting to a directory, or is the drive in question really something like H:?

Answer 2

[Peter]

Thanks for the quick reply, Jason. I'm running in a networked environment where everyone has credentials to run on most machines including the one where the Jenkins service resides. We also use networked drives where my batch script resides, so I guess my example is not really correct. It should read more like:

start /B /WAIT \\networknode\home\username\sandbox\run_matlab.bat

My batch script uses "net use" to mount \\networknode\home as the H: drive. So the call to run the matlab script inside the batch file looks more like:

matlab -noFigureWindow -nosplash -minimize -wait -r "cd %CD%\username\sandbox\matlab; startup; myScript; exit"

where %CD% is the H: drive. At the end of the batch file the H: drive is unmounted.

I'll add the redirects as you suggest and see what kind of results I get.

Thanks again.

[Jason Ross]

Thanks for the update. I'd advise you to look at the properties of the Jenkins service (Computer -> right click -> Manage -> Services, then find the Jenkins service(s) and look at the properties. Check the username there.

Also pay close attention to who is running the script (via Task Manager). Jenkins may incorporate code that allows it to "impersonate" you (kind of like setuid on UNIX), but you want to be really clear on who is running each thing since the drive letters for user A are not the drive letters for user B. So if somehow the username changes between the running of your wrapper script and the MATLAB executable, H: won't be available to the new user.

Also, in MATLAB you can cd to a UNC path (although you can't do this in regular Windows command shell), so you might be able to skip the "net use" part if you are only assigning a drive letter. This would have the benefit of working for any arbitrary username since the UNC path would always be the same, assuming that the account had the ability to reach the network resource.

Answer 3

[Peter]

Hi Jason, It turns out that the problems had more to do with the script I was trying to run and not the Hudson/MATLAB pieces. My script was written to write a figure file and some text to what I thought was a valid location. Because I was writing from a Windows environment to a UNIX server my script was running into permission problems.

I had permission set in the directory I was writing to as drwxrwxr-x (user and group have read/write, everybody else read-only). Once I changed the permissions to allow any one to write to the directory, it worked fine.

Your tip of redirecting stdout and stderr gave me the biggest clue about what I needed to do (that part was failing until I fixed the permissions). Looking at the run output was very helpful and will definitely be a great help as I continue working on this.

Thanks again for your help.

[Jason Ross]

Glad I could be of help! I've had to do a lot of remote and cross-platform debugging during my career. Permissions are always an interesting piece, to be sure.

FWIW you should be able to configure some sort of username and credential mapping between UNIX and Windows. Especially if you have "nobody" showing up on the UNIX side as the UID/GID, this can cause problems (at the absolute worst time, of course) with some downstream process. Ways I've seen this done are as follows:

• Username on Windows maps to username on UNIX (most preferred). Generally you need to configure UNIX to strip the domain name information and you of course also need to have the same basic underlying UID structure (e.g. DOMAINNAME\jross = jross). There is also some configuration to let the UNIX side talk to the Windows side to get UIDs. This works really well because it scales as other people want to use the same resources, and you just need to do the initial setup, after that you rely on existing infrastructure to do your mapping.
• Use a role account to set up the mapping. You set up an account (e.g. jenkinsservice) and configure your systems to accept the mapping and do all the work only as this user. The downsides with a shared account are many (the password is known by more than one person, no clear owner, etc), but there are some upsides (you can control the environment tightly, the role persists as people change jobs, etc)
• Hardcoded list of user IDs that map. Generally quick and dirty, less setup, but doesn't scale well since you have to touch it as people come into the group, leave the group, etc.