Running sbatch in HPC using "system command" in MATLAB script is not working.

I have a problem running a .sbatch file inside a matlab script using "system" command

The .sbatch files (my.sbatch) are something like:

----------------------------------

#!/bin/bash
#SBATCH --job-name=Al_iter1
#SBATCH --partition=longjobs
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=2

module load parallel

srun parallel ./run.sh   t{1}_g{2}_z{3}_v{4}_x13{5} ::: 3547 ::: 4.9 ::: +0.17 ::: 1.00 ::: -0.35 -0.34 -0.33 -0.32 -0.31 -0.30

---------------------------------

where run.sh is a shell script for running a spectral synthesis code.

When I run the command below directly at the terminal in the HPC:

sbatch my.sbatch

there is no problem and the run is perfectly performed, no matter how many CPUs (from 2 to 20) I request (#SBATCH --cpus-per-task). However, I am automating a long code and need to run such sbatch runs inside a matlab script. I am using

system('sbatch my.sbatch')

in my matlab script. But I get a strange error as below:

-------------------------------

srun: error: CPU binding outside of job step allocation, allocated CPUs are: 0x000400000400.

srun: error: Task launch for StepId=370.0 failed on node cn001: Unable to satisfy cpu bind request

srun: error: Application launch failed: Unable to satisfy cpu bind request

-------------------------------

even I request for 1 CPU. I think the problem originates from the way I call the system command. I would be thankful if someone would help me with this issue.

Answer 1

[Namnendra]

Hi Neda,

I understand that you are facing issues with running a .sbatch file inside a matlab script using "system" command.

The issue you're encountering might be related to how the `system` command in MATLAB interacts with the environment and resource allocation on the HPC system. Here are some steps you can take to troubleshoot and resolve the issue:

1. Check Environment Variables

Ensure that the environment variables and modules loaded in your terminal session are also available when MATLAB runs the `system` command. Sometimes, the environment within MATLAB might differ from the one in your terminal.

2. Use Full Paths

Ensure that you use full paths for the `sbatch` command and the `.sbatch` file within MATLAB. This helps avoid any issues related to the current working directory.

3. Debugging Output

Capture and print the output of the `system` command to see more detailed error messages. This can help you diagnose what might be going wrong.

Example MATLAB Script

Here's an example MATLAB script that incorporates these suggestions:

% Full path to the sbatch command and the .sbatch file
sbatchCmd = '/usr/bin/sbatch';  % Adjust this path as necessary
sbatchFile = '/path/to/my.sbatch';  % Adjust this path as necessary

% Construct the command string
cmd = sprintf('%s %s', sbatchCmd, sbatchFile);

% Execute the system command and capture the output
[status, result] = system(cmd);

% Display the status and result
disp(['Status: ', num2str(status)]);
disp(['Result: ', result]);

% Check for errors
if status ~= 0
    error('Failed to submit job: %s', result);
end

4. Use a Wrapper Script

If the above suggestions do not resolve the issue, consider creating a wrapper shell script that sets up the environment and then calls `sbatch`. This script can be invoked from MATLAB.

Wrapper Script Example (`submit_job.sh`):

#!/bin/bash
# Load necessary modules
module load parallel

# Submit the sbatch job
sbatch /path/to/my.sbatch

MATLAB Script to Call Wrapper:

% Full path to the wrapper script
wrapperScript = '/path/to/submit_job.sh';  % Adjust this path as necessary

% Execute the system command and capture the output
[status, result] = system(wrapperScript);

% Display the status and result
disp(['Status: ', num2str(status)]);
disp(['Result: ', result]);

% Check for errors
if status ~= 0
    error('Failed to submit job: %s', result);
end

5. Check CPU Binding and Resource Allocation

The error message suggests an issue with CPU binding. Ensure that the resources requested in the `.sbatch` file are appropriate and available on the HPC system.

6. Verify Permissions

Ensure that MATLAB has the necessary permissions to execute the `sbatch` command and that the `.sbatch` file is accessible from the MATLAB environment.

By following these steps, you should be able to diagnose and resolve the issue with running `.sbatch` files from within a MATLAB script.

I hope above information helps you.

Thank you.