Debug Experiments for Deep Learning

In Experiment Manager, you use functions to configure the training data, network architecture, and training options for the experiment, specify the training procedure used by the experiment, and evaluate the results of the experiment. To debug these functions, set breakpoints in the Live Editor and call the functions in the MATLAB Command Window. The inputs you use for calling Experiment Manager functions differ based on whether you are debugging a built-in training experiment or a custom training experiment.

Debug Built-In Training Experiments

Experiments that use the built-in training function trainNetwork contain two types of functions, setup functions and metric functions. Calling the setup function, the trainNetwork function, and metric functions from the MATLAB Command Window is analogous to running a single trial in Experiment Manager.

Find Errors in Setup Functions

The setup function configures the training data, network architecture, and training options for the experiment. For more information, see Configure Built-In Training Experiment. To diagnose problems in your setup function, follow these steps:

1. Open the experiment.

   

2. Under Setup Function, click Edit. The setup function opens in the Live Editor.

3. In the Live Editor, set breakpoints in the setup function as described in Set Breakpoints.

   

4. In the MATLAB Command Window, create a structure with a field for each hyperparameter defined in the experiment. Assign a value to each field from the range indicated for the corresponding hyperparameter. For example, if your experiment has two hyperparameters called Probability and Filters, enter:

   params = struct(Probability=0.1,Filters=8);

5. Call the setup function using the hyperparameter structure as the input to the function. For example, if the setup function for your experiment is called RegressionExperiment_setup1 and has four outputs, enter:

   [XTrain,YTrain,layers,options] = RegressionExperiment_setup1(params);

6. MATLAB pauses at each line of code indicated by a breakpoint. When the function execution stops at a breakpoint, you can view the values of your variables, step through the code line by line, or continue to the next breakpoint. For more information, see Examine Values While Debugging and Debug MATLAB Code Files. After the function runs to completion, you can examine the output values. For example, you can visualize the network layers in the Deep Network Designer app and inspect the training data and training options in the MATLAB Workspace browser.

Find Errors in Metric Functions

Custom metric functions are used to evaluate the results of an experiment. For more information, see Evaluate Deep Learning Experiments by Using Metric Functions. To diagnose problems in your metric function, follow these steps:

1. In the MATLAB Command Window, create a hyperparameter structure and call the setup function, as described in Find Errors in Setup Functions. The output of the setup function must match the input of the trainNetwork function.

2. Use the output of the setup function to call trainNetwork. For example, if your setup function has four outputs, enter:

   [tnet,tInfo] = trainNetwork(XTrain,YTrain,layers,options);

3. In the experiment, under Metrics, select the name of the metric function and click Edit. The metric function opens in the Live Editor.

4. In the Live Editor, set breakpoints in the metric function as described in Set Breakpoints.

   

5. Create a structure called trialInfo that contains the fields trainedNetwork, trainingInfo, and parameters. For values, use the outputs of the trainNetwork function and your hyperparameter structure.

   trialInfo = struct(trainedNetwork=tnet, ...
       trainingInfo=tInfo, ...
       parameters=params);

6. Call the metric function using the trialInfo structure as the input to the function. For example, if your metric function is called Accuracy, enter:

   metricOutput = Accuracy(trialInfo);

7. MATLAB pauses at each line of code indicated by a breakpoint. When the function execution stops at the breakpoint, you can view the values of your variables, step through the code line by line, or continue to the next breakpoint. For more information, see Examine Values While Debugging and Debug MATLAB Code Files. After the function runs to completion, you can examine the output value. The output must be a scalar number, a logical value, or a string.

Debug Custom Training Experiments

Custom experiments contain a training function that specifies the training data, network architecture, training options, and training procedure used by the experiment. For more information, see Configure Custom Training Experiment. Calling the training function from the MATLAB Command Window is analogous to running a single trial in Experiment Manager. To diagnose problems in your training function, follow these steps:

1. Open the experiment.

   

2. Under Training Function, click Edit. The training function opens in the Live Editor.

3. In the Live Editor, set breakpoints in the training function as described in Set Breakpoints.

   

4. In the MATLAB Command Window, create a structure with a field for each hyperparameter defined in the experiment. Assign a value to each field from the range indicated for the corresponding hyperparameter. For example, if your experiment has two hyperparameters called WeightsInitializer and BiasInitializer, enter:

   params = struct(WeightsInitializer="he", ...
       BiasInitializer="narrow-normal");

5. Create an experiments.Monitor object.

   monitor = experiments.Monitor;

6. Call the training function using the hyperparameter structure and the experiments.Monitor object as the inputs to the function. For example, if your training function is called ImageComparisonExperiment_training1, enter:

   net = ImageComparisonExperiment_training1(params,monitor);

7. MATLAB pauses at each line of code indicated by a breakpoint. When the function execution stops at the breakpoint, you can view the values of your variables, step through the code line by line, or continue to the next breakpoint. For more information, see Examine Values While Debugging and Debug MATLAB Code Files. After the function runs to completion, you can examine the output values to verify your results.