RegressionQuantileNeuralNetwork

Quantile neural network model for regression

Since R2024b

Description

A RegressionQuantileNeuralNetwork object is a trained quantile neural network regression model. The first fully connected layer of the neural network has a connection from the network input (predictor data X), and each subsequent layer has a connection from the previous layer. Each fully connected layer multiplies the input by a weight matrix (LayerWeights) and then adds a bias vector (LayerBiases). An activation function follows each fully connected layer, excluding the last (Activations and OutputLayerActivation). The final fully connected layer produces the network's output, the predicted response values for each quantile (Quantiles).

After training a RegressionQuantileNeuralNetwork model object, you can use the loss object function to compute the quantile loss, and the predict object function to predict the response for new data.

Creation

Create a RegressionQuantileNeuralNetwork object by using the fitrqnet function.

Properties

Neural Network Properties

Quantiles — Quantiles

0.5 (default) | vector of values in the range [0,1]

This property is read-only.

Quantiles used to train the quantile neural network regression model, returned as a vector of values in the range [0,1].

Data Types: double

LayerSizes — Sizes of fully connected layers

positive integer vector

This property is read-only.

Sizes of the fully connected layers in the quantile neural network regression model, returned as a positive integer vector. Element i of LayerSizes is the number of outputs in the fully connected layer i of the model.

LayerSizes does not include the size of the final fully connected layer. This layer always has one output for each quantile in Quantiles.

Data Types: single | double

LayerWeights — Learned layer weights

cell array

This property is read-only.

Learned layer weights for the fully connected layers, returned as a cell array. Entry i in the cell array corresponds to the layer weights for the fully connected layer i. For example, Mdl.LayerWeights{1} returns the weights for the first fully connected layer of the model Mdl.

LayerWeights includes the weights for the final fully connected layer.

Data Types: cell

LayerBiases — Learned layer biases

cell array

This property is read-only.

Learned layer biases for the fully connected layers, returned as a cell array. Entry i in the cell array corresponds to the layer biases for the fully connected layer i. For example, Mdl.LayerBiases{1} returns the biases for the first fully connected layer of the model Mdl.

LayerBiases includes the biases for the final fully connected layer.

Data Types: cell

Activations — Activation functions for fully connected layers

'relu' | 'tanh' | 'sigmoid' | 'none' | cell array of character vectors

This property is read-only.

Activation functions for the fully connected layers of the quantile neural network regression model, returned as a character vector or cell array of character vectors with values from this table.

Value	Description
"relu"	Rectified linear unit (ReLU) function — Performs a threshold operation on each element of the input, where any value less than zero is set to zero, that is, $$f\left(x\right)=\{\begin{array}{cc}x,& x\ge 0\\ 0,& x<0\end{array}$$
"tanh"	Hyperbolic tangent (tanh) function — Applies the tanh function to each input element
"sigmoid"	Sigmoid function — Performs the following operation on each input element: $$f(x)=\frac{1}{1+e^{-x}}$$
"none"	Identity function — Returns each input element without performing any transformation, that is, f(x) = x

• If Activations contains only one activation function, then it is the activation function for every fully connected layer of the model, excluding the final fully connected layer, which does not have an activation function (OutputLayerActivation).

• If Activations is an array of activation functions, then element i is the activation function for layer i of the model.

Data Types: char | cell

OutputLayerActivation — Activation for final fully connected layer

'none'

This property is read-only.

Activation function for the final fully connected layer, returned as 'none'.

Data Types: char

Lambda — Regularization term strength

nonnegative scalar

This property is read-only.

Regularization term strength for the ridge (L2) penalty, returned as a nonnegative scalar.

Data Types: double | single

Solver — Solver used to train model

'LBFGS'

This property is read-only.

Solver used to train the quantile neural network regression model, returned as 'LBFGS'.

Data Types: char

ModelParameters — Parameter values used to train model

NeuralNetworkParams object

This property is read-only.

Parameter values used to train the quantile neural network regression model, returned as a NeuralNetworkParams object. ModelParameters contains parameter values such as the name-value arguments used to train the model.

Access the properties of ModelParameters by using dot notation. For example, access the function used to initialize the fully connected layer weights of a model Mdl by using Mdl.ModelParameters.LayerWeightsInitializer.

ConvergenceInfo — Convergence information

structure array

This property is read-only.

Convergence information, returned as a structure array.

Field	Description
Iterations	Number of training iterations used to train the quantile neural network regression model
TrainingLoss	Training mean squared error (MSE) for the returned model
Gradient	Gradient of the loss function with respect to the weights and biases at the iteration corresponding to the returned model
Step	Step size at the iteration corresponding to the returned model
Time	Total time spent across all iterations (in seconds)
ValidationLoss	Validation MSE for the returned model
ValidationChecks	Maximum number of consecutive times that the validation loss was greater than or equal to the minimum validation loss
ConvergenceCriterion	Criterion for convergence
History	Table of training history

Data Types: struct

Predictor Properties

PredictorNames — Predictor variable names

cell array of character vectors

This property is read-only.

Predictor variable names, returned as a cell array of character vectors. The order of the elements of PredictorNames corresponds to the order in which the predictor names appear in the training data.

Data Types: cell

CategoricalPredictors — Categorical predictor indices

vector of positive integers | []

This property is read-only.

Categorical predictor indices, returned as a vector of positive integers. Assuming that the predictor data contains observations in rows, CategoricalPredictors contains index values corresponding to the columns of the predictor data that contain categorical predictors. If none of the predictors are categorical, then this property is empty ([]).

Data Types: double

ExpandedPredictorNames — Expanded predictor names

cell array of character vectors

This property is read-only.

Expanded predictor names, returned as a cell array of character vectors. If the model uses encoding for categorical variables, then ExpandedPredictorNames includes the names that describe the expanded variables. Otherwise, ExpandedPredictorNames is the same as PredictorNames.

Data Types: cell

Mu — Predictor means

numeric vector | []

This property is read-only.

Predictor means, returned as a numeric vector. If you set Standardize to 1 or true when you train the neural network model, then the length of the Mu vector is equal to the number of expanded predictors (ExpandedPredictorNames). The vector contains 0 values for dummy variables corresponding to expanded categorical predictors.

If you set Standardize to 0 or false when you train the neural network model, then the Mu value is an empty vector ([]).

Data Types: double

Sigma — Predictor standard deviations

numeric vector | []

This property is read-only.

Predictor standard deviations, returned as a numeric vector. If you set Standardize to 1 or true when you train the neural network model, then the length of the Sigma vector is equal to the number of expanded predictors (ExpandedPredictorNames). The vector contains 1 values for dummy variables corresponding to expanded categorical predictors.

If you set Standardize to 0 or false when you train the neural network model, then the Sigma value is an empty vector ([]).

Data Types: double

X — Unstandardized predictors

numeric matrix | table

This property is read-only.

Unstandardized predictors used to train the neural network model, returned as a numeric matrix or table. X retains its original orientation, with observations in rows or columns depending on the value of the ObservationsIn name-value argument in the call to fitrqnet.

Data Types: single | double | table

Response Properties

ResponseName — Response variable name

character vector

This property is read-only.

Response variable name, returned as a character vector.

Data Types: char

Y — Response values

numeric vector

This property is read-only.

Response values used to train the model, returned as a numeric vector. Each row of Y represents the response value of the corresponding observation in X.

Data Types: single | double

ResponseTransform — Response transformation function

'none' | function handle

Response transformation function, specified as 'none' or a function handle. ResponseTransform describes how the software transforms raw response values.

For a MATLAB^® function or a function that you define, enter its function handle. For example, you can enter Mdl.ResponseTransform = @function, where function accepts a numeric vector of the original responses and returns a numeric vector of the same size containing the transformed responses.

Data Types: char | function_handle

Other Data Properties

NumObservations — Number of observations

positive numeric scalar

This property is read-only.

Number of observations in the training data stored in X and Y, returned as a positive numeric scalar.

Data Types: double

RowsUsed — Observations of original training data stored

logical vector | []

This property is read-only.

Observations of the original training data stored in the model, returned as a logical vector. This property is empty if all observations are stored in X and Y.

Data Types: logical

W — Observation weights

numeric vector

This property is read-only.

Observation weights used to train the model, returned as an n-by-1 numeric vector. n is the number of observations (NumObservations).

The software normalizes the observation weights specified by the Weights name-value argument in the call to fitrqnet so that the elements of W sum to 1.

Data Types: single | double

Object Functions

loss	Loss for quantile neural network regression model
predict	Predict response for quantile neural network regression model

Examples

Fit Quantile Neural Network Regression Model

Fit a quantile neural network regression model using the 0.25, 0.50, and 0.75 quantiles.

Load the carbig data set, which contains measurements of cars made in the 1970s and early 1980s. Create a matrix X containing the predictor variables Acceleration, Displacement, Horsepower, and Weight. Store the response variable MPG in the variable Y.

load carbig
X = [Acceleration,Displacement,Horsepower,Weight];
Y = MPG;

Delete rows of X and Y where either array has missing values.

R = rmmissing([X Y]);
X = R(:,1:end-1);
Y = R(:,end);

Partition the data into training data (XTrain and YTrain) and test data (XTest and YTest). Reserve approximately 20% of the observations for testing, and use the rest of the observations for training.

rng(0,"twister") % For reproducibility of the partition
c = cvpartition(length(Y),"Holdout",0.20);
 
trainingIdx = training(c);
XTrain = X(trainingIdx,:);
YTrain = Y(trainingIdx);
 
testIdx = test(c);
XTest = X(testIdx,:);
YTest = Y(testIdx);

Train a quantile neural network regression model. Specify to use the 0.25, 0.50, and 0.75 quantiles (that is, the lower quartile, median, and upper quartile). To improve the model fit, standardize the numeric predictors. Use a ridge (L2) regularization term of 1. Adding a regularization term can help prevent quantile crossing.

Mdl = fitrqnet(XTrain,YTrain,Quantiles=[0.25,0.50,0.75], ...
    Standardize=true,Lambda=0.05)

Mdl = 
  RegressionQuantileNeuralNetwork
             ResponseName: 'Y'
    CategoricalPredictors: []
               LayerSizes: 10
              Activations: 'relu'
    OutputLayerActivation: 'none'
                Quantiles: [0.2500 0.5000 0.7500]

Mdl is a RegressionQuantileNeuralNetwork model object. You can use dot notation to access the properties of Mdl. For example, Mdl.LayerWeights and Mdl.LayerBiases contain the weights and biases, respectively, for the fully connected layers of the trained model.

In this example, you can use the layer weights, layer biases, predictor means, and predictor standard deviations directly to predict the test set responses for each of the three quantiles in Mdl.Quantiles. In general, you can use the predict object function to make quantile predictions.

firstFCStep = (Mdl.LayerWeights{1})*((XTest-Mdl.Mu)./Mdl.Sigma)' ...
    + Mdl.LayerBiases{1};
reluStep = max(firstFCStep,0);
finalFCStep = (Mdl.LayerWeights{end})*reluStep + Mdl.LayerBiases{end};
predictedY = finalFCStep'

predictedY = 78×3

   13.9602   15.1340   16.6884
   11.2792   12.2332   13.4849
   19.5525   21.7303   23.9473
   22.6950   25.5260   28.1201
   10.4533   11.3377   12.4984
   17.6935   19.5194   21.5152
   12.4312   13.4797   14.8614
   11.7998   12.7963   14.1071
   16.6860   18.3305   20.2070
   24.1142   27.0301   29.7811
      ⋮

isequal(predictedY,predict(Mdl,XTest))

ans = logical
   1

Each column of predictedY corresponds to a separate quantile (0.25, 0.5, or 0.75).

Visualize the predictions of the quantile neural network regression model. First, create a grid of predictor values.

minX = floor(min(X))

minX = 1×4

           8          68          46        1613

maxX = ceil(max(X))

maxX = 1×4

          25         455         230        5140

gridX = zeros(100,size(X,2));
for p = 1:size(X,2)
    gridp = linspace(minX(p),maxX(p))';
    gridX(:,p) = gridp;
end

Next, use the trained model Mdl to predict the response values for the grid of predictor values.

gridY = predict(Mdl,gridX)

gridY = 100×3

   31.2419   35.0661   38.6357
   30.8637   34.6317   38.1573
   30.4854   34.1972   37.6789
   30.1072   33.7627   37.2005
   29.7290   33.3283   36.7221
   29.3507   32.8938   36.2436
   28.9725   32.4593   35.7652
   28.5943   32.0249   35.2868
   28.2160   31.5904   34.8084
   27.8378   31.1560   34.3300
      ⋮

For each observation in gridX, the predict object function returns predictions for the quantiles in Mdl.Quantiles.

View the gridY predictions for the second predictor (Displacement). Compare the quantile predictions to the true test data values.

predictorIdx = 2;
plot(XTest(:,predictorIdx),YTest,".")
hold on
plot(gridX(:,predictorIdx),gridY(:,1))
plot(gridX(:,predictorIdx),gridY(:,2))
plot(gridX(:,predictorIdx),gridY(:,3))
hold off
xlabel("Predictor (Displacement)")
ylabel("Response (MPG)")
legend(["True values","0.25 predicted values", ...
    "0.50 predicted values","0.75 predicted values"])
title("Test Data")

The red curve shows the predictions for the 0.25 quantile, the yellow curve shows the predictions for the 0.50 quantile, and the purple curve shows the predictions for the 0.75 quantile. The blue points indicate the true test data values.

Notice that the quantile prediction curves do not cross each other.

Version History

Introduced in R2024b