discardSupportVectors

R2026b

Discard support vectors for linear support vector machine (SVM) regression model

Syntax

MdlWithoutSV = discardSupportVectors(Mdl)

Description

MdlWithoutSV = discardSupportVectors(Mdl) returns the trained, linear support vector machine (SVM) regression model MdlWithoutSV, which is similar to the trained, linear SVM regression model Mdl, except for the following:

• The Alpha and SupportVectors properties are empty ([]).

• If you display MdlWithoutSV, the software lists the Beta property instead of the Alpha property.

Examples

Discard Support Vectors for SVM Regression Model

Reduce the disk space used by a trained, linear SVM regression model by discarding the support vectors and other related parameters.

Load the carsmall data set. Specify Horsepower and Weight as the predictor variables (X), and MPG as the response variable (Y).

load carsmall
X = [Horsepower,Weight];
Y = MPG;

Train a linear SVM regression model, standardizing the data. Display the number of support vectors.

Mdl = fitrsvm(X,Y,Standardize=true)

Mdl = 
  RegressionSVM
             ResponseName: 'Y'
    CategoricalPredictors: []
        ResponseTransform: 'none'
                    Alpha: [77×1 double]
                     Bias: 22.9131
         KernelParameters: [1×1 struct]
                       Mu: [109.3441 2.9625e+03]
                    Sigma: [45.3545 805.9668]
          NumObservations: 94
           BoxConstraints: [94×1 double]
          ConvergenceInfo: [1×1 struct]
          IsSupportVector: [94×1 logical]
                   Solver: 'SMO'


  Properties, Methods

numSV = size(Mdl.SupportVectors,1)

numSV = 
77

By default, fitrsvm trains a linear SVM regression model. The software lists Alpha in the display. The model has 77 support vectors.

Note that the predictor and response variables contain several NaN values. When training a model, fitrsvm removes rows that contain NaN values from both the predictor and response data. As a result, the trained model uses only 93 of the 100 total observations contained in the sample data.

Discard the support vectors and other related parameters.

MdlOut = discardSupportVectors(Mdl)

MdlOut = 
  RegressionSVM
             ResponseName: 'Y'
    CategoricalPredictors: []
        ResponseTransform: 'none'
                     Beta: [2×1 double]
                     Bias: 22.9131
         KernelParameters: [1×1 struct]
                       Mu: [109.3441 2.9625e+03]
                    Sigma: [45.3545 805.9668]
          NumObservations: 94
           BoxConstraints: [94×1 double]
          ConvergenceInfo: [1×1 struct]
          IsSupportVector: [94×1 logical]
                   Solver: 'SMO'


  Properties, Methods

MdlOut.Alpha

ans =

     []

MdlOut.SupportVectors

ans =

     []

The software lists Beta in the display instead of Alpha. The Alpha and SupportVectors properties are empty.

Compare the sizes of the models.

whos("Mdl","MdlOut")

  Name        Size            Bytes  Class            Attributes

  Mdl         1x1             15183  RegressionSVM              
  MdlOut      1x1             13335  RegressionSVM              

MdlOut consumes less memory than Mdl because it does not store the support vectors.

Reduce Memory Consumption of SVM Regression Model

Reduce the memory consumption of a full, trained SVM regression model by compacting the model and discarding the support vectors.

Load the carsmall sample data.

load carsmall
rng(0,"twister") % For reproducibility

Train a linear SVM regression model using Weight as the predictor variable and MPG as the response variable. Standardize the data.

Mdl = fitrsvm(Weight,MPG,Standardize=true);

Note that MPG contains several NaN values. When training a model, fitrsvm removes rows that contain NaN values from both the predictor and response data. As a result, the trained model uses only 94 of the 100 total observations contained in the sample data.

Compact the regression model to discard the training data and some information related to the training process.

compactMdl = compact(Mdl);

compactMdl is a CompactRegressionSVM model that has the same parameters, support vectors, and related estimates as Mdl, but no longer stores the training data.

Discard the support vectors and related estimates for the compact model.

noSupportVectorsMdl = discardSupportVectors(compactMdl);

noSupportVectorsMdl is a CompactRegressionSVM model that has the same parameters as Mdl and compactMdl, but no longer stores the support vectors and related estimates.

Compare the sizes of the three SVM regression models: Mdl, compactMdl, and noSupportVectorsMdl.

whos("Mdl","compactMdl","noSupportVectorsMdl")

  Name                     Size            Bytes  Class                                          Attributes

  Mdl                      1x1             13779  RegressionSVM                                            
  compactMdl               1x1              4564  classreg.learning.regr.CompactRegressionSVM              
  noSupportVectorsMdl      1x1              3268  classreg.learning.regr.CompactRegressionSVM              

The full model Mdl consumes more memory than the models compactMdl and noSupportVectorsMdl.

Input Arguments

Mdl — Trained, linear SVM regression model

RegressionSVM model object | CompactRegressionSVM model object

Trained, linear SVM regression model, specified as a RegressionSVM or CompactRegressionSVM model object.

If you train the model using a kernel function that is not linear (that is, if the field Mdl.KernelFunction is something other than 'linear'), the software returns an error. You can only discard support vectors for linear models.

Output Arguments

MdlWithoutSV — Trained, linear SVM regression model

RegressionSVM model object | CompactRegressionSVM model object

Trained, linear SVM regression model, returned as a RegressionSVM or CompactRegressionSVM model object. MdlWithoutSV is the same type as Mdl.

After discarding the support vectors, the properties Alpha and SupportVectors are empty ([]). The software lists the property Beta in its display, and does not list the property Alpha. The predict and resubPredict object functions compute predicted responses using the coefficients stored in the Beta property.

Tips

For a trained, linear SVM regression model, the SupportVectors property is an n_sv-by-p matrix. n_sv is the number of support vectors (at most the training sample size) and p is the number of predictor variables. If any of the predictors are categorical, then p includes the number of dummy variables necessary to account for all of the categorical predictor levels. The Alpha property is a vector with n_sv elements.

The SupportVectors and Alpha properties can be large for complex data sets that contain many observations or examples. However, the Beta property is a vector with p elements, which may be considerably smaller. You can use a trained SVM regression model to predict response values even if you discard the support vectors because the predict and resubPredict object functions use Beta to compute the predicted responses.

If the trained, linear SVM regression model has many support vectors, use discardSupportVectors to reduce the amount of disk space that the trained, linear SVM regression model consumes. You can display the size of the support vector matrix by entering size(Mdl.SupportVectors).

Algorithms

The predict and resubPredict functions estimate response values using the formula

$$f\left(x\right)=\left(\frac{X}{S}\right)\beta +{\beta }_0,$$

where:

• β is the Beta value, stored as Mdl.Beta.

• β₀ is the bias value, stored as Mdl.Bias.

• X is the training data.

• S is the kernel scale value, stored as Mdl.KernelParameters.Scale.

In this way, the software can use the value of Mdl.Beta to make predictions even after discarding the support vectors.

Extended Capabilities

GPU Arrays

Accelerate code by running on a graphics processing unit (GPU) using Parallel Computing Toolbox™. (since R2023a)

This function fully supports GPU arrays. For more information, see Run MATLAB Functions on a GPU (Parallel Computing Toolbox).

Version History

Introduced in R2015b

R2023a: GPU array support

Starting in R2023a, discardSupportVectors fully supports GPU arrays.