**Class: **FeatureSelectionNCARegression

Refit neighborhood component analysis (NCA) model for regression

`mdlrefit = refit(mdl,Name,Value)`

refits
the model `mdlrefit`

= refit(`mdl`

,`Name,Value`

)`mdl`

, with modified parameters specified
by one or more `Name,Value`

pair arguments.

`mdl`

— Neighborhood component analysis model for regression`FeatureSelectionNCARegression`

objectNeighborhood component analysis model or classification, specified
as a `FeatureSelectionNCARegression`

object.

Specify optional
comma-separated pairs of `Name,Value`

arguments. `Name`

is
the argument name and `Value`

is the corresponding value.
`Name`

must appear inside quotes. You can specify several name and value
pair arguments in any order as
`Name1,Value1,...,NameN,ValueN`

.

`'FitMethod'`

— Method for fitting the model`mdl.FitMethod`

(default) | `'exact'`

| `'none'`

| `'average'`

Method for fitting the model, specified as the comma-separated
pair consisting of `'FitMethod'`

and one of the following.

`'exact'`

— Performs fitting using all of the data.`'none'`

— No fitting. Use this option to evaluate the generalization error of the NCA model using the initial feature weights supplied in the call to`fsrnca`

.`'average'`

— The function divides the data into partitions (subsets), fits each partition using the`exact`

method, and returns the average of the feature weights. You can specify the number of partitions using the`NumPartitions`

name-value pair argument.

**Example: **`'FitMethod','none'`

`'Lambda'`

— Regularization parameter`mdl.Lambda`

(default) | non-negative scalar valueRegularization parameter, specified as the comma-separated pair
consisting of `'Lambda'`

and a non-negative scalar
value.

For *n* observations, the best `Lambda`

value
that minimizes the generalization error of the NCA model is expected
to be a multiple of 1/*n*

**Example: **`'Lambda',0.01`

**Data Types: **`double`

| `single`

`'Solver'`

— Solver type`mdl.Solver`

(default) | `'lbfgs'`

| `'sgd'`

| `'minibatch-lbfgs'`

Solver type for estimating feature weights, specified as the
comma-separated pair consisting of `'Solver'`

and
one of the following.

`'lbfgs'`

— Limited memory BFGS (Broyden-Fletcher-Goldfarb-Shanno) algorithm (LBFGS algorithm)`'sgd'`

— Stochastic gradient descent`'minibatch-lbfgs'`

— Stochastic gradient descent with LBFGS algorithm applied to mini-batches

**Example: **`'solver','minibatch-lbfgs'`

`'InitialFeatureWeights'`

— Initial feature weights`mdl.InitialFeatureWeights`

(default) | Initial feature weights, specified as the comma-separated pair
consisting of `'InitialFeatureWeights'`

and a *p*-by-1
vector of real positive scalar values.

**Data Types: **`double`

| `single`

`'Verbose'`

— Indicator for verbosity level`mdl.Verbose`

(default) | 0 | 1 | >1Indicator for verbosity level for the convergence summary display,
specified as the comma-separated pair consisting of `'Verbose'`

and
one of the following.

0 — No convergence summary

1 — Convergence summary including iteration number, norm of the gradient, and objective function value.

>1 — More convergence information depending on the fitting algorithm

When using solver

`'minibatch-lbfgs'`

and verbosity level >1, the convergence information includes iteration log from intermediate minibatch LBFGS fits.

**Example: **`'Verbose',2`

**Data Types: **`double`

| `single`

`'GradientTolerance'`

— Relative convergence tolerance`mdl.GradientTolerance`

(default) | positive real scalar valueRelative convergence tolerance on the gradient norm for solver `lbfgs`

,
specified as the comma-separated pair consisting of `'GradientTolerance'`

and
a positive real scalar value.

**Example: **`'GradientTolerance',0.00001`

**Data Types: **`double`

| `single`

`'InitialLearningRate'`

— Initial learning rate for solver `sgd`

`mdl.InitialLearningRate`

(default) | positive real scalar valueInitial learning rate for solver `sgd`

, specified
as the comma-separated pair consisting of `'InitialLearningRate'`

and
a positive scalar value.

When using solver type `'sgd'`

, the learning
rate decays over iterations starting with the value specified for `'InitialLearningRate'`

.

**Example: **`'InitialLearningRate',0.8`

**Data Types: **`double`

| `single`

`'PassLimit'`

— Maximum number of passes for solver `'sgd'`

`mdl.PassLimit`

(default) | positive integer value Maximum number of passes for solver `'sgd'`

(stochastic
gradient descent), specified as the comma-separated pair consisting
of `'PassLimit'`

and a positive integer. Every pass
processes `size(mdl.X,1)`

observations.

**Example: **`'PassLimit',10`

**Data Types: **`double`

| `single`

`'IterationLimit'`

— Maximum number of iterations`mdl.IterationLimit`

(default) | positive integer valueMaximum number of iterations, specified as the comma-separated
pair consisting of `'IterationLimit'`

and a positive
integer.

**Example: **`'IterationLimit',250`

**Data Types: **`double`

| `single`

`mdlrefit`

— Neighborhood component analysis model for regression`FeatureSelectionNCARegression`

objectNeighborhood component analysis model or classification, returned as a `FeatureSelectionNCARegression`

object. You can either save the
results as a new model or update the existing model as ```
mdl =
refit(mdl,Name,Value)
```

.

Load the sample data.

load(fullfile(matlabroot,'examples','stats','robotarm.mat'))

The `robotarm`

(pumadyn32nm) dataset is created using a robot arm simulator with 7168 training and 1024 test observations with 32 features [1], [2]. This is a preprocessed version of the original data set. Data are preprocessed by subtracting off a linear regression fit followed by normalization of all features to unit variance.

Compute the generalization error without feature selection.

nca = fsrnca(Xtrain,ytrain,'FitMethod','none','Standardize',1); L = loss(nca,Xtest,ytest)

L = 0.9017

Now, refit the model and compute the prediction loss with feature selection, with $$\lambda $$ = 0 (no regularization term) and compare to the previous loss value, to determine feature selection seems necessary for this problem. For the settings that you do not change, `refit`

uses the settings of the initial model `nca`

. For example, it uses the feature weights found in `nca`

as the initial feature weights.

nca2 = refit(nca,'FitMethod','exact','Lambda',0); L2 = loss(nca2,Xtest,ytest)

L2 = 0.1088

The decrease in the loss suggests that feature selection is necessary.

Plot the feature weights.

```
figure()
plot(nca2.FeatureWeights,'ro')
```

Tuning the regularization parameter usually improves the results. Suppose that, after tuning $$\lambda $$ using cross-validation as in Tune Regularization Parameter in NCA for Regression, the best $$\lambda $$ value found is 0.0035. Refit the `nca`

model using this $$\lambda $$ value and stochastic gradient descent as the solver. Compute the prediction loss.

nca3 = refit(nca2,'FitMethod','exact','Lambda',0.0035,... 'Solver','sgd'); L3 = loss(nca3,Xtest,ytest)

L3 = 0.0573

Plot the feature weights.

```
figure()
plot(nca3.FeatureWeights,'ro')
```

After tuning the regularization parameter, the loss decreased even more and the software identified four of the features as relevant.

**References**

[1] Rasmussen, C. E., R. M. Neal, G. E. Hinton, D. van Campand, M. Revow, Z. Ghahramani, R. Kustra, R. Tibshirani. The DELVE Manual, 1996, http://mlg.eng.cam.ac.uk/pub/pdf/RasNeaHinetal96.pdf

A modified version of this example exists on your system. Do you want to open this version instead?

You clicked a link that corresponds to this MATLAB command:

Run the command by entering it in the MATLAB Command Window. Web browsers do not support MATLAB commands.

Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .

Select web siteYou can also select a web site from the following list:

Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.

- América Latina (Español)
- Canada (English)
- United States (English)

- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)

- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)