# Dimensionality Reduction and Feature Extraction

*Feature transformation* techniques reduce
the dimensionality in the data by transforming data into new features.
*Feature selection* techniques are preferable
when transformation of variables is not possible, e.g., when there
are categorical variables in the data. For a feature selection technique
that is specifically suitable for least-squares fitting, see Stepwise Regression.

## Functions

## Objects

## Topics

### Feature Selection

**Introduction to Feature Selection**

Learn about feature selection algorithms and explore the functions available for feature selection.

This topic introduces to sequential feature selection and provides an example that selects features sequentially using a custom criterion and the `sequentialfs`

function.

**Neighborhood Component Analysis (NCA) Feature Selection**

Neighborhood component analysis (NCA) is a non-parametric method for selecting features with the goal of maximizing prediction accuracy of regression and classification algorithms.

**Robust Feature Selection Using NCA for Regression****Tune Regularization Parameter to Detect Features Using NCA for Classification**

**Regularize Discriminant Analysis Classifier**

Make a more robust and simpler model by removing predictors without compromising the predictive power of the model.

**Select Predictors for Random Forests**

Select split-predictors for random forests using interaction test algorithm.

### Feature Extraction

Feature extraction is a set of methods to extract high-level features from data.

This example shows a complete workflow for feature extraction from image data.

This example shows how to use `rica`

to disentangle mixed audio signals.

### t-SNE Multidimensional Visualization

t-SNE is a method for visualizing high-dimensional data by nonlinear reduction to two or three dimensions, while preserving some features of the original data.

**Visualize High-Dimensional Data Using t-SNE**

This example shows how t-SNE creates a useful low-dimensional embedding of high-dimensional data.

This example shows the effects of various `tsne`

settings.

Output function description and example for t-SNE.

### PCA and Canonical Correlation

**Principal Component Analysis (PCA)**

Principal Component Analysis reduces the dimensionality of data by replacing several correlated variables with a new set of variables that are linear combinations of the original variables.

**Analyze Quality of Life in U.S. Cities Using PCA**

Perform a weighted principal components analysis and interpret the results.

### Factor Analysis

Factor analysis is a way to fit a model to multivariate data to estimate interdependence of measured variables on a smaller number of unobserved (latent) factors.

**Analyze Stock Prices Using Factor Analysis**

Use factor analysis to investigate whether companies within the same sector experience similar week-to-week changes in stock prices.

**Perform Factor Analysis on Exam Grades**

This example shows how to perform factor analysis using Statistics and Machine Learning Toolbox™.

### Nonnegative Matrix Factorization

**Nonnegative Matrix Factorization**

*Nonnegative matrix factorization* (*NMF*) is a dimension-reduction technique based on a low-rank approximation of the feature space.

**Perform Nonnegative Matrix Factorization**

Perform nonnegative matrix factorization using the multiplicative and alternating least-squares algorithms.

### Multidimensional Scaling

Multidimensional scaling allows you to visualize how near points are to each other for many kinds of distance or dissimilarity metrics and can produce a representation of data in a small number of dimensions.

**Classical Multidimensional Scaling**

Use `cmdscale`

to perform classical (metric) multidimensional scaling, also known as principal coordinates analysis.

**Classical Multidimensional Scaling Applied to Nonspatial Distances**

This example shows how to perform classical multidimensional scaling using the `cmdscale`

function in Statistics and Machine Learning Toolbox™.

**Nonclassical Multidimensional Scaling**

This example shows how to visualize dissimilarity data using nonclassical forms of multidimensional scaling (MDS).

**Nonclassical and Nonmetric Multidimensional Scaling**

Perform nonclassical multidimensional scaling using `mdscale`

.

### Procrustes Analysis

Procrustes analysis minimizes the differences in location between compared landmark data using the best shape-preserving Euclidean transformations.

**Compare Handwritten Shapes Using Procrustes Analysis**

Use Procrustes analysis to compare two handwritten numerals.