Backward and Forward stepwise regression?

i'm using stepwiselm to find a feature subset. I know the function uses a combination of forward and backward elimination. I need to know how to do only forward and only backward. I thin kit has something to do with the PEnter and PRemove but couldn't figure it out. Also I don't wanna go higher than linear for upepr bound. This is how I've used the function:

mdl = stepwiselm(X,y,'constant','Upper','linear');

Answer 1

[Manikanta Aditya]

Hi Azarang,

As per my understanding, you would like to know how to do either forward or backward elimination in stepwise regression.

You can control the direction of selection by setting the Probability to Enter(‘PEnter’) and Probability to Remove(‘PRemove’) values to control the significance level of adding or removing feature respectively.

• For Forward elimination - You can set larger values for Probability to Remove(‘PRemove’) such as ‘PRemove=1’

mdl = stepwiselm(X,y,’constant’,’Upper’,’linear’,’PRemove’,1);  

• For Backward elimination - You can set smaller values for Probability to Enter(‘PEnter’)such as ‘PEnter=0.05’

mdl = stepwiselm(X,y,’constant’,’Upper’,’linear’,’PEnter’,0.05);  

This means that only features with a Probability value (p-value) less than 0.05 will be considered for addition, effectively turning off the forward selection step.

Note that if you set both 'PEnter’ and ‘PRemove’ to small values, you may end up with no selected features or all features selected, depending on the significance level and the strength of the relationships between the features and the response variable.

For further reference, to check more about ‘stepwiselm’ function please go through this link:

https://www.mathworks.com/help/stats/stepwiselm.html

I hope this resolves the issue you were facing.

[azarang asadi]

okay, I tried this and I get same results using forward and backward. you said this is backward :

mdl = stepwiselm(X,y,’constant’,’Upper’,’linear’,’PEnter’,0.05);

but doesn't this says to start with the constant model? isn't it should look like this:

mdl = stepwiselm(X,y,’linear’,’Upper’,’linear’,’PEnter’,0.05);

[Manikanta Aditya]

The correct code to perform stepwise regression with forward selection in MATLAB would be:

mdl = stepwiselm(X, y, 'linear', 'Upper', 'linear', 'PEnter', 0.05);

This code will start with a simple linear model and use forward selection to add variables to the model until the stopping criteria (specified by the 'PEnter' parameter) are met. The 'Upper' parameter specifies the maximum number of predictor variables to include in the model.

Thank you for bringing this to my attention, and please let me know if you have any further questions!

[azarang asadi]

I think you are saying it wrong, this is backward not forwad. Forward starts with just a constant (intercept) and then add variables with upper bound linear. I think the correct code is as follows:

mdlForward = stepwiselm(Xtrain,ytrain,'constant','Upper','linear','PEnter',0.05);

mdlBackward = stepwiselm(Xtrain,ytrain,'linear','Upper','linear','PRemove',0.1);