Normalization and scaling features in ML

Hello everyone

its is very important to scale and normalize data for training ML algorithme, lets take for exemple the mean normalization , so to normalize one feature we take the each instance of vector feature so we substract the mean of this vector x_vector - mean (x_vector), and divide by max(x_vector)-min(x_vector ) . Now the question is : should i normalize and scale the data test as well ?, if yes on based on what after the integration of our model for an real time application to normalize our data from sensors ? (real data without normaliziation and scaling ) , like we dont have vector and then we calculate the mean and max min to normalize it.

Thanks for your contribution

Answer 1

[Shantanu Dixit]

Hi Nor,

This is a concept related to feature scaling, here's how you would handle normalization for real-time data using the statistics from the training data:

1. Calculate the mean (x_train_mean), maximum (x_train_max), and minimum (x_train_min) values for each feature in the training data (x_train).
2. Normalize the real-time data (x_realtime) using the formula: x_normalized = (x_realtime - x_train_mean) / (x_train_max - x_train_min).
3. Use the normalized real-time data (x_normalized) as input to your trained model for prediction.

By using the mean and range values from the training data, you ensure consistency in the normalization process.