Verify Predictive Ability Robustness of a regARIMA Model
This example shows how to forecast a regression model with ARIMA errors, and how to check the model predictability robustness.
Load the Credit Defaults data set, assign the response (
y and the predictors
X. For illustration, specify that the response series is a regression model with AR(1) errors. To avoid distraction from the purpose of this example, assume that all predictor series are stationary.
load Data_CreditDefaults y = Data(:,5); X = Data(:,[1 3:4]); T = size(X,1); % Sample size Mdl = regARIMA(1,0,0);
Vary the validation sample size (
m), and forecast responses from
Mdl recursively. That is, for each validation sample size:
Fit the model to the data (
Forecast responses from the estimated model (
Compute the two performance statistics, root mean square error (RMSE) and root prediction mean square error (RPMSE).
m = 4:10; % Validation sample lengths rPMSE = m; % Preallocate rPMSE rMSE = m; % Preallocate rMSE for k = 1:numel(m); yEst = y(1:(T-m(k))); % Response data for estimation yVal = y((T-m(k)+1):T); % Validation sample EstMdlY = estimate(Mdl,yEst,'X',X,'display','off'); yHat = EstMdlY.Intercept + X(1:(T-m(k)),:)*EstMdlY.Beta';... % Estimation sample predicted values [e0,u0] = infer(EstMdlY,yEst,'X',X); yF = forecast(EstMdlY,m(k),'Y0',yEst,... 'X0',X(1:T-m(k),:),'XF',X((T-m(k)+1):T,:));... % Validation sample predicted values rMSE(k) = sqrt(mean((yEst - yHat).^2)); rPMSE(k) = sqrt(mean((yF - yVal).^2)); end
rPMSE are vectors that contain the RMSE and RPMSE, respectively, for each validation sample.
Display the performance measures.
fprintf('\n m | rMSE | rPMSE\n')
m | rMSE | rPMSE
for k = 1:length(m) fprintf('%2d | %0.4f | %0.4f\n',m(k),rMSE(k),rPMSE(k)) end
4 | 0.0947 | 0.2274 5 | 0.0808 | 0.1902 6 | 0.0810 | 0.2036 7 | 0.0714 | 0.1924 8 | 0.0809 | 0.1532 9 | 0.0720 | 0.1557 10 | 0.0899 | 0.1300
The predictive ability of this model is fairly robust because
rPMSE changes slightly for increasing
rMSE is less than
rPMSE for all
m. This signifies poor predictive ability.
Search for a better model by specifying, e.g., more AR or MA lags in the error model, and compare the PMSEs over these models. Choose the model with the lowest PMSE for a given validation sample size.
- Verify Predictive Ability Robustness of a regARIMA Model
- Forecast a Regression Model with Multiplicative Seasonal ARIMA Errors