Power of the Chow Test

This example shows how to estimate the power of a Chow test using a Monte Carlo simulation.

Introduction

Statistical power is the probability of rejecting the null hypothesis given that it is actually false. To estimate the power of a test:

1. Simulate many data sets from a model that typifies the alternative hypothesis.

2. Test each data set.

3. Estimate the power, which is the proportion of times the test rejects the null hypothesis.

The following can compromise the power of the Chow test:

• Linear model assumption departures

• Relatively large innovation variance

• Using the forecast test when the sample size of the complementary subsample is greater than the number of coefficients in the test [36].

Departures from model assumptions allow for an examination of the factors that most affect the power of the Chow test.

Consider the model

`$\text{y}=\left[\begin{array}{cc}\text{X1}& 0\\ 0& \text{X2}\end{array}\right]\left[\begin{array}{c}\text{beta1}\\ \text{beta2}\end{array}\right]+\text{innov}$`

• `innov` is a vector of random Gaussian variates with mean zero and standard deviation `sigma`.

• `X1` and `X2` are the sets of predictor data for initial and complementary subsamples, respectively.

• `beta1` and `beta2` are the regression coefficient vectors for the initial and complementary subsamples, respectively.

Simulate Predictor Data

Specify four predictors, 50 observations, and a break point at period 44 for the simulated linear model.

```numPreds = 4; numObs = 50; bp = 44; rng(1); % For reproducibility```

Form the predictor data by specifying means for the predictors, and then adding random, standard Gaussian noise to each of the means.

```mu = [0 1 2 3]; X = repmat(mu,numObs,1) + randn(numObs,numPreds);```

To indicate an intercept, add a column of ones to the predictor data.

```X = [ones(numObs,1) X]; X1 = X(1:bp,:); % Initial subsample predictors X2 = X(bp+1:end,:); % Complementary subsample predictors```

Specify the true values of the regression coefficients.

`beta1 = [1 2 3 4 5]'; % Initial subsample coefficients`

Estimate Power for Small and Large Jump

Compare the power between the break point and forecast tests for jumps of different sizes small in the intercept and second regression coefficient. In this example, a small jump is a 10% increase in the current value, and a large jump is a 15% increase. Complementary subsample coefficients

```beta2Small = beta1 + [beta1(1)*0.1 0 beta1(3)*0.1 0 0 ]'; beta2Large = beta1 + [beta1(1)*0.15 0 beta1(3)*0.15 0 0 ]';```

Simulate 1000 response paths of the linear model for each of the small and large coefficient jumps. Specify that `sigma` is 0.2. Choose to test the intercept and the second regression coefficient.

```M = 1000; sigma = 0.2; Coeffs = [true false true false false]; h1BP = nan(M,2); % Preallocation h1F = nan(M,2); for j = 1:M innovSmall = sigma*randn(numObs,1); innovLarge = sigma*randn(numObs,1); ySmall = [X1 zeros(bp,size(X2,2)); ... zeros(numObs - bp,size(X1,2)) X2]*[beta1; beta2Small] + innovSmall; yLarge = [X1 zeros(bp,size(X2,2)); ... zeros(numObs - bp,size(X1,2)) X2]*[beta1; beta2Large] + innovLarge; h1BP(j,1) = chowtest(X,ySmall,bp,'Intercept',false,'Coeffs',Coeffs,... 'Display','off')'; h1BP(j,2) = chowtest(X,yLarge,bp,'Intercept',false,'Coeffs',Coeffs,... 'Display','off')'; h1F(j,1) = chowtest(X,ySmall,bp,'Intercept',false,'Coeffs',Coeffs,... 'Test','forecast','Display','off')'; h1F(j,2) = chowtest(X,yLarge,bp,'Intercept',false,'Coeffs',Coeffs,... 'Test','forecast','Display','off')'; end```

Estimate the power by computing the proportion of times `chowtest` correctly rejected the null hypothesis of coefficient stability.

```power1BP = mean(h1BP); power1F = mean(h1F); table(power1BP',power1F','RowNames',{'Small_Jump','Large_Jump'},... 'VariableNames',{'Breakpoint','Forecast'})```
```ans=2×2 table Breakpoint Forecast __________ ________ Small_Jump 0.717 0.645 Large_Jump 0.966 0.94 ```

In this scenario, the Chow test can detect a change in the coefficient with more power when the jump is larger. The break point test has greater power to detect the jump than the forecast test.

Estimate Power for Large Innovations Variance

Simulate 1000 response paths of the linear model for a large coefficient jump. Specify that `sigma` is 0.4. Choose to test the intercept and the second regression coefficient.

```sigma = 0.4; h2BP = nan(M,1); h2F = nan(M,1); for j = 1:M innov = sigma*randn(numObs,1); y = [X1 zeros(bp,size(X2,2)); ... zeros(numObs - bp,size(X1,2)) X2]*[beta1; beta2Large] + innov; h2BP(j) = chowtest(X,y,bp,'Intercept',false,'Coeffs',Coeffs,... 'Display','off')'; h2F(j) = chowtest(X,y,bp,'Intercept',false,'Coeffs',Coeffs,... 'Test','forecast','Display','off')'; end power2BP = mean(h2BP); power2F = mean(h2F); table([power1BP(2); power2BP],[power1F(2); power2F],... 'RowNames',{'Small_sigma','Large_Sigma'},... 'VariableNames',{'Breakpoint','Forecast'})```
```ans=2×2 table Breakpoint Forecast __________ ________ Small_sigma 0.966 0.94 Large_Sigma 0.418 0.352 ```

For larger innovation variance, both Chow tests have difficulty detecting the large structural breaks in the intercept and second regression coefficient.