risk.validation.generalizedAreaUnderCurveTest

R2026b

Generalized area under curve test

Since R2026a

Syntax

hGAUCTest = risk.validation.generalizedAreaUnderCurveTest(BaselineExpectedGrade,BaselineRealizedGrade,TargetExpectedGrade,TargetRealizedGrade)

hGAUCTest = risk.validation.generalizedAreaUnderCurveTest(BaselineExpectedGrade,BaselineRealizedGrade,TargetExpectedGrade,TargetRealizedGrade,ConfidenceLevel=confidenceLevel)

[hGAUCTest,GAUCOutput] = risk.validation.generalizedAreaUnderCurveTest(___)

Description

hGAUCTest = risk.validation.generalizedAreaUnderCurveTest(BaselineExpectedGrade,BaselineRealizedGrade,TargetExpectedGrade,TargetRealizedGrade) returns the result of a generalized area under curve (gAUC) test, hGAUCTest, which compares scores and responses in a baseline and target portfolio. The output is 1 if the test rejects the null hypothesis at the 95% confidence level, or 0 otherwise.

hGAUCTest = risk.validation.generalizedAreaUnderCurveTest(BaselineExpectedGrade,BaselineRealizedGrade,TargetExpectedGrade,TargetRealizedGrade,ConfidenceLevel=confidenceLevel) specifies the confidence level for the gAUC test.

[hGAUCTest,GAUCOutput] = risk.validation.generalizedAreaUnderCurveTest(___) also returns a structure GAUCOutput that contains summary metrics. Specify GAUCOutput as the second output argument with any of the input argument combinations in the previous syntaxes.

Examples

Apply Generalized AUC Test of Loss Given Default Data

Load the loss given default (LGD) data, which contains the expected and realized LGD grades for 200 defaulted customers, as well as the year that the recovery process was closed.

lgdData = readtable("LGDRatingGradeData.csv")

lgdData = 200×5 table
    ExpectedLGD    ExpectedLGDGrade    RealizedLGD    RealizedLGDGrade    RecoveryClosedYear
    ___________    ________________    ___________    ________________    __________________

      0.21067              4             0.11221              3                  2023       
      0.68432              8             0.90534             11                  2023       
       0.4815              6             0.35169              5                  2023       
      0.86389             10             0.91321             11                  2023       
      0.85797             10             0.84803             10                  2023       
      0.29985              4             0.24432              4                  2023       
      0.30411              5                   0              1                  2023       
      0.88206             10             0.65058              8                  2023       
        1.054             12             0.73065              9                  2023       
      0.96353             11             0.61439              8                  2023       
       0.3936              5            0.061432              2                  2023       
      0.55109              7             0.59951              7                  2023       
      0.75181              9             0.55973              7                  2023       
      0.78397              9             0.68201              8                  2023       
      0.40728              6             0.27822              4                  2023       
      0.61732              8             0.38996              5                  2023       
      ⋮

The ExpectedLGDGrade and RealizedLGDGrade table variables contain values from 1 to 12, which correspond to 12 rating grades. The values in ExpectedLGDGrade were calculated based on the borrowers' characteristics before the default event and are predictions for the rating grade at the end of the recovery period. RealizedLGDGrade contains the actual grades of the portfolios at the end of the observation period.

Create a baseline portfolio table from observations whose recovery processes were closed in 2023 and a target portfolio table from observations whose recovery processes were closed in 2024.

baselinePortfolio = lgdData(lgdData.RecoveryClosedYear==2023,:);
targetPortfolio = lgdData(lgdData.RecoveryClosedYear==2024,:);

Extract the predicted and actual grades from each portfolio.

BaselineExpectedGrade = baselinePortfolio.ExpectedLGDGrade;
BaselineRealizedGrade = baselinePortfolio.RealizedLGDGrade;
TargetExpectedGrade = targetPortfolio.ExpectedLGDGrade;
TargetRealizedGrade = targetPortfolio.RealizedLGDGrade;

Perform the generalized AUC test for the grades.

[hGAUCTest,GAUCOutput] = risk.validation.generalizedAreaUnderCurveTest(BaselineExpectedGrade,BaselineRealizedGrade,TargetExpectedGrade,TargetRealizedGrade)

hGAUCTest = logical
   1

GAUCOutput = struct with fields:
                        RejectTest: 1
                            PValue: 0.0031
                     TestStatistic: 2.7421
                     CriticalValue: 1.6449
            BaselineGeneralizedAUC: 0.8390
    BaselineGeneralizedAUCVariance: 3.3724e-04
              TargetGeneralizedAUC: 0.7431
      TargetGeneralizedAUCVariance: 0.0012
                   ConfidenceLevel: 0.9500

The output includes the result of the hypothesis test and the p-value, among other statistics. The small p-value indicates that enough evidence exists to reject the null hypothesis that the gAUC for the baseline portfolio is smaller than the gAUC for the target portfolio.

Input Arguments

BaselineExpectedGrade — Predicted baseline portfolio grades

nonnegative vector

Predicted baseline portfolio grades, specified as a nonnegative vector. BaselineExpectedGrade must be the same size as BaselineRealizedGrade. The baseline portfolio is sometimes called the initial portfolio.

BaselineRealizedGrade — Actual baseline portfolio grades

nonnegative vector

Actual baseline portfolio grades, specified as a nonnegative vector. BaselineRealizedGrade must be the same size as BaselineExpectedGrade.

TargetExpectedGrade — Expected target portfolio grades

nonnegative vector

Expected target portfolio grades, specified as a nonnegative vector. TargetExpectedGrade must be the same size as TargetRealizedGrade. The target portfolio is sometimes called the current portfolio.

TargetRealizedGrade — Actual target portfolio grades

nonnegative vector

Actual target portfolio grades, specified as a nonnegative vector. TargetRealizedGrade must be the same size as TargetExpectedGrade.

confidenceLevel — Confidence level

0.95 (default) | numeric scalar in the range (0,1)

Confidence level of the hypothesis test, specified as a numeric scalar in the range (0,1).

Output Arguments

hGAUCTest — Hypothesis test results

1 | 0

Hypothesis test result, returned as a 1 or 0 of data type logical.

• A value of 1 rejects the null hypothesis at the specified confidence level.

• A value of 0 fails to reject the null hypothesis at the specified confidence level.

GAUCOutput — Output metrics

structure

Output metrics, returned as a structure with the following fields:

• RejectTest — Logical scalar indicating whether the null hypothesis was rejected. This field represents the same values as hGAUCTest.

• PValue — Numeric scalar with values in the range [0,1] representing the p-value for the hypothesis test. A small value indicates that the null hypothesis might not be valid.

• TestStatistic — Numeric scalar representing the value of the test statistic for the hypothesis test.

• CriticalValue — Numeric scalar representing the minimum test statistic at which the test rejects the null hypothesis for the given probability and confidence level.

• BaselineGeneralizedAUC — Numeric scalar representing the gAUC for the baseline portfolio.

• BaselineGeneralizedAUCVariance — Numeric scalar representing the estimated variance of the gAUC for the baseline portfolio.

• TargetGeneralizedAUC — Numeric scalar representing the gAUC for the target portfolio.

• TargetGeneralizedAUCVariance — Numeric scalar representing the estimated variance of the gAUC for the target portfolio.

• ConfidenceLevel — Numeric scalar representing the confidence level for the hypothesis test.

For more information about the gAUC test and its corresponding statistics, see More About.

More About

Current gAUC vs. gAUC at Initial Validation Test

The current gAUC vs. gAUC at initial validation test [1], also called the gAUC test, requires you to input predicted and actual values for a baseline portfolio and a target portfolio. The null hypothesis is that the gAUC for the baseline portfolio is smaller than the gAUC for the target portfolio.

The gAUC is given by

$$gAUC=\frac{D_{ER}+1}{2}\text{,}$$

where D_ER is the Somers' D metric for the expected and realized portfolio grades. For more information about Somers' D, see risk.validation.somersD.

The test statistic for the gAUC test is given by

$$S=\frac{BaselineGeneralizedAUC-TargetGeneralizedAUC}{\sqrt{TargetGeneralizedAUCVariance}}\text{,}$$

where BaselineGeneralizedAUC and TargetGeneralizedAUC are the gAUCs for the baseline and target portfolios, respectively, and TargetGeneralizedAUCVariance is the estimated variance for the target portfolio. The software estimates the gAUCs and the target gAUC variance using the equations described in Section 3.2 of [1].

The p-value for S is equal to $$1-F\left(S\right)$$, where F is the cumulative distribution function for the standard normal distribution.

References

[1] European Central Bank. “Instructions for reporting the validation results of internal models.” February, 2019. https://www.bankingsupervision.europa.eu/activities/internal_models/shared/pdf/instructions_validation_reporting_credit_risk.en.pdf.

Version History

Introduced in R2026a