How to PLOT Receiver operating characteristic in MATLAB: For different threshold used in Kernel Density Estimation

Dear coder, I am aware similar questions has been ask in Q1, Q2 and in the well written in Doc1 and Doc2. However I am having difficulties in understanding the explanation given in the aforementioned references.

I am fully aware the true positive rate and true negative rate can be calculated using the build-in function

[tpr,fpr,thresholds] = roc(targets,outputs)

However, I do not know how I can integrate the information from my coding into the above code. FYI, I already try to understand the example ROC_Iris_DataSet but without success.

In this problem, the ultimate aim was to find which threshold value that give the best classification result. The best threshold value was determine according to the true positive rate and true negative rate value.

To realize this objective, two function was develops

1) FUNCTION EVALTHERSHLD

To compare an array A to array B. If array A is larger than array S then return logical 1, or otherwise. Array A is compromise element by element division between each row at EasyComp(:,1) and EasyComp(:,2). Array S is a vector of similar threshold value. In this example, the array A will be compared with different threshold values. The possible threshold value are 1.07:0.01:1.1.

2) FUNCTION CONFMATRIX_EV

The second part is to evaluate the density estimation by calculating element in the confusion matrix.

As can be seen in the code, I evaluate the true positive rate and true negative rate using the FUNCTION CONFMATRIX_EV. However, I am interested to use build-in MATLAB function such as

[tpr,fpr,thresholds] = roc(targets,outputs)

to calculate all of this information.

Nevertheless, I have some confusion and implementation of FUNCTION ROC. 1) What is the OUTPUT THRESHOLDS from the FUNCTION ROC, is it similar to threshold that I am looking at (i.e., 1.07:0.01:1.1)

2) I am not clear how I can insert the TARGET and OUTPUT from my calculation into the ROC FUNCTION.

I really appreciate if someone can show what I am missing or misunderstand. Thanks in advance

The two functions are as shown below and MAT file is attach together in this thread.

load ('EasyComp')
ThrshldResult=EvalThershld (EasyComp);
ConfResult=ConfMatrix_Ev (EasyComp,ThrshldResult);

x=1.07:0.01:1.1; 

TPR = (cell2mat( ConfResult.TPR ));
FPR = (cell2mat( ConfResult.FPR ));
TPR_L = (cell2mat( ConfResult.TPR ));
FPR_L = (cell2mat( ConfResult.FPR ));

subplot(3,1,1),
plot(FPR,TPR)
subplot(3,1,2)

plot (x,TPR_L);
hold on
plot (x,FPR_L);

function ResThRat=EvalThershld (EasyComp)

txt =1;
x=1.07:0.01:1.1;    % Threshold value
for k=1:numel(x)
    threshold=x(k);
    S = repmat(threshold, (length (EasyComp(:,1))),1 );
    ResThRat {txt} = bsxfun(@ge, S, (EasyComp(:,1)./EasyComp(:,2)));
    txt =txt+1;
end
end

function result=ConfMatrix_Ev (EasyComp,ThrshldResult)

txc=1;

ActClss=EasyComp(:,3);          % Actual class

for kok =1:length(ThrshldResult) % Evaluate every logical in each the cell
    PredClss=ThrshldResult{kok};  % Predicted class by varying diff threshold

      % In the following lines a comparison between elements in the column THREE
      % of the EASYCOMP and Predicted Class.
      %           0 0 = True Negetive
      %           0 1 = False Negetive
      %           1 0 = False Positive
      %           1 1 = True Positive
      % The all this value store as a CATEGORICAL array name CONFUSION MATRIX: CONFMAT

      ConfMat = categorical (repmat({'TN'}, [length(EasyComp) 1]));       % True Negative  (TN)  | Predicted NS, actual NS
      ConfMat (logical (((ActClss==0) .*(PredClss==1))))='FN'; % False Negative (FN)  | Predicted NS, actual S
      ConfMat (logical (((ActClss==1) .*(PredClss==0))))='FP'; % False Positive (FP)  | Predicted S, actual NS
      ConfMat (logical (((ActClss==1) .*(PredClss==1))))='TP'; % True Positive (TP)   | Predicted S, actual S

      % % Calculate the True Positive Rate % False Positive Rate which can be used to Plot the Receiver operating characteristic
      perf.TPR  = (sum(ConfMat(:) == 'TP'))/((sum(ConfMat(:) == 'TP'))+ (sum(ConfMat(:) == 'FN')));   % TP/ (TP+FN)  || Sensetivity,  SENT is very high if FN is low
      perf.FPR  = (sum(ConfMat(:) == 'FP'))/((sum(ConfMat(:) == 'FP'))+ (sum(ConfMat(:) == 'TN')));   % FP/ (FP+TN)  || Specificity,  SPEC is very high is TN is low

      result.TPR {txc} = perf.TPR;
      result.FPR {txc} = perf.FPR;
      %
      txc=txc+1;
  end
  end

Answer 1

[balandong]

Similar concept but does answer the question in this thread can be found in this Link_ROC