How to summarise the perdictions acquired from Named Entity Recognition Model

Hi all,

I have multiple files (let's say 100) in a folder. Each has been processed using the Named Entity Recognition Model, and I have acquired the prediction results. However, I want to summaries those in an Excel sheet (example photo of the table attached). Can someone help me with the code for generating such a summary table?

Thanks in advance.

Answer 1

[Venu]

Hi @Sunpreet Sharma,

You can follow the below code format to generate a summary table in MATLAB and save it as an Excel sheet.

The code assumes that you have the prediction results stored in a structured format (for example, in MATLAB structures, cell arrays, or tables) and that you can iterate over them to check for the presence of 'Person', 'Location', and 'Organization' entities.

summaryTable = {'Files name', 'Person', 'Location', 'Organization'};

folderPath = 'path_to_your_files'; % Replace with your actual folder path

files = dir(fullfile(folderPath, '*.txt')); % Replace '*.txt' with the actual file extension

for i = 1:length(files)
    fileName = files(i).name;
    
    % Load the NER results for the current file
    % Here you would replace this with your actual method of loading results
    % For example, if results are saved in .mat files, you could use load function
    nerResults = load(fullfile(folderPath, fileName));
    
    % Check for the presence of entities in nerResults
    % This part of the code depends on how your NER results are structured
    % You need to replace 'hasPerson', 'hasLocation', and 'hasOrganization'
    % with the actual logic you use to determine the presence of entities
    hasPerson = ~isempty(nerResults.person); % Replace with your actual condition
    hasLocation = ~isempty(nerResults.location); % Replace with your actual condition
    hasOrganization = ~isempty(nerResults.organization); % Replace with your actual condition
    
    % Convert logical values to 'Yes' or 'No' for the Excel sheet
    person = ifelse(hasPerson, 'Yes', 'No');
    location = ifelse(hasLocation, 'Yes', 'No');
    organization = ifelse(hasOrganization, 'Yes', 'No');
    
    % Append the results to the summary table
    summaryTable(end+1, :) = {fileName, person, location, organization};
end

% Convert the cell array to a table
summaryTable = cell2table(summaryTable(2:end, :), 'VariableNames', summaryTable(1, :));

% Write the table to an Excel file
excelFileName = 'NER_Summary.xlsx'; % Choose your desired Excel file name
writetable(summaryTable, excelFileName);

% Helper function to replace if-else
function result = ifelse(condition, trueValue, falseValue)
    if condition
        result = trueValue;
    else
        result = falseValue;
    end
end

You can modify this script by replacing "path_to_your_files" and the logic for determining the presence of 'Person', 'Location', and 'Organization' based on how your NER results are structured and stored.

[Sunpreet Sharma]

Hi @Venu, Thank you so much for your answer. The solution is very close to what I was looking for. I did make some changes and now I have what I was after.

Thanks again.