Quantitative evaluation of the images generated by GAN

Hi,

I run the codes at https://www.mathworks.com/help/deeplearning/ug/train-generative-adversarial-network.html

and want to evaluate the similarity of the generated images with their originals.

I wan to use these quantitative metrics: SSIM, Mean Square Error (MSE), or Peak Signal to Noise Ratio (PSNR).

So, I need two images: 1)Generated image, 2) The original image to match with the generated image

I used the following code to get each generated image separately

I = extractdata( dlXGeneratedNew );
img = I (:,:,:,idx);

But I don't know which original image should be used with which generated image.

Could you please help, how to do this quantitative evaluation?

Answer 1

[Gayathri]

Hello @S C.Carl,

Since standard GANs lack a one-to-one correspondence between input and output, “SSIM” is not an appropriate metric for evaluation.

Instead, we can use “Frechet Inception Distance (FID)” as a metric. The inception score estimates the quality of a collection of synthetic images based on how well the top-performing image classification model “inceptionv3” classifies them as one of 1,000 known objects. For calculating FID, you can use pre-trained “inceptionv3” model from the “Deep Learning Toolbox” as follows.

net = imagePretrainedNetwork("inceptionv3”)
original_features = activations(net, original_images, 'avg_pool');
generated_features = activations(net, generated_images, 'avg_pool');
fid_score = wassersteinDistance(original_features, generated_features);

You can read about the model in the following links.

https://www.mathworks.com/help/deeplearning/ref/inceptionv3.html

https://www.mathworks.com/help/deeplearning/ref/imagepretrainednetwork.html

The implementation of the “Wasserstein-distance” can be found below.

https://www.mathworks.com/matlabcentral/fileexchange/75228-wasserstein-distance

In addition to this, we can also use "No Reference Image Quality" metrics to assess the performance of GAN model like “BRISQUE”, “NIQE” or “PIQE”.

Please refer to the following MathWorks documentation for more information on the "No Reference Image Quality" metrics:

https://www.mathworks.com/help/images/compare-no-reference-image-quality-metrics.html

Hope you find this information helpful.