Separate Video Frames Based on Similarity (SSIM)

I have a video which was made by mixing short scenes from 2 different videos mixed into one. For example few seconds from 1st scene, then few seconds from 2nd scene and so on. There are only 2 scenes. I am trying to separate the video in to two videos. Can someone help me do this in matlab?

Someone told me to use SSIM of each frame and collect all frames with similar SSIM into one video and the rest in the other. But I have no idea how to even start.

Answer 1

[Yash Ubale]

Hello,

You can try the following steps to achieve it :

1. Load the video using 'VideoReader'
2. Extract the frames from the video
3. Using SSIM, compare the extracted frames with two reference frames, one from each scene
4. Higher the scalar value, more similar the frames are
5. Distinguish the frames into two separate scenes
6. Recombine the frames belonging to each scene

To load the video and extract all the frames :

obj = VideoReader('sample_video.mp4');
% total number of frames in the video
countFrames = 0;
while hasFrame(obj)
  this_frame = readFrame(obj);
  % compare the frames with reference frame from each scene
  countFrames = countFrames + 1;
end

To compare the frames using SSIM

% higher the value of 'ssimval' higher the similarity, 'ref' is reference frame
[ssimval,ssimmap] = ssim(this_frame,ref); 

[Alex Van Der Ven]

Hi Yash! I have one question about the ssim() function: the instructions page specifies that the input images must be grayscale images. Not sure if I'm missunderstanding your suggestion, but should each frame be converted to grayscale prior to being input to the ssim() function?

Thanks for the help in advance!