Interactive Image Inpainting Using Exemplar Matching

This example shows how to interactively select image regions and inpaint the selected regions by using the exemplar based-matching method. Interactive inpainting allows you to select a region multiple times and perform inpainting iteratively to achieve the desired results.

In this example, you perform region filling and object removal by:

• Interactively selecting the inpainting region.

• Dynamically updating the parameter values.

• Visualizing the results dynamically.

Read Image

Read an image to inpaint into the workspace. The image has missing image regions to be filled through inpainting.

I = imread('greensdistorted.png');

Create Interactive Figure Window

Create an interactive figure window to display the image to be inpainted. In the window, you can select the region of interest (ROI), and dynamically update the parameter values.

h = figure('Name','Interactive Image Inpainting','Position',[0,0,700,400]);

% Create a panel in the current figure to interactively set the parameter
% values.
dataPanel = uipanel(h,'Position',[0.01 0.5 0.25 0.5],'Title','Set parameter values','FontSize',10);

% Add an user control interface for specifying the patch size.
% Set the default patch size value to 9.
uicontrol(dataPanel,'Style','text','String','Enter Patch Size','FontSize',10,'Position',[1 150 120 20]);
data.patchSize = uicontrol(dataPanel,'Style','edit','String',num2str(9),'Position',[7 130 60 20]);

% Add an user control interface for selecting the fill order.
% Set the default fill order to gradient.
uicontrol(dataPanel,'Style','text','String','Select Filling Order','FontSize',10,'Position',[5 100 120 20]);
data.fillOrder = uicontrol(dataPanel,'Style','popupmenu','String',{'gradient','tensor'},'Position',[7 80 80 20]);

% Create a panel in the current figure to display the image.
viewPanel = uipanel(h,'Position',[0.25 0 0.8 1],'Title','Interactive Inpainting','FontSize',10);
ax = axes(viewPanel);

Display the image in the interactive figure window.

hImage = imshow(I,'Parent',ax); 

Select and Inpaint Image Regions Interactively

Select ROIs interactively and dynamically inpaint the selected ROIs by using the callback function clickCallback. Assign a function handle that references the clickCallback function to the ButtonDownFcn property of the image object.

hImage.ButtonDownFcn = @(hImage,eventdata)clickCallback(hImage,eventdata,data);

Interactively inpaint the image by following these steps.

Step 1: Choose the patch size and the fill order for inpainting. To inpaint with local parameter values, modify the patch size and the fill order to the desired values by using user controls in the interactive figure window.

The choice of patch size and fill order influences the quality of inpainting, and the best values to use depend on the characteristics of the image region to be inpainted.

The default patch size value is set to 9.

• To inpaint regions with regular textures, choose a larger patch size and achieve seamless inpainting.

• To inpaint regions that are locally uniform with respect to a small neighborhood, choose smaller patch size.

The default fill order is set to 'gradient'. You can choose a 'gradient' or 'tensor' based fill order for inpainting image regions. However, 'tensor' based fill order is more suitable for inpainting image regions with linear structures and regular textures.

Step 2: Create a freehand ROI interactively by using your mouse. Position the pointer on the axes and click and drag to draw the ROI shape. Release the pointer to close the shape.

The function dynamically updates the parameter values specified by using the user control interface and inpaints the selected ROI. Repeat steps 1 and 2 in order to inpaint all the desired regions in the image.

Create Callback Function to Select and Inpaint ROIs

Create the clickCallback to be used with the ButtonDownFcn to interactively select and inpaint ROIs.

function clickCallback(src,~,data)
% Get the parameter values for inpainting.
fillOrder = data.fillOrder.String{data.fillOrder.Value};
pSize = data.patchSize.String;
patchSize = str2double(pSize);
% Select and draw freehand ROI.
h = drawfreehand('Parent',src.Parent);
% Create a binary mask of the selected ROI.
mask = h.createMask(src.CData);
% Run exemplar-based inpainting algorithm with user given parameters.
newImage = inpaintExemplar(src.CData,mask,'PatchSize',patchSize,'FillOrder',fillOrder);
% Update input image with output.
src.CData = newImage;
% Delete ROI handle.
delete(h);
end