Interactively Segment Hyperspectral Image Using Segment Anything Model

Since R2025a

This example shows how to interactively preprocess and segment a hyperspectral image using the Segment Anything Model (SAM).

The Segment Anything Model (SAM) is a state-of-the-art image segmentation model that uses deep learning neural networks to accurately and automatically segment objects within images without requiring training. The SAM enables you to instantaneously segment objects by providing feedback through visual prompts, such as marked points, selected ROI, and existing masks. For more information about SAM, see Get Started with Segment Anything Model for Image Segmentation.

This example requires the Hyperspectral Imaging Library for Image Processing Toolbox™ and Image Processing Toolbox Model for Segment Anything Model support packages. You can install the support package add-ons from the Add-On Explorer. For more information about installing add-ons, see Get and Manage Add-Ons. The Hyperspectral Imaging Library for Image Processing Toolbox requires desktop MATLAB®, as MATLAB® Online™ and MATLAB® Mobile™ do not support the library.

Load Hyperspectral Image

Load a hyperspectral image into the workspace.

hcube = imhypercube("paviaU.dat")

hcube = 
  hypercube with properties:

     ImageDims: "[610x340x103 double]"
    Wavelength: [103×1 double]
      Metadata: [1×1 struct]
     BlockSize: [610 340]

Observe the wavelengths of different bands.

wavelengths = hcube.Wavelength;
bands = (1:numel(wavelengths))';
numBands = numel(bands);
band_wavelength = table(bands,wavelengths)

band_wavelength=103×2 table
    bands    wavelengths
    _____    ___________

      1          430    
      2          434    
      3          438    
      4          442    
      5          446    
      6          450    
      7          454    
      8          458    
      9          462    
     10          466    
     11          470    
     12          474    
     13          478    
     14          482    
     15          486    
     16          490    
      ⋮

Select Bands

Segmentation using the SAM requires a three-channel image. Select three bands from the hyperspectral image to create the three-channel image. Select a method to use to colorize the image by setting the colorizeMethod variable. If you select Custom, you can also specify the color value for each band by setting integer values for the variables of b1, b2, and b3.

colorizeMethod = "MostInformative"

colorizeMethod = 
"MostInformative"

switch colorizeMethod
    case "RGB"
        I = colorize(hcube,Method="rgb",ContrastStretching=true);
    case "CIR"
        I = colorize(hcube,Method="cir",ContrastStretching=true);
    case "FalseColor"
        I = colorize(hcube,Method="falsecolored",ContrastStretching=true);
    case "MostInformative"
        numEndmembers = countEndmembersHFC(hcube);
        endmembers = fippi(hcube,numEndmembers);
        [~,selectedBands] = selectBands(hcube,endmembers,NumberOfBands=3);
        I = colorize(hcube,selectedBands,ContrastStretching=true);
    case "Custom"
        b1 = 56;
        b2 = 31;
        b3 = 11;
    otherwise
        disp("Please select one of the 5 options")
end

Visualize the extracted three-channel image.

figure
imshow(I)

Open Image Segmenter App

From the MATLAB® Toolstrip, open the Image Segmenter app. On the Apps tab, in the Image Processing and Computer Vision section of the Apps gallery, click Image Segmenter.

On the app toolstrip, click Load Image, and then select Load Image from Workspace. In the Import from Workspace dialog box, select the image I. The app displays the image you selected.

You can alternatively open the app by using the imageSegmenter command, specifying the name of the image.

imageSegmenter(I)

Interactively Segment Image Using SAM

On the Image Segmenter app toolstrip, in the Segmentation Tools section, select Segment Anything Model.

To perform segmentation using marked points, select Mark Foreground, then click the object in the image to create foreground points. When you interactively add points, the Image Segmenter app automatically adds regions to the mask based on your selections and displays a preview of the mask you create. To erase a point you have marked, select Erase and click on the point. To erase all the points you have marked and restart, select Clear All. You can refine your segmentation mask by specifying parts of the image for SAM to ignore. To remove regions from your mask, select Mark Background, and click the image to set red points on areas extraneous to the object.

Alternatively, you can perform segmentation using SAM by drawing a region of interest (ROI) using the Draw ROI option. For more information on segmentation using ROI, see Segment Objects Using Segment Anything Model (SAM) in Image Segmenter.

When you are satisfied with your mask preview, click Apply to return to the Segment Anything Model menu. The yellow shading indicates the segmented mask area.

To return to the main Image Segmenter app, click Close SAM. You can refine the segmentation using SAM or other algorithms in the Image Segmenter app. For more information on how to refine the segmentation, see Segment Objects Using Segment Anything Model (SAM) in Image Segmenter.

To segment a new object, on the Segmentation tab of the Image Segmenter app, in the app toolstrip, select New Segmentation, then New Segmentation. On the Image Segmenter app toolstrip, in the Segmentation Tools section, select Segment Anything Model and perform the segmentation with your preferred method.

You can also perform automatic segmentation using SAM that splits the entire image into regions by using the Show Region Boundaries option. You can then select regions from the image to add to the segmentation mask. For more information on segmentation using automatic region boundaries, see Segment Objects Using Segment Anything Model (SAM) in Image Segmenter.

When you are done segmenting the image, to save the final binary mask image and the segmented version of the original image, click Export and select Export Images.