Error Combining two pixelLabel​ImageDatas​tores or randomPatc​hExtractio​nDatastore​s

Hi,

I am using Matlab 2019b

Combining two pixelLabelImageDatastores or randomPatchExtractionDatastores gives an error. See below. Reading these datastores gives a table and not a cell array. So Horzcat does not work , see part of CombinedDatastore.m below (replacing it with although vertcat does, but it still is a table in that case)

Just an example to produce the error :

dataSetDir = fullfile(toolboxdir('vision'),'visiondata','triangleImages');
imageDir = fullfile(dataSetDir,'trainingImages');
labelDir = fullfile(dataSetDir,'trainingLabels');

imds = imageDatastore(imageDir);
imds2 = imageDatastore(imageDir);
imdstest= combine(imds,imds2);
a=read(imdstest);

classNames = ["cell","background"];
labelIDs   = [255 0];
pxds = pixelLabelDatastore(labelDir,classNames,labelIDs);
ds = pixelLabelImageDatastore(imds,pxds);
ds2 = pixelLabelImageDatastore(imds,pxds);
dstest= combine(ds,ds2);
a=read(dstest);

Error using matlab.io.datastore.CombinedDatastore/read (line 109)

Duplicate table variable name: 'InputImage'.

rds = randomPatchExtractionDatastore(imds,pxds,[16,16], "PatchesPerImage",4);
rds2 = randomPatchExtractionDatastore(imds,pxds,[16,16], "PatchesPerImage",4);
rdstest= combine(rds,rds2);
a=read(rdstest);

Error using matlab.io.datastore.CombinedDatastore/read (line 109)

Duplicate table variable name: 'InputImage'.

Part of classdef CombinedDatastore < matlab.io.Datastore & matlab.mixin.Copyable

        function [data,info] = read(ds)
            
            if ~hasdata(ds)
                error(message('MATLAB:datastoreio:splittabledatastore:noMoreData'));
            end
            
            numDatastores = numel(ds.UnderlyingDatastores);
            data = cell(1, numDatastores);
            info = cell(1, numDatastores);
            for ii = 1:numel(ds.UnderlyingDatastores)
                [data{ii}, info{ii}] = read(ds.UnderlyingDatastores{ii});
                data{ii} = iMakeUniform(data{ii}, ds.UnderlyingDatastores{ii});
            end
            data = horzcat(data{:});
        end

Answer 1

[Srivardhan Gadila]

The error "Duplicate table variable name:" w.r.t combining certain datastores is a known issue and it might be addressed in the future releases.

In order to get the similar behaviour of combine (like that of imageDatastore or pixelLabelDatastore) the workaround would be that you'll need to put the output in a cell by using transform on your pixelLabelImageDatastore.

dataSetDir = fullfile(toolboxdir('vision'),'visiondata','triangleImages');
imageDir = fullfile(dataSetDir,'trainingImages');
labelDir = fullfile(dataSetDir,'trainingLabels');
imds1 = imageDatastore(imageDir);
imds2 = imageDatastore(imageDir);
classNames = ["triangle","background"];
labelIDs   = [255 0];
pxds1 = pixelLabelDatastore(labelDir,classNames,labelIDs);
pxds2 = pixelLabelDatastore(labelDir,classNames,labelIDs);
pids1 = pixelLabelImageDatastore(imds1,pxds1);
pids2 = pixelLabelImageDatastore(imds2,pxds2);
imdsc = combine(imds1,imds2);
pxdsc = combine(pxds1,pxds2);
tpids1 = transform(pids1, @(x){x});
tpids2 = transform(pids2, @(x){x});
pidsc = combine(pids1,pids2);
tpidsc = combine(tpids1,tpids2);
%% Check the behaviour of combine for each type of datastore
read(imds1)
read(imdsc)
read(pxds1)
read(pxdsc)
read(pids1)
read(tpids1)
read(tpidsc)

[Jack Xiao]

Hi, the same problem. i am working on image inpainting task.

i need to read two imagedata store, one is with image, another is with mask image.

% image with three channels
imds1 = imageDatastore('G:\F\03backup\my sci\tuxiangxiufu\context_encoder_pytorch-master\dataset\train\celea\img_align_celeba','IncludeSubfolders',false,'FileExtensions','.jpg') ;
% mask image with one channel and logical datatype
imds_mask = imageDatastore('G:\F\03backup\pytorch-inpainting-with-partial-conv-master\big_regular','IncludeSubfolders',false,'FileExtensions','.png') ;

imageAugmenter = imageDataAugmenter( ...
    'RandXReflection',true,...
    'RandRotation',[-20,20], ...
    'RandXTranslation',[-3 3], ...
    'RandYTranslation',[-3 3])

imageSize = [128 128 3];
imageSize_mask = [128 128 3]; % 
aug_imds1 = augmentedImageDatastore(imageSize, imds1,'DataAugmentation',imageAugmenter);
aug_imds_mask = augmentedImageDatastore(imageSize_mask, imds_mask, 'ColorPreprocessing',"gray2rgb"); % read default interpolation

imdsCombined = combine(imds1, imds_mask) ;
dataOut = read(imdsCombined) ;  % this is ok when without augment

imdsCombined_aug = combine(aug_imds1, imds_mask) ;
dataOut = read(imdsCombined) ;
% this is wrong with error: 
Error using matlab.io.datastore.CombinedDatastore/read (line 138)
All tables being horizontally concatenated must have the same number of rows. When concatenating cell arrays with tables, the cell arrays must have the same number of rows as the tables.

% then i tried with two augment, the mask augment is for same image size.
imdsCombined_two_aug = combine(aug_imds1, aug_imds_mask) ;
dataOut = read(imdsCombined_two_aug) ;
% this is wrong with error: 
Error using matlab.io.datastore.CombinedDatastore/read (line 138)
Duplicate table variable name: 'input'.

so, i wonder why reading the combined imadatastore does not work on augmented imagedatastore.

what should i do as i need to get the inpainted image using the original image and mask image?

sorry for one more question, what should i do if i want to create one user-defined aumenting method (not covered by matlab).

thank you,

best regards