Issue with Blockproc when using PadPartialBlocks

I have an image that I want to break up into regions and calculate the Standard deviation of (using STD2). I only want the result to contain calculations for COMPLETE blocks. Walter has kindly suggested to use "PadMethod"as NaN, then the resulting data would have NaNs in the partial columns / rows that I can then process out.

However, using PadPartialBlocks doesn't seem to be working as it should

bss = [500,500]; 
fh = @(bs) std2(bs.data);
J = blockproc(IM2, bss, fh,'UseParallel',true,'PadPartialBlocks',true,'PadMethod',NaN);

The last column (=col 21) which is the "partial" column doesn't have values I was expecting. Surely they should all be NaN? Its as though the NaN isn't actally been replaced in the last partial column - what am I doing wrong?

[Stephen23]

It seems to work:

IM2 = rand(10,10);
bss = [3,3];
fh = @(bs) std2(bs.data);
J = blockproc(IM2, bss, fh, 'UseParallel',true, 'PadPartialBlocks',true, 'PadMethod',NaN)

J = 4×4

0.2605 0.3164 0.2651 NaN 0.4085 0.2991 0.2368 NaN 0.3742 0.2664 0.3372 NaN NaN NaN NaN NaN

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

What is size(IM2) ? Even better: upload your data by clicking the paperclip button.

[Jason]

Hi Stephen, heres my part of Image (limited by max of 5MB), so I've reduced the height (Ithe original image is 10242 x 8000 pixels).

Here is my code:

    blockSizeAcross = 500;   
    blockSizeDown = 500;
    bss = [blockSizeAcross,blockSizeDown];     % Each blockproc region
    [rows, columns, numColors] = size(IM);
    numBlocksAcross = floor(columns / blockSizeAcross);
    numPixelsAcross = numBlocksAcross * blockSizeAcross;
    numBlocksTall = floor(rows / blockSizeDown);
    numPixelsTall = numBlocksTall * blockSizeDown;

    % Add Grids on Image so can see the sub images
    hold(ax,'on');
    k=1:numBlocksAcross;
    axis(ax,'image');
    xline(ax,k*blockSizeAcross,'r--'); drawnow; 
    k=1:numBlocksTall;
    yline(ax,k*blockSizeDown,'r--'); drawnow;
    
    fh = @(bs) [mean2(bs.data),std2(bs.data)];    % Can combine operations in one call to blockproc!
    J = blockproc(IM2, bss, fh,'UseParallel',true,'PadPartialBlocks',true,'PadMethod',NaN);
    I2 = J(:,1:2:end);
    SD = J(:,2:2:end);           

    % I'd rather not have to reshape to remove the partial block (column), I want to have it as NaN's 
    % reshape i.e. remove last column if its a partial block
    if columns>numPixelsAcross
        I2(:,end)=[];
        SD(:,end)=[];
    end   
    
    

Answer 1

[Matt J]

Integer types cannot hold NaN values, so you will have to convert your int8 input to floating point:

    load('ImgForBlockProcQuestion.mat');
    
    IM=double(IM);
    
    blockSizeAcross = 500;   
    blockSizeDown = 500;
    bss = [blockSizeAcross,blockSizeDown];     % Each blockproc region
 
    fh = @(bs) [mean2(bs.data),std2(bs.data)];    % Can combine operations in one call to blockproc!
    J = blockproc(IM, bss, fh,'UseParallel',true,'PadPartialBlocks',true,'PadMethod',NaN);

You can now see that NaNs are present:

    
    nnz(isnan(J))

ans = 46

[Jason]

Perfect thats fixed it - thanks Matt!

[Matt J]

You're welcome, but my advice from before still stands. This is not a job for blockproc.

[Jason]

so the reason I haven't tried this yet is whilst I do use std2, I also have my own image sharpness function involving FFT and a few fancy operations (sorry I can't divulge anymore). I wasn't sure if the other approach would work, especially as my fucntion is of the form

sharpness=myfunc(app,Image)

I know using parfor you can't run fucntion where app is an argument and found I had to do this:

sharpness=app.myfunc(Image)

Answer 2

[Matt J]

what am I doing wrong?

Nothing you've shown us currently. The border values will/should be NaNs as you expect.

J = blockproc(rand(6), [5,5], @(bs)std2(bs.data) ,'PadPartialBlocks',true,'PadMethod',NaN)

J = 2×2

0.2842 NaN NaN NaN

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

[Matt J]

Incidentally, if a tiled std operation is really all you want, blockproc is an inefficient choice. Download sepblockfun instead,

bss=[500,500];
IM2=rand(9999);

tic;
IM3=padarray(IM2, [1,1],nan,'post');
J1=sqrt( sepblockfun(IM3.^2,bss,'mean') - sepblockfun(IM3,bss,'mean').^2 );
toc;

Elapsed time is 0.287876 seconds.

tic;
J2= blockproc(IM2, bss, @(bs) std2(bs.data), 'PadPartialBlocks',true,'PadMethod',NaN);
toc

Elapsed time is 0.489024 seconds.

i=~isnan(J2);
percentError = norm(J1(i)-J2(i))/norm(J1(i))*100

percentError = 2.0000e-04

[Jason]

What about if you put useparallel as true?

[Matt J]

You can go ahead and test it, but I suspect UseParallel will make things worse with a Process pool and only marginally better with a Thread pool. At least, on my computer it does (with 6 workers):

parpool('Threads')

Starting parallel pool (parpool) using the 'Threads' profile ...

Connected to parallel pool with 6 workers.

ans =

ThreadPool with properties:

NumWorkers: 6

Busy: false

FileStore: [1x1 parallel.FileStore]

ValueStore: [1x1 parallel.ValueStore]

bss=[500,500];
IM2=rand(9999);

tic;
IM3=padarray(IM2, [1,1],nan,'post');
J1=sqrt( sepblockfun(IM3.^2,bss,'mean') - sepblockfun(IM3,bss,'mean').^2 );
toc;

Elapsed time is 0.704048 seconds.

tic;
J2= blockproc(IM2, bss, @(bs) std2(bs.data), 'UseParallel',false, ...
             'PadPartialBlocks',true,'PadMethod',NaN);
toc

Elapsed time is 1.841515 seconds.

tic;
J2= blockproc(IM2, bss, @(bs) std2(bs.data), 'UseParallel',true, ...
             'PadPartialBlocks',true,'PadMethod',NaN);
toc

Elapsed time is 1.800456 seconds.