Object movement by estimating motion

Hello everyone, I have estimated the motion of the frame using optical flow. I want to move the object using the estimated motion. Here is the code to move the object:

function imgComp = motionComp(imgI,u,v)

[m n C]    = size(imgI);
M          = size(imgI,1);
N          = size(imgI,2);
f          = imgI(1:M, 1:N, 1:C);
imgComp          = zeros(M, N, C);

[x y] = meshgrid(1:size(imgI,2),1:size(imgI,1));

X = min(max(x+round(u), 1), N);
Y = min(max(y+round(v), 1), N);

idx = (X(:)-1)*M + Y(:);

for coloridx = 1:C
    fc = f(:,:,coloridx);
    imgComp(:,:,coloridx) = reshape(fc(idx), M, N);
end
end

Can anybody tell me whats wrong in this logic. As the object is not moving accordingly. Please help. Thanks