Imscrollpanel and axes "View" property

I am using imscrollpanel to create an interactive image annotation GUI. I would like the user to be able to zoom into the image and rotate it in different configurations. As mentioned in the documentation, I don't use the standard figure zoom with the panel, but have implemented it myself using the "windowscrollWheelFcn" to zoom as follows:

hSP=imscrollpanel(handle_imAxes.Parent,imageObject);
hSPapi=iptgetapi(hSP);

if event.VerticalScrollCount > 0
    currMag = hSPapi.getMagnification();
    newMag = currMag + 0.2*currMag;
    hSPapi.setMagnificationAndCenter(newMag,clickPt(1),clickPt(2));               
elseif event.VerticalScrollCount < 0
    currMag = hSPapi.getMagnification();
    newMag=currMag - 0.2*currMag;
    hSPapi.setMagnification(newMag);
end

handle_imAxes is the handle to the axes in which imageObject is plotted, which was obtained from a call to imagesc. clickPt contains the x and y values of the current mouse position that I get from the axes CurrentPoint property. This works ok for the most part.

My question is regarding when I need the user to rotate the image, none of this works as expected. For example, I rotate the image to a certain azimuth by calling:

handle_imAxes.View = [63,-90];

• I do this because the GUI is for annotating the image, and this way the user can still draw on the image and have the correct pixels annotated if I am only rotating the camera, not doing a transform on the image.
• When the axes' camera is rotated like this, the axes' currentPoint and hSPapi.getVisibleImageRect give me the expected values, but when I try to set the magnification, or magnification and center to the correct click point, the scroll panel centers at a completely different place, and I can't figure out a correlation. I have noticed that when I set the View, the axes property "CameraViewAngle" changes, but I'm not sure if this matters.
• I've also tried getting the View's projection matrix with T = viewmtx(obj.handle_imAxes.View) to see if there was a transformation that might work to get the correct point, but I wasn't sure how to use it or if that was the problem.

Any tips would be greatly appreciated.