How do I animate polyshape objects with translate?

I am trying to animate a scene with multiple shapes using "translate." However, whenever the new position of the polyshape overlaps the previous position only the boundary moves as shown. The "fill" part of the pshape stays at the original postion as shown:

The shape still translates ok after that, but the "fill" remains at the original position, though it does change color every time the shape translates, like so:

My code is also checking for collision, which works just fine even though the "fill" has never moved. Here is an example of that from a different run:

I thought it might be confusing the pshape object because the initial translates overlap (like adding more vertices), but that doesn't make sense since the pshape is just translating. I think attaching the pshapes to hgtransforms would work, but I don't want to add the extra code if I can avoid it. Any suggestions?

Here is my code:

figure(1);
ax = axes('XLim',[0 65],'XTick',[],'YLim',[0 65],'YTick',[]);
hold
hp1temp = polyshape([12 12 20 20],[12 22 22 12]);
hp2temp = polyshape([35 35 50 50],[35 45 45 35]);
nFrames = 50;
d1 = 2*[rand() rand()];
d2 = -1*[rand() rand()];

% Move shapes around randomly, [while staying in bounds] (TBA)
for i = 1:0.1:nFrames
    hp1 = hp1temp;
    hp2 = hp2temp;
    check = separating_axis_theorem(hp1.Vertices,hp2.Vertices);
    pt1 = plot(hp1);
    pt2 = plot(hp2);
     if(check)
        text(5,40,'Collision Detected','FontSize',20,'FontWeight','bold','Color','red');
        break
    end
   pause(0.5)
    hp1temp = translate(hp1,d1);
    hp2temp = translate(hp2,d2);
    delete(pt1);
    delete(pt2);
end