# リアルタイムから重心を求める

8 views (last 30 days)
tsuyoshi tsunoda on 7 Nov 2021
Commented: tsuyoshi tsunoda on 10 Nov 2021

そこで今回教えていただきたいのは、目として四角で囲んだ部分の重心点を求める事です。

ご教授願います。
% Create the face detector object.
% Create the point tracker object.
pointTracker = vision.PointTracker('MaxBidirectionalError', 2);
% Create the webcam object.
cam = webcam();
% Capture one frame to get its size.
videoFrame = snapshot(cam);
frameSize = size(videoFrame);
% Create the video player object.
videoPlayer = vision.VideoPlayer('Position', [100 100 [frameSize(2), frameSize(1)]+30]);
runLoop = true;
numPts1 = 0;
numPts2 = 0;
while runLoop
% Get the next frame.
videoFrame = snapshot(cam);
videoFrameGray = rgb2gray(videoFrame);
if numPts1 < 10
% Detection mode.
bbox1 = eyeDetector.step(videoFrameGray);
bbox2 = mouthDetector.step(videoFrameGray);
if ~isempty(bbox1)
% Find corner points inside the detected region.
points1 = detectMinEigenFeatures(videoFrameGray, 'ROI', bbox1(1, :));
points2 = detectMinEigenFeatures(videoFrameGray, 'ROI', bbox2(1, :));
% Re-initialize the point tracker.
xyPoints1 = points1.Location;
numPts1 = size(xyPoints1,4);
release(pointTracker);
initialize(pointTracker, xyPoints1, videoFrameGray);
xyPoints2 = points2.Location;
numPts2 = size(xyPoints2,4);
release(pointTracker);
initialize(pointTracker, xyPoints2, videoFrameGray);
% Save a copy of the points.
oldPoints1 = xyPoints1;
oldPoints2 = xyPoints2;
% Convert the rectangle represented as [x, y, w, h] into an
% M-by-2 matrix of [x,y] coordinates of the four corners. This
% is needed to be able to transform the bounding box to display
% the orientation of the face.
bboxPoints1 = bbox2points(bbox1(1, :));
bboxPoints2 = bbox2points(bbox2(1, :));
% Convert the box corners into the [x1 y1 x2 y2 x3 y3 x4 y4]
% format required by insertShape.
bboxPolygon1 = reshape(bboxPoints1', 1, []);
bboxPolygon2 = reshape(bboxPoints2', 1, []);
% Display a bounding box around the detected face.
videoFrame = insertShape(videoFrame, 'Polygon', bboxPolygon1, 'LineWidth', 3);
videoFrame = insertShape(videoFrame, 'Polygon', bboxPolygon2, 'LineWidth', 3);
% Bounding Boxの位置は[x, y, width, height] ⇒ x座標に幅の1/6,4/6を足し、幅を1/6にする
lefteye = [bbox1(1)+bbox1(3)*1/6 bbox1(2) bbox1(3)/6 bbox1(4)];
righteye = [bbox1(1)+bbox1(3)*4/6 bbox1(2) bbox1(3)/6 bbox1(4)];
videoFrame = insertShape(videoFrame,'rectangle', lefteye,'Color','red','LineWidth',3);
videoFrame = insertShape(videoFrame,'rectangle',righteye,'Color','red','LineWidth',3);
end
else
% Tracking mode.
[xyPoints1, isFound1] = step(pointTracker, videoFrameGray);
visiblePoints1 = xyPoints1(isFound1, :);
oldInliers1 = oldPoints1(isFound1, :);
[xyPoints2, isFound2] = step(pointTracker, videoFrameGray);
visiblePoints2 = xyPoints2(isFound2, :);
oldInliers2 = oldPoints2(isFound2, :);
numPts1 = size(visiblePoints1, 1);
numPts2 = size(visiblePoints2, 1);
if numPts1 >= 10
% Estimate the geometric transformation between the old points
% and the new points.
[xform1, inlierIdx] = estimateGeometricTransform2D(...
oldInliers1, visiblePoints1, 'similarity', 'MaxDistance', 4);
oldInliers1 = oldInliers1(inlierIdx, :);
visiblePoints1 = visiblePoints1(inlierIdx, :);
[xform2, inlierIdx] = estimateGeometricTransform2D(...
oldInliers2, visiblePoints2, 'similarity', 'MaxDistance', 4);
oldInliers2 = oldInliers2(inlierIdx, :);
visiblePoints2 = visiblePoints2(inlierIdx, :);
% Apply the transformation to the bounding box.
bboxPoints1 = transformPointsForward(xform1, bboxPoints1);
bboxPoints2 = transformPointsForward(xform1, bboxPoints2);
% Convert the box corners into the [x1 y1 x2 y2 x3 y3 x4 y4]
% format required by insertShape.
bboxPolygon1 = reshape(bboxPoints1', 1, []);
bboxPolygon2 = reshape(bboxPoints2', 1, []);
% Display a bounding box around the face being tracked.
videoFrame = insertShape(videoFrame, 'Polygon', bboxPolygon1, 'LineWidth', 3);
videoFrame = insertShape(videoFrame, 'Polygon', bboxPolygon2, 'LineWidth', 3);
% Bounding Boxの位置は[x, y, width, height] ⇒ x座標に幅の1/6,4/6を足し、幅を1/6にする
lefteye = [bbox1(1)+bbox1(3)*1/6 bbox1(2) bbox1(3)/6 bbox1(4)];
righteye = [bbox1(1)+bbox1(3)*4/6 bbox1(2) bbox1(3)/6 bbox1(4)];
videoFrame = insertShape(videoFrame,'rectangle', lefteye,'Color','red','LineWidth',3);
videoFrame = insertShape(videoFrame,'rectangle',righteye,'Color','red','LineWidth',3);
% Reset the points.
oldPoints1 = visiblePoints1;
setPoints(pointTracker, oldPoints1);
oldPoints2 = visiblePoints1;
setPoints(pointTracker, oldPoints2);
end
end
% Display the annotated video frame using the video player object.
step(videoPlayer, videoFrame);
% Check whether the video player window has been closed.
runLoop = isOpen(videoPlayer);
end
% Clean up.
clear cam;
release(videoPlayer);
release(pointTracker);
release(eyeDetector);
release(mouthDetector);
##### 2 CommentsShowHide 1 older comment
tsuyoshi tsunoda on 9 Nov 2021
プログラムについてまとめてくださりありがとうございます。改めて勉強になりました。

Atsushi Ueno on 9 Nov 2021
【回答】Bounding Boxの重心の求め方
lefteye = bbox2points([bbox1(1,1)+bbox1(1,3)*1/6 bbox1(1,2) bbox1(1,3)/6 bbox1(1,4)]);
lefteye_cx = (lefteye(1,1) + lefteye(3,1)) / 2;
lefteye_cy = (lefteye(1,2) + lefteye(3,2)) / 2;
righteye = bbox2points([bbox1(1,1)+bbox1(1,3)*4/6 bbox1(1,2) bbox1(1,3)/6 bbox1(1,4)]);
righteye_cx = (righteye(1,1) + righteye(3,1)) / 2;
righteye_cy = (righteye(1,2) + righteye(3,2)) / 2;
Bounding Boxの回転に対応する為、bbox2points関数で[x, y, w, h]形式から[x,y]の行列に変換しています。この際、Bounding Boxの四隅が反時計回りに列挙される事が重要です。MATLABの配列は0ベースではなく1ベースなので、1番目と3番目、または2番目と4番目の座標を選択する事により対角線上の座標ペアを取得する事が出来ます。
% 直方体の重心はどちらかの対角線の中点です。重心の座標計算に異なる点のペアを使用します。
% 異なる点のペアが隣接した順序ではないことに注意してください。
% 対角線上の点は、1と2、0と3です。どちらかのペアを使用してください。
% 矩形のコーナーのリスト。[x,y]座標の4行2列の行列または[x,y]座標の4行2列のM配列として返されます．
% 矩形の出力点は，左上隅から反時計回りに列挙されます．
【参考】重心の求め方にも影響するので、Bounding Boxを回転に対応させる方法についてもコメント致します。
'EyePairBig'の検出器eyeDetectorの検出結果はbbox1([x, y, w, h])です。検出後すぐにbboxPoints1([x,y]の行列)に変換し、Bounding Box情報はこの形で検出モードと追跡モードを行き来します。検出モードではBounding Boxが垂直なのでbbox1([x, y, w, h])を'rectangle'表示しても問題ありませんが、追跡モードではBounding Boxの回転にも対応する為'Polygon'表示が必要です。bboxPolygon1([x1 y1 x2 y2 x3 y3 x4 y4])は'Polygon'表示に必要で、表示の都度bboxPoints1([x,y]の行列)から変換します。

% こちらは検出モード
% Convert the rectangle represented as [x, y, w, h] into an
% M-by-2 matrix of [x,y] coordinates of the four corners. This
% is needed to be able to transform the bounding box to display
% the orientation of the face.
bboxPoints1 = bbox2points(bbox1(1, :));
lefteye = bbox2points([bbox1(1,1)+bbox1(1,3)*1/6 bbox1(1,2) bbox1(1,3)/6 bbox1(1,4)]);
righteye = bbox2points([bbox1(1,1)+bbox1(1,3)*4/6 bbox1(1,2) bbox1(1,3)/6 bbox1(1,4)]);
bboxPoints2 = bbox2points(bbox2(1, :));
% Convert the box corners into the [x1 y1 x2 y2 x3 y3 x4 y4]
% format required by insertShape.
bboxPolygon1 = reshape(bboxPoints1', 1, []);
lefteyePolygon = reshape(lefteye', 1, []);
righteyePolygon = reshape(righteye', 1, []);
bboxPolygon2 = reshape(bboxPoints2', 1, []);
% Display a bounding box around the detected face.
videoFrame = insertShape(videoFrame, 'Polygon', bboxPolygon1, 'LineWidth', 3);
videoFrame = insertShape(videoFrame, 'Polygon', lefteyePolygon, 'LineWidth', 3);
videoFrame = insertShape(videoFrame, 'Polygon', righteyePolygon, 'LineWidth', 3);
videoFrame = insertShape(videoFrame, 'Polygon', bboxPolygon2, 'LineWidth', 3);
% こちらは追跡モード
% Apply the transformation to the bounding box.
bboxPoints1 = transformPointsForward(xform1, bboxPoints1);
lefteye = transformPointsForward(xform1, lefteye);
righteye = transformPointsForward(xform1, righteye);
bboxPoints2 = transformPointsForward(xform1, bboxPoints2);
% Convert the box corners into the [x1 y1 x2 y2 x3 y3 x4 y4]
% format required by insertShape.
bboxPolygon1 = reshape(bboxPoints1', 1, []);
lefteyePolygon = reshape(lefteye', 1, []);
righteyePolygon = reshape(righteye', 1, []);
bboxPolygon2 = reshape(bboxPoints2', 1, []);
% Display a bounding box around the face being tracked.
videoFrame = insertShape(videoFrame, 'Polygon', bboxPolygon1, 'LineWidth', 3);
videoFrame = insertShape(videoFrame, 'Polygon', lefteyePolygon, 'LineWidth', 3);
videoFrame = insertShape(videoFrame, 'Polygon', righteyePolygon, 'LineWidth', 3);
videoFrame = insertShape(videoFrame, 'Polygon', bboxPolygon2, 'LineWidth', 3);
tsuyoshi tsunoda on 10 Nov 2021
とても丁寧に教えてくださりありがとうございます。勉強しながら理解していこうと思います。