リアルタイムから重心を求める

8 views (last 30 days)
tsuyoshi tsunoda
tsuyoshi tsunoda on 7 Nov 2021
Commented: tsuyoshi tsunoda on 10 Nov 2021
以前、下のサイトのように画像で重心等を求めました。
今回、それをリアルタイムからの映像で行おうと思っています。
下のサイトの内容を組み込み、画像のような目の設定が出来るようになりました。
そこで今回教えていただきたいのは、目として四角で囲んだ部分の重心点を求める事です。
色々と試みたのですが、上手く反映されず出来ませんでした。
ご教授願います。
% Create the face detector object.
eyeDetector = vision.CascadeObjectDetector('EyePairBig');
mouthDetector = vision.CascadeObjectDetector('Mouth');
% Create the point tracker object.
pointTracker = vision.PointTracker('MaxBidirectionalError', 2);
% Create the webcam object.
cam = webcam();
% Capture one frame to get its size.
videoFrame = snapshot(cam);
frameSize = size(videoFrame);
% Create the video player object.
videoPlayer = vision.VideoPlayer('Position', [100 100 [frameSize(2), frameSize(1)]+30]);
runLoop = true;
numPts1 = 0;
numPts2 = 0;
while runLoop
% Get the next frame.
videoFrame = snapshot(cam);
videoFrameGray = rgb2gray(videoFrame);
if numPts1 < 10
% Detection mode.
bbox1 = eyeDetector.step(videoFrameGray);
bbox2 = mouthDetector.step(videoFrameGray);
if ~isempty(bbox1)
% Find corner points inside the detected region.
points1 = detectMinEigenFeatures(videoFrameGray, 'ROI', bbox1(1, :));
points2 = detectMinEigenFeatures(videoFrameGray, 'ROI', bbox2(1, :));
% Re-initialize the point tracker.
xyPoints1 = points1.Location;
numPts1 = size(xyPoints1,4);
release(pointTracker);
initialize(pointTracker, xyPoints1, videoFrameGray);
xyPoints2 = points2.Location;
numPts2 = size(xyPoints2,4);
release(pointTracker);
initialize(pointTracker, xyPoints2, videoFrameGray);
% Save a copy of the points.
oldPoints1 = xyPoints1;
oldPoints2 = xyPoints2;
% Convert the rectangle represented as [x, y, w, h] into an
% M-by-2 matrix of [x,y] coordinates of the four corners. This
% is needed to be able to transform the bounding box to display
% the orientation of the face.
bboxPoints1 = bbox2points(bbox1(1, :));
bboxPoints2 = bbox2points(bbox2(1, :));
% Convert the box corners into the [x1 y1 x2 y2 x3 y3 x4 y4]
% format required by insertShape.
bboxPolygon1 = reshape(bboxPoints1', 1, []);
bboxPolygon2 = reshape(bboxPoints2', 1, []);
% Display a bounding box around the detected face.
videoFrame = insertShape(videoFrame, 'Polygon', bboxPolygon1, 'LineWidth', 3);
videoFrame = insertShape(videoFrame, 'Polygon', bboxPolygon2, 'LineWidth', 3);
% Bounding Boxの位置は[x, y, width, height] ⇒ x座標に幅の1/6,4/6を足し、幅を1/6にする
lefteye = [bbox1(1)+bbox1(3)*1/6 bbox1(2) bbox1(3)/6 bbox1(4)];
righteye = [bbox1(1)+bbox1(3)*4/6 bbox1(2) bbox1(3)/6 bbox1(4)];
videoFrame = insertShape(videoFrame,'rectangle', lefteye,'Color','red','LineWidth',3);
videoFrame = insertShape(videoFrame,'rectangle',righteye,'Color','red','LineWidth',3);
end
else
% Tracking mode.
[xyPoints1, isFound1] = step(pointTracker, videoFrameGray);
visiblePoints1 = xyPoints1(isFound1, :);
oldInliers1 = oldPoints1(isFound1, :);
[xyPoints2, isFound2] = step(pointTracker, videoFrameGray);
visiblePoints2 = xyPoints2(isFound2, :);
oldInliers2 = oldPoints2(isFound2, :);
numPts1 = size(visiblePoints1, 1);
numPts2 = size(visiblePoints2, 1);
if numPts1 >= 10
% Estimate the geometric transformation between the old points
% and the new points.
[xform1, inlierIdx] = estimateGeometricTransform2D(...
oldInliers1, visiblePoints1, 'similarity', 'MaxDistance', 4);
oldInliers1 = oldInliers1(inlierIdx, :);
visiblePoints1 = visiblePoints1(inlierIdx, :);
[xform2, inlierIdx] = estimateGeometricTransform2D(...
oldInliers2, visiblePoints2, 'similarity', 'MaxDistance', 4);
oldInliers2 = oldInliers2(inlierIdx, :);
visiblePoints2 = visiblePoints2(inlierIdx, :);
% Apply the transformation to the bounding box.
bboxPoints1 = transformPointsForward(xform1, bboxPoints1);
bboxPoints2 = transformPointsForward(xform1, bboxPoints2);
% Convert the box corners into the [x1 y1 x2 y2 x3 y3 x4 y4]
% format required by insertShape.
bboxPolygon1 = reshape(bboxPoints1', 1, []);
bboxPolygon2 = reshape(bboxPoints2', 1, []);
% Display a bounding box around the face being tracked.
videoFrame = insertShape(videoFrame, 'Polygon', bboxPolygon1, 'LineWidth', 3);
videoFrame = insertShape(videoFrame, 'Polygon', bboxPolygon2, 'LineWidth', 3);
% Bounding Boxの位置は[x, y, width, height] ⇒ x座標に幅の1/6,4/6を足し、幅を1/6にする
lefteye = [bbox1(1)+bbox1(3)*1/6 bbox1(2) bbox1(3)/6 bbox1(4)];
righteye = [bbox1(1)+bbox1(3)*4/6 bbox1(2) bbox1(3)/6 bbox1(4)];
videoFrame = insertShape(videoFrame,'rectangle', lefteye,'Color','red','LineWidth',3);
videoFrame = insertShape(videoFrame,'rectangle',righteye,'Color','red','LineWidth',3);
% Reset the points.
oldPoints1 = visiblePoints1;
setPoints(pointTracker, oldPoints1);
oldPoints2 = visiblePoints1;
setPoints(pointTracker, oldPoints2);
end
end
% Display the annotated video frame using the video player object.
step(videoPlayer, videoFrame);
% Check whether the video player window has been closed.
runLoop = isOpen(videoPlayer);
end
% Clean up.
clear cam;
release(videoPlayer);
release(pointTracker);
release(eyeDetector);
release(mouthDetector);
  2 Comments
tsuyoshi tsunoda
tsuyoshi tsunoda on 9 Nov 2021
プログラムについてまとめてくださりありがとうございます。改めて勉強になりました。
両目検出して6コマに分割した中の2コマ目と5コマ目の四角の重心を求めたいという事で取り組んでいるのですが、難しく理解するのも一苦労でして、教えていただければ幸いです。

Sign in to comment.

Accepted Answer

Atsushi Ueno
Atsushi Ueno on 9 Nov 2021
【回答】Bounding Boxの重心の求め方
lefteye = bbox2points([bbox1(1,1)+bbox1(1,3)*1/6 bbox1(1,2) bbox1(1,3)/6 bbox1(1,4)]);
lefteye_cx = (lefteye(1,1) + lefteye(3,1)) / 2;
lefteye_cy = (lefteye(1,2) + lefteye(3,2)) / 2;
righteye = bbox2points([bbox1(1,1)+bbox1(1,3)*4/6 bbox1(1,2) bbox1(1,3)/6 bbox1(1,4)]);
righteye_cx = (righteye(1,1) + righteye(3,1)) / 2;
righteye_cy = (righteye(1,2) + righteye(3,2)) / 2;
Bounding Boxの回転に対応する為、bbox2points関数で[x, y, w, h]形式から[x,y]の行列に変換しています。この際、Bounding Boxの四隅が反時計回りに列挙される事が重要です。MATLABの配列は0ベースではなく1ベースなので、1番目と3番目、または2番目と4番目の座標を選択する事により対角線上の座標ペアを取得する事が出来ます。
% 直方体の重心はどちらかの対角線の中点です。重心の座標計算に異なる点のペアを使用します。
% 異なる点のペアが隣接した順序ではないことに注意してください。
% 対角線上の点は、1と2、0と3です。どちらかのペアを使用してください。
% 矩形のコーナーのリスト。[x,y]座標の4行2列の行列または[x,y]座標の4行2列のM配列として返されます.
% 矩形の出力点は,左上隅から反時計回りに列挙されます.
【参考】重心の求め方にも影響するので、Bounding Boxを回転に対応させる方法についてもコメント致します。
'EyePairBig'の検出器eyeDetectorの検出結果はbbox1([x, y, w, h])です。検出後すぐにbboxPoints1([x,y]の行列)に変換し、Bounding Box情報はこの形で検出モードと追跡モードを行き来します。検出モードではBounding Boxが垂直なのでbbox1([x, y, w, h])を'rectangle'表示しても問題ありませんが、追跡モードではBounding Boxの回転にも対応する為'Polygon'表示が必要です。bboxPolygon1([x1 y1 x2 y2 x3 y3 x4 y4])は'Polygon'表示に必要で、表示の都度bboxPoints1([x,y]の行列)から変換します。
元のプログラムを踏襲しスタイルを変えずに右目と左目に分けるなら、下記の様にしてはいかがでしょうか。
% こちらは検出モード
% Convert the rectangle represented as [x, y, w, h] into an
% M-by-2 matrix of [x,y] coordinates of the four corners. This
% is needed to be able to transform the bounding box to display
% the orientation of the face.
bboxPoints1 = bbox2points(bbox1(1, :));
lefteye = bbox2points([bbox1(1,1)+bbox1(1,3)*1/6 bbox1(1,2) bbox1(1,3)/6 bbox1(1,4)]);
righteye = bbox2points([bbox1(1,1)+bbox1(1,3)*4/6 bbox1(1,2) bbox1(1,3)/6 bbox1(1,4)]);
bboxPoints2 = bbox2points(bbox2(1, :));
% Convert the box corners into the [x1 y1 x2 y2 x3 y3 x4 y4]
% format required by insertShape.
bboxPolygon1 = reshape(bboxPoints1', 1, []);
lefteyePolygon = reshape(lefteye', 1, []);
righteyePolygon = reshape(righteye', 1, []);
bboxPolygon2 = reshape(bboxPoints2', 1, []);
% Display a bounding box around the detected face.
videoFrame = insertShape(videoFrame, 'Polygon', bboxPolygon1, 'LineWidth', 3);
videoFrame = insertShape(videoFrame, 'Polygon', lefteyePolygon, 'LineWidth', 3);
videoFrame = insertShape(videoFrame, 'Polygon', righteyePolygon, 'LineWidth', 3);
videoFrame = insertShape(videoFrame, 'Polygon', bboxPolygon2, 'LineWidth', 3);
% こちらは追跡モード
% Apply the transformation to the bounding box.
bboxPoints1 = transformPointsForward(xform1, bboxPoints1);
lefteye = transformPointsForward(xform1, lefteye);
righteye = transformPointsForward(xform1, righteye);
bboxPoints2 = transformPointsForward(xform1, bboxPoints2);
% Convert the box corners into the [x1 y1 x2 y2 x3 y3 x4 y4]
% format required by insertShape.
bboxPolygon1 = reshape(bboxPoints1', 1, []);
lefteyePolygon = reshape(lefteye', 1, []);
righteyePolygon = reshape(righteye', 1, []);
bboxPolygon2 = reshape(bboxPoints2', 1, []);
% Display a bounding box around the face being tracked.
videoFrame = insertShape(videoFrame, 'Polygon', bboxPolygon1, 'LineWidth', 3);
videoFrame = insertShape(videoFrame, 'Polygon', lefteyePolygon, 'LineWidth', 3);
videoFrame = insertShape(videoFrame, 'Polygon', righteyePolygon, 'LineWidth', 3);
videoFrame = insertShape(videoFrame, 'Polygon', bboxPolygon2, 'LineWidth', 3);
  1 Comment
tsuyoshi tsunoda
tsuyoshi tsunoda on 10 Nov 2021
とても丁寧に教えてくださりありがとうございます。勉強しながら理解していこうと思います。

Sign in to comment.

More Answers (0)

Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!