Multi-camera calibration of four D435's and bringing all the cameras to common reference axis, followed by point cloud stitching.

[manojTHI]

• Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):

  • Consider checking out SDK examples.
  • Have you looked in our documentations?
  • Is you question a frequently asked one?
  • Try searching our GitHub Issues (open and closed) for a similar issue.

• All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)

---

Required Info
Camera Model	{D435 }
Firmware Version	(Open RealSense Viewer --> Click info)
Operating System & Version	{Win 11}
Kernel Version (Linux Only)
Platform
SDK Version	{ 2.55 latest (D400) }
Language	{python}
Segment	{others }

Issue Description

Hello,
I need help with the multi-camera calibration(extrinsic), which are placed as in the image. Since the cameras are surrounded, I'm not sure which method of calibration to consider, since the calibration board is not clearly visible to all the cameras when it is placed in the center on the ground. For my application I need the cameras at a height of 10 to 12 mm from the ground in order to capture the volume of the object that would be placed in the center. My goal is to determine the extrinsic parameters of the cameras relative to each other and bring them to a common reference axis. Later for 3D reconstruction, I need to merge or stitch the point clouds of all the four cameras to form a solid shape and compute the volume of that object. (Fyi: I'm using four Realsense D435 cameras). Please help me out here, I'm using python.

Thanks,
Manoj

[MartyG-RealSense]

Hi @manojTHI The RealSense SDK provides a Python multiple-camera example program for calibrating cameras together and then calculating object volume. It is called 'box_dimensioner_multicam.py'.

https://github.com/IntelRealSense/librealsense/tree/master/wrappers/python/examples/box_dimensioner_multicam

https://dev.intelrealsense.com/docs/box-measurement-and-multi-camera-calibration

When the program launches, it first calibrates the cameras together by calculating their positions around the calibration chessboard relative to each other. After asking for an object to be placed on the chessboard, it calculates a pointcloud from the camera perspectives and uses this to calculate a green bounding-box around the object on the board. Finally, it calculates and displays on-screen the estimated volume of the object.

Placing the cameras in an elevated position at the four corners of the board about a meter away from the board's center, tilted diagonally downwards, will work well.

[manojTHI]

Actually I wanted to measure the volume of irregular shapes, will this be possible with this algorithm??

[MartyG-RealSense]

box_dimensioner_multicam does not perform so well with irregular shaped objects as a rectangular bounding box cannot fit tightly around it.

A possible approach for creating a bounding box for irregularly shaped objects is to use OpenCV contour code to form a bounding around the object from multiple rectangles and circles instead of a single rectangle. There is not information about how this approach could be applied to box_dimensioner_multicam though.

https://docs.opencv.org/3.4/da/d0c/tutorial_bounding_rects_circles.html

The 'Measure' function that is built into the 3D pointcloud mode of the RealSense Viewer tool allows you to select more than 2 coordinates on a pointcloud image to measure volume, which should work with irregular objects. I appreciate that this may not be useful for your multiple-camera application, however.