HI, I recorded sequences for offline reconstruction.I got DepthImage and ColorImage

RecFusion
1

HI,

I recorded sequences for offline reconstruction.I got DepthImage and ColorImage through readImage().
1

How do I get a nicely aligned 3D point cloud like the one in the RecFusion 3D window?

2
2390×511 118 KB

The processing I get with Open3D is terrible.

And I found that the provided method alignToColor may not work, the DepthImage data obtained is all empty.

2

With RecFusion 3D window, do you mean the view in the RecFusion application or the output of the viewer class in the RecFusion SDK? If you’re using the SDK you need to set the depth to color transformation for the reconstruction via ReconstructionParams::setDepthToColorTransformation. You can obtain this from the recorded sequence via RFSPlayback::depthToColorT.

3

Thank you very much for your prompt reply. Here is my code snippet, could you please check what I am missing, thank you very much!

RecFusion::RFSPlayback rfs;
if (!rfs.open("F:/feet-0802/3/2.rfs"))
{
	std::cerr << "" << std::endl;
	exit(EXIT_FAILURE);
}
RecFusion::DepthImage depth_image(rfs.depthWidth(), rfs.depthHeight());
RecFusion::ColorImage color_image(rfs.colorWidth(), rfs.colorHeight());
auto depthIntrinsics = rfs.intrinsics();
auto colorIntrinsics = rfs.colorIntrinsics();
auto depthToColorT = rfs.depthToColorT();
auto colorToDepthT = depthToColorT.inverse();
auto frameCount = rfs.frameCount();

for (int k = 0; k < rfs.frameCount(); ++k) {
        if (!rfs.readImage(k, depth_image, color_image)) {
            std::cerr << "Couldn't read frame number " << k << std::endl;
        }
		cv::Mat imDepth = cv::Mat(rfs.depthHeight(), rfs.depthWidth(), CV_32FC1, depthPtr);
        imDepth.convertTo(imDepth, CV_16UC1);
        cv::imwrite((std::string("F:/feet-0802/3_2_depth/frame_") + std::to_string(k) + ".tiff").c_str(),imDepth);
		
		auto depth_after = depth_image.alignToColor(m_params, 1);
		auto depth_after_ptr = depth_after->data();
        //The transformed depth image is all null
		RecFusion::PngIO::writeImage((std::string("F:/rfs-2-images/bin/Debug/depth/frame_") + std::to_string(k) + ".png").c_str(),
            depth_after->data(), rfs.colorWidth(), rfs.colorHeight(), 1, 0);
		//The transformed depth image is all null
		RecFusion::PngIO::writeImage((std::string("F:/rfs-2-images/bin/Debug/depth/frame_") + std::to_string(k) + ".png").c_str(),
            depth_after->data(), rfs.depthWidth(), rfs.depthHeight(), 1);
}
4

Can you please also post the code in which you set up m_params used in depth_image.alignToColor?

5

As per your request

    RecFusion::ReconstructionParams m_params;
    // Set reconstruction parameters
    m_params.setImageSize(rfs.colorWidth(), rfs.colorHeight(), rfs.depthWidth(), rfs.depthHeight());
    m_params.setIntrinsics(depthIntrinsics);
    m_params.setColorIntrinsics(colorIntrinsics);
    m_params.setDepthToColorTransformation(depthToColorT);
    m_params.setVolumePosition(Vec3(0, 0, 1000));
    m_params.setVolumeResolution(Vec3i(256, 256, 256));
    m_params.setVolumeSize(Vec3(1000, 1000, 1000));