I use demo version with one Realsense D415 with turntable (30 sec/turn) and have some questions:
after reconstruction (pg7a.jpg) I click on “re-texture” and get (pg7aTexturing.jpg) the image that is placed on the modelling is better (sharper), but not correct (two faces). How can i avoid that? Can anyone give me some advice to get or configure a good texture?
I want to use the scanned person as a character for iClone 3D animation in obj format.
I get the sample www.recfusion.net/downloads/samples/D415_person.zip (Total tris: 422505).
Can I have the sample with less tris (max. 100000)?
In this case the tracking does not seem to have worked well, which causes the texturing issues. Please make sure that there is no movement of the person during the scan and that the reconstruction FPS is at least 12.
There’s a mesh decimation option in post-processing which you can use to reduce the mesh resolution. Re-texturing needs to be done after decimation.
I have a very similar problem. I am also utilizing a turntable 30 sec/turn and want to use the scan exactly for the same purpose as you - Character for iClone 3D (.obj format). Only difference is that I am utilizing 3 Intel Realsense D435 sensors.
Could you please let me know if @ladikos tips supported you in solving the issue? I would be very grateful if you could share some details, in case you managed to make it work.
I did tests with different people but unfortunately that doesn’t work with live people (turntable 30 sec/turn). Each people always moved a little.
Just work only with object bit.
I made tests with Kinect and Microsoft 3D Scan software but the result is the same.
Result: this method is not suitable for live scanning of people.
Thank you for the feedback, and I am sad to hear about your outcome. Would you mind sharing with me which laptop you were using (if you had an external GPU, the specifications of it), and which FPS you obtained during the scanning process?
The quality of the texturing strongly depends on the tracking quality. If there’s even small errors in tracking this can lead to wrong results. Whether or not this is visually noticeable depends on where the texture mismatch happens. There’s no guaranteed way to make sure that the texturing improvement always works well. It depends on the individual acquisition. In general having the reconstruction run at a high FPS and making sure that there is no movement of the person during the scan can help. It can also help to start scanning at the back rather than at the front of the person. Both measures however only improve the chances of getting a good texturing, but do not guarantee this.
I have the next problem with the reconstruction.
I use single Realsense D415, a turntable: 30sec/turn and a motorized vertical Camera Slider (full length movement: 4:20 Min).
Settings:
Depth format: 1280x720 @ 30 fps
Color format: 1280x720 (RGB24) @ 30 fps
Volume Size: Width= 100 cm
Height= 220 cm
Depth= 70 cm
Volume Position: 0/0/100 cm
Depth Cutoff 120 cm
Start: Record sequences → test33.rfs → 0…9913 frames
The test33.rfs is later reloaded and I start the Reconstruction.
Under Reconstruction my PC:
26,0 fps, CPU= 40-50%, GPU=20-30% Reconstruction: OK
This works up to about frame 8300, but then fps suddenly drop to 8 and GPU load 70%.
What is the problem?
Thank you for the feedback.
Apparently my graphics card (NVIDIA Quatro RTX 3000 (6GB)) is overwhelmed.
Task Manager: GPU 3D over 91%.
Unfortunately I cannot use the program with my laptop.
Maybe you have a solution for me.
From the screenshot it seems that the tracking is indeed lost, there’s no green area on the depth image. When running the reconstruction on sequences all frames are used, so the FPS doesn’t really matter. Perhaps your sensor is moving too fast. In terms of the software there’s nothing we can do about this.
I’ve tested the Camera Slider at different speeds.
Full length movement:
160, 200, 242, 390, 530 sec
But the result is always the same:
The reconstruction works well up to 5-6000 frames(GPU load 40-60%), but after that the tracking is lost (GPU load 80-95%).
Is that a mistake in recording or in reconstruction?
Why does the error come after 5-6000 frames?
Maybe the reconstruction program has some buffering problem?
Hi,
Settings for test:
Camera slider full length movement: 160 sec
Turntable: 30sec/turn
Depth format: 1280x720 @ 30 fps
Color format: 1280x720 (RGB24) @ 30 fps
Volume Size: Width= 80 cm
Height= 220 cm
Depth= 70 cm
I found the next problems:
Record Sequence
After restarting the PC, approx. 10 GB/6000 frames rfs file is saved for 160 sec recording.
(test47b.rfs/10,7GB/4762 frames).
After using the software several times, the fps sequence is recorded again with the same settings and the new rfs file has doubled in size (test47.rfs/19 GB/8637 frames).
Why is this difference?
Start reconstruction
After PC restart with max. approx. 10 GB/6000 frames OK.
With larger rfs files the reconstruction over 6000 frames breaks off.
After using the software several times, the reconstruction doesn’t work with small files either.
Restarting the PC always helps.
Reconstruction settings
The person’s foot (scanning from top to bottom) or head (scanning from bottom to top) is cut off.
The modification of the size and location of the reconstruction volume doesn’t help. (test2.jpg)
If I start the reconstruction, e.g. on frame 2000, the reconstruction runs until the end.
Of course, then the initial images are missing up to frame 2000. (test2b.jpg)
Is there an upper limit for rfs file?
Is there a reason why you are not doing a live reconstruction?
RFS sequence recording depends on the available RAM and the general load. Typically the hard disk is not fast enough to save the data at the rate at which it comes in so it is buffered in memory. When we run out of memory frames get dropped, which results in the different number of images you see in the RFS file. These drops can also cause problems for the tracking from the sequence later on since the visual difference between frames is becoming too large.
You could try to capture with a lower color image resolution to save memory.
Thanks for the answer.
Now I’ve done the live reconstruction and that works much better.
Only the problem remains: for 3 minutes no person can stand on the turntable without some movement.
Everything is better when the person is seated and the scanning only takes 30 sec.
Of course, this is not a full body scan, just a head scan.
My main goal is: to create an avatar for iClone from a living person.
Is it possible to edit, repair the scanned OBJ file with a 3D software? E.g. Blender?
