GPU-Native Real-Time Textured Mesh Reconstruction with Seam-Aware Multi-View Radiometric Correction

Hiroyuki Sato
MinD in a Device Co., Ltd.
WACV 2027 (Accepted in Round 1)
Back of a person reconstructed from five cameras, without and with seam correction: patches of different brightness from different cameras disappear with the correction.

Textured mesh from an in-house five-camera capture, without and with our real-time seam correction.

Abstract

We present a GPU-native volumetric-video system for real-time reconstruction of graphics-compatible textured triangle meshes from multiple RGB-D sensors. Existing TSDF systems accelerate fusion and mesh extraction, but extending them to per-frame textured meshing adds CPU mesh processing, CPU-side texturing work, or extra CPU–GPU transfers between stages. After input staging, our method keeps the reconstruction and texturing stages (TSDF fusion, Marching Cubes, connected-component noise removal, view-based texture mapping, inpainting, and radiometric seam correction) resident on the GPU; no intermediate mesh or image data returns to the host between these stages. A fixed-size-array representation of mesh topology gives constant-time neighbor access on the GPU and makes the downstream mesh-level operations data-parallel. Because we texture each face from a single camera, exposure seams that vary smoothly across each image remain; we estimate a per-camera low-frequency gain field and bake it into the atlas on the GPU. On the CMU Panoptic dataset, our method is faster than the baselines at every tested voxel resolution and number of sensors, achieves peak speedups of more than an order of magnitude over CUDA-accelerated nvblox and CPU-based Open3D, and approaches an offline textured-meshing baseline in perceptual re-rendering quality.

Video

Method

Each frame uploads the color and depth images once. TSDF fusion, Marching Cubes, connected-component noise removal, per-face view selection, inpainting and radiometric seam correction then run on the GPU, and only the finished mesh, UVs and texture atlas return to the host. Faces store their three edge-adjacent neighbors in a fixed-size array, so neighbor access takes constant time and the mesh-level stages are data-parallel per face.

Pipeline: RGB-D input is uploaded once; TSDF fusion, Marching Cubes, noise removal, texture mapping, inpainting and radiometric seam correction run on the GPU; the mesh, UVs and corrected atlas return to the host.

Seam-aware radiometric correction

Texturing each face from a single camera leaves exposure seams where neighboring faces use different cameras. We sample both images along every seam and solve a 24×16 log-gain field per camera with smoothness and unit-gain anchor terms, using a few matrix-free Jacobi iterations on the GPU. The fields are baked into the atlas, and a moving average over frames keeps the correction stable in video.

Seam correction: seams between cameras give paired samples that constrain per-camera gain grids; the solved fields are applied to the images.

Results

Runtime

Per-frame time versus voxel size for Ours, nvblox, Open3D (CPU), Open3D (CPU, Uniform) and Open3D (CUDA).

Per-frame time versus voxel size (office2, five sensors).

Per-frame time versus number of sensors.

Per-frame time versus number of sensors (2 cm voxels).

Re-rendering from held-out cameras

We reconstruct from five Kinects and render the result from the other five, which are not used as input.

Ground-truth image of a held-out camera on office2
Ground truth
Open3D (CPU) reconstruction with per-vertex color
Open3D (CPU)
nvblox reconstruction with per-vertex color
nvblox
Our textured mesh rendered from the held-out camera
Ours (13.1 ms/frame)
Offline baseline: screened Poisson reconstruction and mvs-texturing
Offline (~13 s/frame)

office2

Ground truth, band2
Ground truth
Open3D (CPU), band2
Open3D (CPU)
nvblox, band2
nvblox
Ours, band2
Ours
Offline baseline, band2
Offline

band2

Seam correction

Held-out view without seam correction

Without seam correction

The same view with seam correction

With seam correction

Wall, ground truth
Wall: GT
Wall, without correction
w/o
Wall, with correction
w/
Floor, ground truth
Floor: GT
Floor, without correction
w/o
Floor, with correction
w/
Each input camera's image and its solved low-frequency gain field.

Each input camera (top) and its gain field (bottom; red brightens, blue darkens).

Luminance vs. RGB seam correction

On an in-house five-camera capture whose cameras run independent white balance, both the luminance-only and the per-channel RGB field equalize the seams on the back.

Without seam correction
Without
With the luminance gain field
Luminance
With the RGB gain field
RGB
Source-camera mosaic
Source camera
T-shirt, without correction
T-shirt: w/o
T-shirt, luminance field
luminance
T-shirt, RGB field
RGB
Waist, without correction
Waist: w/o
Waist, luminance field
luminance
Waist, RGB field
RGB
Each input camera: masked input view, luminance gain field and RGB gain field.

Each input camera: input with the background masked (top), luminance gain (middle) and RGB gain shown as the color it gives a neutral gray (bottom, tint exaggerated).

Noise removal and inpainting

On an in-house capture with noisy depth, noise removal suppresses isolated fragments and inpainting fills faces that no camera textures.

Texture mapping only
Texture mapping
With noise removal
+ noise removal
With noise removal and inpainting
+ inpainting

BibTeX

@InProceedings{Sato_2027_WACV,
    author    = {Sato, Hiroyuki},
    title     = {GPU-Native Real-Time Textured Mesh Reconstruction with Seam-Aware Multi-View Radiometric Correction},
    booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
    year      = {2027}
}