Cross3DGS

Unified Vulkan-Based 3D Gaussian Splatting Training
from Workstations to Mobile Devices

Hiroyuki Sato, Yusuke Takimoto
Capture, sparse SfM, 3DGS optimization, and free-view rendering, all on a mobile device.

An end-to-end 3DGS reconstruction application running entirely on a mobile device: capture, sparse structure from motion, on-device 3DGS optimization, and interactive viewing. Available for both Android and iOS.

Abstract

Scenes captured with handheld and wearable cameras are still shipped to a CUDA workstation for 3D Gaussian Splatting (3DGS) training. Training on the device that captured them, or on its companion phone or laptop, requires more than translating kernels: GPUs and drivers differ in parallel features, memory limits, and command submission, and these differences decide whether a training schedule executes and returns the correct update.

We present Cross3DGS, a cross-vendor, cross-platform Vulkan trainer that keeps one optimizer and chooses its execution schedule per device in two steps: rules on device limits, workload size, and observed driver failures exclude the schedules known not to run correctly, and a fixed preference order selects one of the remaining feasible schedules. The same core completes standard-quality training (30k updates, one million Gaussians) on every configuration we measured, spanning NVIDIA, Intel, Arm, Qualcomm, AMD, and Apple GPUs and including Android devices and an iPhone, whereas the existing portable trainers VkSplat and Brush do not run on some of the mobile GPUs.

Averaged over 11 scenes on a workstation, Cross3DGS has higher quality than VkSplat with no significant difference in training time, and Brush reports about three times the training time at comparable quality. The final quality of Cross3DGS is nearly identical across devices. A mobile application turns a handheld capture into a viewable model entirely on the device.

Mobile Application: One Codebase, Android and iOS

SfM and 3DGS training are the same native code on both platforms; only the camera capture (ARCore / ARKit) and a thin UI (Kotlin / Objective-C++) are platform-specific.

The same capture-to-view application on Android (Xiaomi 17T Pro, left; Xiaomi Pad 6S Pro 12.4, middle) and iOS (iPhone 12 Pro, right): capture, sparse SfM, 3DGS optimization, and viewing, all on the device. Stages are time-lapsed per device (factor shown under each); the clock is real time since capture start, and the viewer plays in real time. These are 60-frame runs with the screen recorder on; the paper reports 100-frame sessions without it.

Two outdoor scenes, a shrine (left) and a stone statue (right), processed on the Android phone (Xiaomi 17T Pro) from captures saved on it: frame extraction, sparse SfM, 3DGS optimization, and viewing, all on the device. Stages are time-lapsed as above; the clock is real time since frame extraction started. These are 60-frame runs with the screen recorder on, so their times differ from the numbers in the paper. In the stone-statue recording, pedestrians and vehicles are blurred in post-processing.

One Trainer Across Six GPU Vendors

Existing 3DGS trainers are tied to CUDA or fail on some devices; Cross3DGS runs one optimizer on Vulkan across NVIDIA, AMD, Intel, Arm, Qualcomm, and Apple GPUs.
CUDA trainers run only on NVIDIA. The portable trainers VkSplat (Vulkan) and Brush (WebGPU) fail on some of our devices. VkSplat checks device limits and drops the gradient kernels that exceed shared memory before timing the rest; Brush runs one gradient kernel with native or CAS atomics. Cross3DGS extends such checks to all six schedule options, adds the workload and observed driver failures, picks one feasible schedule with a fixed preference order, and runs on Vulkan from workstations to phones.

Method

One optimizer, many devices. Instead of hand-tuning a separate port per GPU, Cross3DGS keeps a catalog of execution schedules and picks one per device. Step 1 drops every schedule that fails a feasibility rule on the device limits, the workload, and known driver failures. Step 2 picks one of the remaining schedules with a fixed preference order. Every gradient method in the catalog is checked offline against the reference update.

Two-step schedule selection: feasibility rules, then a fixed preference order.
Choosing a schedule for a device.

Results

Full training on all eight tested configurations, at the same quality

Execution coverage, training time, and PSNR on bicycle across devices and methods.
Standard-quality time, quality, and execution coverage on bicycle. Left: which method runs on which GPU. Right: three-run median training time (log scale) against mean-view PSNR; color marks the GPU and marker shape the method. The shaded band is the eight-device Cross3DGS PSNR span.
Renders and absolute error maps on bicycle for three devices and two baselines.
Standard-quality renders and error maps on bicycle for three devices whose deployed gradient accumulation differs, and for VkSplat and Brush on the workstation GPU.

On-device capture to view

Held-out targets, on-device reconstructions, and error maps for mobile captures.
A plush bear reconstructed on an iPhone, an Android phone, and an Android tablet (first three rows, each from its own capture), followed by two outdoor scenes on the Android phone at the demonstration and standard-quality settings. Each row shows a held-out target, its reconstruction, and a mean absolute RGB error map. The plush-bear frames are shown as recorded.

BibTeX

@InProceedings{Sato_2027_WACV_Cross3DGS,
    author    = {Sato, Hiroyuki and Takimoto, Yusuke},
    title     = {Cross3DGS: Unified Vulkan-Based 3D Gaussian Splatting Training from Workstations to Mobile Devices},
    booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Workshops},
    year      = {2027}
}