ReconSplat: Generalizable 3D Scene Reconstruction beyond Observed Views

TL;DR

Plausible Appearance and Accurate Geometry, jointly

Previous generative approaches can synthesize plausible appearance in unobserved regions, but the newly generated content is not accompanied by equally plausible and consistent geometry. As a result, methods such as MVSplat360 and latentSplat often produce visually convincing novel views while yielding noisy or fragmented depth. Conversely, regression-based methods such as DepthSplat recover reliable geometry from observed evidence, but cannot generate missing scene content beyond the input views.

ReconSplat closes this gap by making both appearance and geometry generative. It represents the observed scene with a multi-view-consistent variational 3DGS latent field and uses these latents to guide a multi-view diffusion model that jointly refines observed evidence and completes unobserved RGB and depth. This yields photorealistic novel views together with sharp, consistent geometry across views and modalities.

DL3DV-10K

	Novel View 1		Novel View 2
	Color	Depth	Color	Depth
ReconSplat (Ours)
MVSplat360
DepthSplat

RealEstate10K

	Novel View 1		Novel View 2
	Color	Depth	Color	Depth
ReconSplat (Ours)
latentSplat
DepthSplat

Overview

Abstract

We introduce ReconSplat, a feed-forward model for 3D scene reconstruction that aims to address the longstanding trade-off between plausible view generation for unobserved regions and geometric consistency, providing both geometrically aligned novel views and sharp depth estimates.

Our approach builds on 3D Gaussian splatting (3DGS) as an intermediate differentiable scene representation and integrates it with a multi-view latent diffusion model (MV-LDM) trained to act simultaneously as a refiner and an inpainter for appearance and scene geometry. We enforce geometric consistency by guiding the diffusion process with variational 3D latent features for appearance and geometry, encoded by the feed-forward 3DGS representation and rasterized to 2D latent space.

ReconSplat produces both photorealistic novel views and accurate depth maps on real-world benchmarks, RealEstate10K and DL3DV-10K, outperforming existing methods in challenging extrapolation setups. Notably, ReconSplat allows the extrapolation of unseen and challenging viewpoints jointly with coherent and precise scene geometry.

Approach

Method

ReconSplat architecture overview: a two-stage pipeline combining a feed-forward 3DGS encoder with a multi-view latent diffusion model.

Overview of ReconSplat. From $N$ sparse context views $\mathcal{I}^c = \{\mathbf{I}^c_i\}_{i=1}^{N}$, we regress a 3DGS representation $\mathcal{G}$ encoding variational appearance and geometry latent fields. Given target viewpoints $\mathcal{P}^{\odot}$, we rasterize these variational distributions from 3D into 2D, $\mathcal{R}(\mathcal{G} \mid \mathcal{P}^{\odot})$, and sample preliminary 2D latents $\tilde{\mathcal{Z}}^{\odot}$. A multi-view diffusion model then refines these latents into target latents $\mathcal{Z}^{\odot}$, conditioned on the context views $\mathcal{I}^c$, yielding high-quality novel views and depth maps.

Explore

Point Clouds

Unprojected color and depth predictions, as 3D point clouds you can orbit. Pick a scene on the left, and optionally toggle individual frames in the strip below. Blue borders mark the model's input views.

You can also open the viewer in a new tab.

Trajectory NVS

Video Comparisons

Novel views rendered along dense camera trajectories, with color on top and the corresponding depth below. Drag the divider in any tile to wipe between ReconSplat and the selected baseline: one divider controls both the color and the depth view, so appearance and geometry are always compared at the same camera pose. Use the arrows to step through the scenes and the buttons on top to select the highlighted baseline.

DL3DV-10K 35 scenes

RealEstate10K 12 scenes

Set NVS

Qualitative Results

Each example shows the sparse input views alongside the ground-truth target and the novel-view color/depth renderings produced by ReconSplat and by each baseline. Additional results on dense camera trajectories, rendered as videos, can be found above.

DL3DV-10K 256×448

Three scenes compared against every baseline, in color and depth. Use the Generative-based and Regression-based buttons at the top of the table to switch which two baselines are shown.


Input Views	Target View	ReconSplat (Ours)	MVSplat360DepthSplat	latentSplatpixelSplat
					Color
					Depth
					Color
					Depth
					Color
					Depth

RealEstate10K 256×256

Four scenes compared against every baseline, in color and depth. Use the Generative-based and Regression-based buttons at the top of the table to switch which two baselines are shown.


Input Views	Target View	ReconSplat (Ours)	MVSplat360DepthSplat	latentSplatpixelSplat
					Color
					Depth
					Color
					Depth
					Color
					Depth
					Color
					Depth

Evaluation

Quantitative Results

RealEstate10K

ReconSplat outperforms both regression-based and generative-based prior methods, often substantially, in the challenging extrapolation setting (except in SSIM), demonstrating the generalizability of our approach. ReconSplat also outperforms generative approaches in the interpolation setting across all metrics (except SSIM, where all generative methods are close). Bold is best, underlined is second-best.

Method	Interpolation					Extrapolation
Method	↓ FID	↓ LPIPS	↓ DISTS	↑ PSNR	↑ SSIM	↓ FID	↓ LPIPS	↓ DISTS	↑ PSNR	↑ SSIM
Regression-based
pixelSplat	4.58	0.178	0.109	24.29	0.820	11.18	0.259	0.154	20.65	0.728
MVSplat	3.49	0.161	0.097	24.12	0.845	9.83	0.245	0.145	20.49	0.752
DepthSplat	2.70	0.109	0.069	27.65	0.894	11.81	0.227	0.139	20.71	0.776
Generative-based
latentSplat	3.84	0.169	0.097	23.83	0.807	6.93	0.234	0.127	21.60	0.732
MVSplat360	4.94	0.180	0.120	22.77	0.813	5.49	0.245	0.148	20.46	0.748
ReconSplat (Ours)	4.26	0.155	0.093	24.56	0.808	4.89	0.222	0.120	21.92	0.735

DL3DV-10K

ReconSplat outperforms both regression-based and generative-based methods on reconstruction, perceptual, and FID metrics, demonstrating its generalizability. Note: DepthSplat* is trained on the full DL3DV-10K dataset (∼5× more data), making direct comparison less meaningful.

Method	Two-round (N_traj = 300)					One round (N_traj = 150)
Method	↓ FID	↓ LPIPS	↓ DISTS	↑ PSNR	↑ SSIM	↓ FID	↓ LPIPS	↓ DISTS	↑ PSNR	↑ SSIM
Regression-based
pixelSplat	132.79	0.472	0.287	17.15	0.494	108.63	0.424	0.255	18.46	0.551
MVSplat	97.73	0.476	0.235	15.59	0.358	70.23	0.429	0.191	16.14	0.356
DepthSplat*	55.84	0.297	0.155	19.41	0.633	43.67	0.221	0.115	21.51	0.716
Generative-based
latentSplat	73.08	0.411	0.182	16.32	0.472	65.04	0.363	0.164	17.50	0.535
MVSplat360	52.86	0.461	0.191	14.17	0.332	44.16	0.419	0.170	14.84	0.346
ReconSplat (Ours)	43.87	0.351	0.137	17.58	0.508	35.83	0.274	0.112	19.71	0.598

ScanNet++ novel-view depth, cross-dataset

Novel-view depth synthesis on ScanNet++, which provides ground-truth depth. This is a cross-dataset generalization setting: we evaluate the DL3DV-10K model weights directly, with no ScanNet++ training.

ReconSplat improves AbsRel, RMSE, and RMSE_log, which measure how closely the predicted depth matches the ground truth across absolute and relative depth scales. DepthSplat, in contrast, retains the best threshold accuracy δ₁. Bold indicates the best result, while underlined indicates the second-best.

Method	↑ δ₁	↓ AbsRel	↓ RMSE	↓ RMSE_log
MVSplat360	42.17	0.410	0.753	0.404
DepthSplat	70.58	0.255	0.492	0.266
ReconSplat (Ours)	67.32	0.206	0.467	0.238

Citation

BibTeX

@inproceedings{stracquadanio2026reconsplat,
  title     = {ReconSplat: Generalizable 3D Scene Reconstruction Beyond Observed Views},
  author    = {Stracquadanio, Giuseppe and Raj, Kevin and Grabinski, Julia and Roth, Stefan},
  booktitle = {European Conference on Computer Vision (ECCV)},
  year      = {2026}
}

					GT
					LVSM
					Ours