Skip to content

About

No description, website, or topics provided.

Resources

Stars

41 stars

Watchers

6 watching

Forks

Latest commit

Β 

History

8 Commits

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 

Repository files navigation

ReconDreamer++: Harmonizing Generative and Reconstructive Models for Driving Scene Representation

News

  • [2025/03/24] Repository Initialization.

Abstract

Combining reconstruction models with generative models has emerged as a promising paradigm for closed-loop simulation in autonomous driving. For example, ReconDreamer has demonstrated remarkable success in rendering large-scale maneuvers. However, a significant gap remains between the generated data and real-world sensor observations, particularly in terms of fidelity for structured elements, such as the ground surface. To address these challenges, we propose ReconDreamer++, an enhanced framework that significantly improves the overall rendering quality by mitigating the domain gap and refining the representation of the ground surface. Specifically, ReconDreamer++ introduces the Novel Trajectory Deformable Network (NTDNet), which leverages learnable spatial deformation mechanisms to bridge the domain gap between synthesized novel views and original sensor observations. Moreover, for structured elements such as the ground surface, we preserve geometric prior knowledge in 3D Gaussians, and the optimization process focuses on refining appearance attributes while preserving the underlying geometric structure. Experimental evaluations conducted on multiple datasets (Waymo, nuScenes, PandaSet, and EUVS) confirm the superior performance of ReconDreamer++. Specifically, on Waymo, ReconDreamer++ achieves performance comparable to Street Gaussians for the original trajectory while significantly outperforming ReconDreamer on novel trajectories. In particular, it achieves substantial improvements, including a 6.1% increase in NTA-IoU, a 23. 0% improvement in FID, and a remarkable 4.5% gain in the ground surface metric NTL-IoU, highlighting its effectiveness in accurately reconstructing structured elements such as the road surface.

key_feature

Comparison of ReconDreamer++ with SOTA methods, Street Gaussians and ReconDreamer, on original and novel trajectories. Left: ReconDreamer++ demonstrates superior rendering performance for both vehicle foregrounds and road surfaces compared to existing SOTA methods. Right: ReconDreamer++ significantly improves performance on novel trajectories while maintaining high rendering quality on the original trajectory.

ReconDreamer++ Framework

framework

Rendering Results on Various Datasets

Waymo

Original Trajectory
reconstruction_comparison.mp4
Lane Shift @ 3m
lane_shift_left_3m_comparison.mp4
Lane Shift @ 6m
lane_shift_left_6m_comparison.mp4

nuScenes

Original Trajectory
reconstruction_comparison.mp4
Lane Shift @ 3m
lane_shift_left_3m_comparison.mp4
Lane Shift @ 6m
lane_shift_left_6m_comparison.mp4

PandaSet

Original Trajectory
reconstruction_comparison.mp4
Lane Shift @ 2m
lane_shift_left_2m_comparison.mp4
Lane Shift @ 3m
lane_shift_left_3m_comparison.mp4

EUVS

Train Set
train_set_comparison.mp4
Test Set
test_set_comparison.mp4

Ablation Studies on depth loss, ground model and NTDNet.

Original Trajectory
reconstruction_comparison.mp4
Lane Shift @ 3m
lane_shift_left_3m_comparison.mp4
Lane Shift @ 6m
lane_shift_left_6m_comparison.mp4

Acknowledgements

We would like to thank the following works and projects, for their open research and exploration: DriveStudio, DriveDreamer, DriveDreamer-2, DriveDreamer4D, ReconDreamer.

Bibtex

If this work is helpful for your research, please consider citing the following BibTeX entry.

@inproceedings{zhao2025recon,
    title={ReconDreamer++: Harmonizing Generative and Reconstructive Models for Driving Scene Representation}, 
    author={Guosheng Zhao and Xiaofeng Wang and Chaojun Ni and Zheng Zhu and and Wenkang Qin and Guan Huang and Xingang Wang},
    journal={arxiv arXiv preprint arXiv:2503.18438},
    year={2025},
}


About

No description, website, or topics provided.

Resources

Stars

41 stars

Watchers

6 watching

Forks

Releases

Packages

Contributors