Pipeline
This paper presents REPARO, an innovative approach designed to address the challenges of compositional 3D assets generation from single images. Traditional image-to-3D models often struggle with scenes containing multiple objects due to biases and occlusion complexities.
To address this challenge, we present REPARO, a novel approach for compositional 3D asset generation from single images. REPARO employs a two-step process: first, it extracts individual objects from the scene and reconstructs their 3D meshes using off-the-shelf image-to-3D models; then, it optimizes the layout of these meshes through differentiable rendering techniques, ensuring coherent scene composition.
By integrating advanced techniques such as Optimal Transport for layout alignment and high-level semantic integration for contextual coherence, REPARO significantly enhances object independence, detail accuracy, and overall scene coherence. Extensive evaluation on multi-object scenes demonstrates REPARO's effectiveness, establishing it as a robust solution for multi-object 3D scene generation. With promising applications in augmented reality, virtual reality, gaming, and beyond, REPARO offers a comprehensive approach to address the complexities of multi-object 3D scene generation from single images.
We conducted a quantitative experiment to evaluate the performance of REPARO compared to existing methods. The results demonstrate that REPARO achieves superior accuracy and coherence in generating multi-object 3D scenes from single images.
Reference
DreamGaussian
LRM
Wonder3D
REPARO(ours)
@misc{han2024reparo,
title={REPARO: Compositional 3D Assets Generation with Differentiable 3D Layout Alignment},
author={Haonan Han and Rui Yang and Huan Liao and Jiankai Xing and Zunnan Xu and Xiaoming Yu and Junwei Zha and Xiu Li and Wanhua Li},
year={2024},
eprint={2405.18525},
archivePrefix={arXiv},
primaryClass={cs.CV}
}