烈火灼冰 - AIGC
type: channel
link: https://blog.4c43.work/index.php/tag/AIGC/
items: 【学术成果展示】【ECCV2024】DreamScene: 3D Gaussian-based Text-to-3D Scene Generation via Formation Pattern Sampling

【学术成果展示】【ECCV2024】DreamScene: 3D Gaussian-based Text-to-3D Scene Generation via Formation Pattern Sampling
type: item
date: 2024-07-01T10:00:00+00:00
description: DreamScene: 3D Gaussian-based Text-to-3D Scene Generation via Formation Pattern Sampling 基于三维高斯和形成模式采样的三维物体及场景生成方法。 ECCV 2024, Milano Haoran Li1, 2, Haolin Shi1, 2, Wenli Zhang1, 2, Wenjun Wu1, 2, Yong Liao1, 2*, Lin Wang3, Lik-Hang Lee4, Peng Yuan Zhou5* 1 University of Science and Technology of China 2 CCCD Key Lab of Ministry of Culture and Tourism 3 AI Thrust, HKUST(GZ) and Dept. of Computer Science Eng., HKUST 4 The Hong Kong Polytechnic University 5 Aarhus University arXiv:2404.02905 Project Page: dreamscene-project.github.io Code: DreamScene-Project/DreamScene Abstract Text-to-3D scene generation holds immense potential for the gaming, film, and architecture sectors. Despite significant progress, existing methods struggle with maintaining high quality, consistency, and editing flexibility. In this paper, we propose DreamScene, a 3D Gaussian-based novel text-to-3D scene generation framework, to tackle the aforementioned three challenges mainly via two strategies. First, DreamScene employs Formation Pattern Sampling (FPS), a multi-timestep sampling strategy guided by the formation patterns of 3D objects, to form fast, semantically rich, and high-quality representations. FPS uses 3D Gaussian filtering for optimization stability, and leverages reconstruction techniques to generate plausible textures. Second, DreamScene employs a progressive three-stage camera sampling strategy, specifically designed for both indoor and outdoor settings, to effectively ensure object-environment integration and scene-wide 3D consistency. Last, DreamScene enhances scene editing flexibility by integrating objects and environments, enabling targeted adjustments. Extensive experiments validate DreamScene's superiority over current state-of-the-art techniques, heralding its wide-ranging potential for diverse applications. Text-to-3D (Scenes) High quality, scene-wide 3D consistency, scene editing flexibility, accommodating a vast number of objects. A DSLR photo of a living room A DSLR ph…