Framework for Realtime 3-D Reconstruction by Space Carving Using Graphics Hardware

Inhaltsangabe:Introduction: Reconstruction of real-world scenes from a set of multiple images is a topic in Computer Vision and 3D Computer Graphics with many interesting applications. There is a relation to Augmented and Mixed Reality (AR/MR), Computer-Supported Collaborative Work (CSCW), Computer-Aided industrial/architectural Design (CAD), modeling of the real-world (e.g. computer games, scenes/effects in movies), entertainment (e.g. 3D TV/Video) and recognition/analyzing of real-world characteristics by computer systems and robots. There exists a powerful algorithm theory for shape reconstruction from arbitrary viewpoints, called shape from photo-consistency. However, it is computationally expensive and hence can not be used with applications in the field of 3D video or CSCW as well as interactive 3D model creation. Attempts have been made to achieve real-time framerates using PC cluster systems. While these provide enough performance they are also expensive and less flexible. Approaches that use GPU hardware-acceleration on single workstations achieve interactive framerates for novel-view synthesis, but do not provide an explicit volumetric representation of the whole scene. The proposed approach shows the efforts in developing a GPU hardware-accelerated framework for obtaining the volumetric photo hull of a dynamic 3D scene as seen from multiple calibrated cameras. High performance is achieved by employing a shape from silhouette technique in advance to obtain a tight initial volume for shape from photo-consistency. Also several speed-up techniques are presented to increase efficiency. Since the entire processing is done on a single PC, the framework can be applied to mobile setups, enabling a wide range of further applications. The approach is explained using programmable vertex and fragment processors and compared to highly optimized CPU implementations. It is shown that the new approach can outperform the latter by more than one magnitude. The thesis is organized as follows: Chapter 1 contains an introduction, giving an overview with classification of related techniques, statement of the main problem, novelty of the proposed approach and its fields of application. Chapter 2 surveys related work in the area of dynamic scene reconstruction by shape from silhouette and shape from photo-consistency. The focus lies on high performance reconstruction and hardware-acceleration. Chapter 3 introduces the theoretical basis for the proposed approach within three main parts. (1) Camera geometry is important to relate images captured from multiple viewpoints to an unknown 3D scene. (2) When taking a photograph of a scene, the light emitted towards the camera is captured. Therefore light and color is discussed as necessary for this work. (3) At last, the theory behind shape from silhouette and shape from photo-consistency needs to be explained. Chapter 4 continues with defining the Basic Algorithm as a hybrid approach to scene reconstruction with introducing features like (1) a robust and fast image-segmentation to account for shadows, (2) a set of nested volumes that are sequentially computed to speed-up computation and (3) an implicit visibility computation. The Basic Algorithm is extended and mapped onto the GPU in chapter 5. The corresponding Advanced Algorithm enables efficient real-time computation by employing a multipass-rendering strategy. Chapter 6 explains and discusses several experiments to analyze the proposed system in terms of performance and reconstruction quality. Chapter 7 concludes with giving a summary and discussing limitations and future works. Due to their number, diagrams and figures relating to experiments are grouped into additional appendix chapters A, B, C and D. Inhaltsverzeichnis:Table of Contents: 1.Introduction1 1.1Application1 1.2Classification2 1.3Pe...