Project topics and materials
Get Complete Project Material File(s) Now! »
Dynamic shape-from-silhouette
While many authors have focused on computing the visual hull in the case of static images, leading to several established techniques mentioned above, very little work has dealt with the case of dynamic scenes captured by multiple video sequences, from an actual spatio-temporal perspective, i.e. by going beyond independent frame-byframe computations.
The most notable related work is that of Cheung et al. [Cheung et al., 2005a] on computing visual hull across time. However, their purpose is not to reconstruct a dynamic scene. Their goal is to virtually increase the number of input silhouettes by registering many views of a rigidly-moving object in a common reference frame, in order to approximate the shape of the object more accurately. The virtual views are computed by estimating the motion of the object between successive frames. Unfortunately, the motion cannot be uniquely determined from only the silhouette information. In order to resolve the alignment ambiguity, they propose to combine color information with silhouette cue to extract a point cloud on the surface for each frame. The motion can then be computed by aligning the point sets. See also [Vijayakumar et al., 1996] for an early work in this direction. More related to the dynamic reconstruction is the extension of [Cheung et al.,
2005a] to articulated motion [Cheung et al., 2005b]. Under the articulated assumption, their approach exploits temporal coherence to improve shape reconstruction while estimating the skeleton of the moving object. However, their approach is experimentally validated in the case of a single joint only. Hence it is not relevant to the case of complex human motion or non-rigid scenes. In a recent work [Vlasic et al., 2008] more complex human motion is handled using silhouette geometric information. In their method visual hull is used to capture the motion of a human performer in order to make a detailed mesh animation. First, the skeletal pose is tracked, then an articulated template is deformed to t the recovered pose at each frame. The obtained meshes are therefore in full correspondence and have correct topology. This is particularly suitable for the mesh editing processes such as texturing. However, their method is limited to articulated objects and produces incorrect geometry in presence of silhouette errors. More works have been done in the domain of dynamic reconstruction. However, most of them are based on the photometric information and do not reconstruct the visual hull approximation. Hence, they do not belong to the shape-from-silhouette category and are described in a separated section 1.2.
Dynamic Multi-view reconstruction
Spatio-temporal modeling of a dynamic scene is a challenging problem which has recieved a great attention in computer vision. In recent years, several methods for automatic generation of complete models from multiple videos have been proposed. In particular, the most recent ones have proven eective for full-body marker-less motion capture of non-rigid scenes. However, many approaches make assumptions on the shape, among which the rigidity or piecewise-rigidity of the scene. In the following, dierent varieties of the problem is discussed. Existing approaches are reviewed describing dierent directions made to deal with dynamic scenes.
Convex hulls, Polytops, Simplices, and Complexes
Let M= fM1; : : : ;Mng be a set of points in Rd. The convex hull of M, conv(M), is the smallest convex set containing M. This is a polytope, the convex hull of a nite set of points (cf. Figure 2.1 for a two-dimensional example). A plane H is a supporting plane of a polytope P if it intersects P and if P lies completely in one of the two closed half-spaces bounded by H. Faces of P are its intersections with its supporting planes. In the sequel, we call k-simplex the convex hull of k + 1 anely independent points. For example, a 0-simplex is a point, a 1-simplex is a line segment, a 2-
simplex is a triangle and a 3-simplex is a tetrahedron. In this work, we also considerm 4-simplices: they are known as pentachorons or pentatopes. Any simplex dened by a subset of vertices of another simplex is a face of it.
Table of contents :
Notations
Introduction
I Multi-view Reconstruction
1 Introduction
1.1 Shape from Silhouette
1.1.1 Denition
1.1.2 State of the art
1.2 Dynamic Multi-view reconstruction
1.2.1 State of the art
1.3 Discussion and Conclusion
2 Background
2.1 Convex hulls, Polytops, Simplices, and Complexes
2.2 Voronoi Diagram
2.3 Delaunay Triangulation
3 Shape from silhouette
3.1 Introduction
3.2 Background
3.2.1 Restricted Delaunay triangulation
3.3 Methods
3.3.1 Static Visual Hull Computation
3.3.2 Spatio-Temporal Visual Hull Computation
3.3.3 Implementation Aspects
3.4 Experimental Results
3.4.1 Static 3D Visual Hull Reconstruction
3.4.2 Spatio-Temporal Visual Hull Computation
3.5 Discussion and Conclusion
3.6 Publication
4 Dynamic Multi-view Stereo
4.1 Introduction
4.1.1 Contributions
4.2 Background
4.2.1 Energy minimization via graph cuts
4.3 Method
4.3.1 4D point cloud generation, 4D Delaunay triangulation
4.3.2 4D hyper-surface extraction
4.3.3 3D surface extraction
4.4 Experimental results
4.5 Discussion and Conclusion
4.6 Publication
5 Photo-consistent Surface Reconstruction from Noisy Point Clouds
5.1 Introduction
5.2 Background
5.2.1 Power Diagram and Regular Triangulation
5.2.2 Medial axis transform
5.2.3 Poles and polar balls
5.2.4 Simulated annealing
5.3 Approach
5.3.1 Formulation
5.3.2 Energy Variation
5.3.3 Algorithm
5.4 Experimental Results
5.5 Discussion and Conclusion
5.6 Future Works
5.6.1 Dynamic Photo-consistent Surface Reconstruction from Spatio-temporal Noisy Point Clouds
5.6.2 Surface Reconstruction from Noisy Point Clouds using Power Distance
5.6.3 The Part-time Post Oce Problem
5.7 Publication
II Multi-view Texturing
6 Multiview Texturing
6.1 Introduction
6.2 Morphing images to adapt to the mesh
6.2.1 Overview of the algorithm
6.2.2 Interest point matching
6.2.3 Reprojection of 3D points through the mesh
6.2.4 Dense deformation
6.2.5 Texture mapping
6.3 Experiments
6.4 Conclusion
6.4.1 Future work
6.5 Publication
Conclusion
Bibliography
