Adrian Hilton

Research Overview

Real Face Dynamics

Research Objective: Facial animation of people with the visual quality of captured video of a real person.

Video-Rate Capture of Facial Dynamics

A system for simultaneous capture of video sequences of face shape and colour appearance. Shape capture uses a projected infra-red structured light pattern together with stereo reconstruction to simultaneously acquire full resolution shape and colour image sequences at video rate. Captured shape and colour from multiple views is transformed to a 3D Video representation of the face. 3D video sequences of speech and expression have been captured for 51 people.

I.A. Ypsilos, A. Hilton, and S. Rowe. Video-rate Capture of Dynamic Face Shape and Appearance.
IEEE Face and Gesture Recognition, 2004. pdf.

3D video of faces: Alison Speaking (avi 3Mb) Rachel Smile (avi 1Mb) Alistair Speaking (avi 2Mb) Gregor Smile (avi 1.5Mb)

Animation of Facial Dynamics

A framework for speech-driven synthesis of real faces from captured 3D video of a person speaking. Viseme primitives are identified from the captured 3D video using a phonetic decomposition of the speech. A novel algorithm using both shape and apperance is introduced to estimate the non-rigid correspondence between visemes. Face synthesis from novel speech is performed by concatenation of the correpsponding viseme sequence to produce 3D video of the person speaking.

I.A. Ypsilos, A. Hilton, A. Turkmani, and P. Jackson. Speech Driven Face Synthesis from 3D Video.
IEEE Symposium on 3D Data Processing, Visualisation and Transmission, 2004. pdf.

3D video of face synthesis: Alison (avi 3Mb) Ioannis (avi 3Mb)

Video-Based Animation

Research Objective: Video quality animation of people from video with user control of viewpoint, movement and illumination

Free-Viewpoint Video

A view-dependent optimisation approach is introduced to synthesise novel views of dynamics scenes without visual artifacts.
View-dependent optimisation of reconstructed surface shape is performed to align multiple view wide-baseline images. The reconstructed surface is initialised from the visual-hull surface and then optimised for multiple view correspondence. This approach overcomes visual artifacts which occur with visual-hull or photo-hull based view synthesis. Results are genereated for 8 camears of virtual camera moves producing a visual quality comparable to the captured multiple view video.

J. Starck and A. Hilton. Towards a 3d virtual studio for human apperance capture.
IMA International Conference on Vision, Video and Graphics, Bath, 2003. pdf.

A. Hilton and J. Starck. Multiple View Reconstruction of People.
IEEE Conference on 3D Data Processing, Visualisation and Transmission, 2004. pdf.

movies: Ella jump (avi) Ella jump frozen (avi) Adrian jump (avi) Adrian jump frozen (avi)

Modelling People from Images

Research Objective: Capture of photo-realistic 3D animated models of a persons shape, apperance and movement.

	Modelling People from Multiple Camera Views Model-based multiple view optimisation of a persons pose, shape and apperance is introduced to reconstruct animated models of people. The optimisation framework uses both visual-hull and stereo observations to reconstruct shape and texture from multiple views.The model-based framework overcomes visual ambiguities and allows wide-baseline matching. Models are reconstructed from 13 camera viewsin a chroma-key studio. J. Starck and A. Hilton. Model-based multiple view reconstruction of people. IEEE International Conference on Computer Vision, pages 915-922, 2003. pdf. Models: lady rotate, man dance, group kick Further information
	Human Motion Capture from Multiple Views A hierarchical approach has been developed to simultaneously reconstruct the movement of multiple people. Hierachical stochastic sampling is employed to search the space of human pose starting from the torso and propagating hypothesis to the limbs. This approach allows robust and efficient markerless motion capture for multiple people including person-to-person interactions and complex movement. J. Mitchelson and A. Hilton. Hierarchical tracking of multiple people. British Machine Vision Conference, 2003.pdf Y. Li, A. Hilton, and J. Illingworth. A relaxation algorithm for real-time multiview 3d-tracking. Image and Vision Computing, 2002. pd f.
	Virtual People This research introduced a model-based approach for automatic reconstruction of animated models of people using images from a single camera. A generic humanoid model is deformed to match the silhouette images of a person from front and side views. The model is then texture mapped to reproduce the apperance of the specific person. This approach was developed into an AvatarBooth system and has been used to capture of animated models of over 250K people for use in games, virtual worlds and tele-communication applications. A. Hilton, D. Beresford, T. Gentils, R. Smith, W. Sun, and J. Illingworth. Whole-body modelling of people from multi-view images to populate virtual worlds. Visual Computer: International Journal of Computer Graphics, 16(7):411--436, 2000. pdf. A. Hilton, D. Beresford, T. Gentils, R. Smith, and W. Sun. Virtual people: Capturing human models to populate virtual worlds. In IEEE International Conference on Computer Animation, pages 174--185, May 1999. pdf. Further information

Animation of Captured Data

Research Objective: Animation of real-objects from captured 3D surface measurements

From Surface Measurements to Animated Models

A model-based framework has been developed for conversion of captured 3D surface data to a form suitable for animation.
Captured data are restructured into a hierarchical layered model comprising: a skeleton model for animation control; a control surface to provide
a coarse approximation of the object geometry and allow interactive animation; and a displacement map to represent the fine detail of the captured surface geometry. A generic animated model is initially fitted to the surace measurement data to obtain a coarse approximation. Surface detail is then represented by displacement mapping using normal-volume mapping. Normal-volume mapping has been introduced to provide a continous mapping between 3D-space and the control model mesh surface. The resulting animated model provides an efficient structured representation of the captured 3D surface data suitable for animation.

J. Starck, G. Collins, R. Smith, A. Hilton, and J. Illingworth. Animated statues.
Journal of Machine Vision Applications, 14(4):248--259, 2003. pdf.

W. Sun, A. Hilton, R. Smith, and J.Illingworth. Layered animation of captured data.
Visual Computer: International Journal of Computer Graphics, 17(8):457--474, 2001. pdf.

Example: (female 12Mb)

Building Models from Surface Measurements

Research Objective: Reconstruct highly detailed models of real objects from 3D surface measurements

Hand-held 3D Capture for Object Modelling

Research result in technologies used to develop the first commercial hand-held system for capturing 3D models of real objects - 3D Scanners ModelMaker. The system captured 3D surface as the operator moves the laser stripe across the object surface in a process analagous to painting the surfaces. Algorithms were introduced for integration of captured surface measurements into a single object surface. Unlike conventional range sensor the hand-held sensor viewpoint changes over time. Therefore, novel methods were introduced based on implicit surface representations to reliably integrate overlapping measurements. This system has been used to capture models for animation production such as the BBC series 'Walking with Dinosaurs' and is now widely used in the automotive industry for product design and measurement.

A. Hilton and J. Illingworth. Geometric fusion for a hand-held 3d sensor.
Machine Vision Applications, 12(1):44-51, 2000. pdf.

Integration of Range Images

This research introduced an implicit surface based approach to integration of 3D surface measuremetns. Multiple sets of overlapping surface measurements are represented in an implicit form (f(x)=0). The normal-volume envelope of a mesh surface is used to transform an arbitrary mesh into an implicit representation. Overlapping implicit functions are then integrated into a single representation and re-triangulated to generate a single surface model. This approach provides a robust approach to integration of multiple range images.

A. Hilton, A.J. Stoddart, J. Illingworth, and T. Windeatt. Implicit surface based geometric fusion.
International Journal of Computer Vision and Image Understanding, Special Issue on CAD Based Vision, 69(3):273--291, March 1998. pdf.

Further information

Implicit Surface Polygonisation

Research Objective: Delaunay surface triangulation in 3D

Marching Triangles

This research introduced the Marching Triangles algorithm for triangulation of implicit surfaces. Marching trianges uses a 3D Delaunay sphere constraint to construct a triangulation of an implicit surface which uses approximately equilateral triangles. This approach is a surface continuation method which iteratively grows a triangulated mesh across the implicit surface according to the local 3D Delaunay constraint. This results in an efficient representation of the surface which overcomes limitations of the widely used Marching Cubes algorithms

A. Hilton, A.J. Stoddart, J. Illingworth, and T. Windeatt.
Marching triangles: Range image fusion for complex object modelling.
In International Conf. on Image Processing, pages 381--384. Lausanne, 1996.

Technical Report: pdf

Surface Curvature

Research Objective: Estimation of surface curvature from 3D surface measurements

Statistics of Surface Curvature Estimates

This research analysed the effect of measurement noise in estimation of surface curvature. Surface curvature has been widely used in the clasification of surface type from measurement data. Previous work had shown that measurement noise resulted in unrealiable classification of surface type. In this research analytic expressions for the effect of noise on suface curvature were derived and used to show the inherent ambiguities in classification of surface type from measurement data.

A. Hilton, J. Illingworth, and T. Windeatt.
Statistics of surface curvature estimates.
Pattern Recognition, 28(8):1201--1221, 1995.

Adrian Hilton