IMAGING AND CHARACTERIZATION OF FACIAL STRAIN IN LONG VIDEO SEQUENCES

by Matthew A. Shreve, Shaun J. Canavan, Yong Zhang, John R. Sullins, and Rupali Patil

Date Submitted: June 2007
Paper

ABSTRACT

This paper presents a method for computing strain images of a deformable object in a video sequence. The method includes two steps: in the first step, the motion data between a pair of video frames is generated using a robust optical flow al- gorithm. In the second step, a strain image is computed by applying a gradient Filter to the motion data. The efficacy of the method was demonstrated using 30 video sequences that captured human facial expressions under different light- ing conditions. Several key factors and their impact on the quality of strain images were also discussed.

Index Terms: Strain Imaging, Optical Flow, Face Video.

INTRODUCTION

Imaging objects' elastic properties based on the observed de- formation has a broad range of applications. For example, a large amount of research has been done in elastography for cancer diagnosis in the breast, kidney and heart [1, 2], because diseased tissues are correlated with change of elasticity (stiff- ness). Measuring tissue elasticity also plays an important role in biomechanical modeling for image registration and surgery planning, because modeling accuracy is dependent upon the material parameters being used [3, 4]. Strain imaging has also found applications in damage detection in composite materi- als [5]. Recently, dynamic strain images have been used in face recognition and forensic investigations [6].

There are two basic approaches to image elastic proper- ties:

  1. Recover the absolute values of elastic moduli by solving an inverse problem;
  2. Compute strain from measured displacement (motion) and then use the spatial variation of strain as an indicator of underlying tissue properties. Since an inverse problem is often ill-posed and highly nonlin- ear, the computational complexity of the first approach is rel- atively high. Various regularization techniques must be used in order to stabilize an inverse solution [7]. The second ap- proach is essentially a forward problem and therefore can be implemented with conventional image filtering methods.


Modalities that have been used in strain imaging include ultrasonic, magnetic resonance (MR) and optical sensors. Elas- tograms generated from ultrasonic and MR sensors are suit- able for examining property abnormalities of internal organs. However, ultrasonic images are plagued by artifacts while high resolution MR images are more expensive. In addition, the imaging devices are often designed to be operated in a well-controlled clinical environment, which restrict their us- age to medical fields only.

In this paper, we propose a strain imaging method that is based on the optical flow technique and the gradient filtering. The proposed method has several advantages:

  • It is efficient and can be used to process large amounts of video in a reasonable time framework. With further optimization, it can also be considered for real time ap- plications.
  • Video data can be acquired using optical camcorders. Strain images derived from those videos are adequate for many applications. The method can be used in both indoor and outdoor settings. For example, it can be used to monitor the structural fatigue and damage of endangered bridges and buildings. It can also be used to test the strength and durability of fabrics and other man-made materials.
  • Because of its non-invasive nature, the proposed method can be applied to areas besides facial strain analysis. For example, it is particularly suited for skin cancer di- agnosis and quantitative assessment of burn scars.




References

  1. J. Bishop A. Samani and D. B. Plewes, “A constrained modulus reconstruction technique for breast cancer as- sessment,” IEEE Transactions on Med. Imag., vol. 20, pp. 877-885, 2001.
  2. J. D'hooge E.E. Konofagou and J. Ophir, “Myocar- dial elastography - a feasibility study,” Ultrasound in Medicine and Biology, vol. 28, pp. 475-482, 2002.
  3. F. I. Parke and K. Waters, Computer Facial Animation, A.K. Peters, Wellesley, Massachusetts, 1997.
  4. F. R. Carls D. F. von Buren G. Fankhauser R. M. Koch, M. H. Gross and Y. I. H. Parish, “Simulating facial surgery using ?nite element models,” in Proceedings of SIGGRAPH, 1996, pp. 421-428.
  5. M. Johnson M. Kunzler, E. Udd and K. Mildenhall, “Use of multidimensional fiber grating strain sensors for damage detection in composite pressure vessels,” in Proceedings of the SPIE, 2005, vol. 5758, pp. 83-92.
  6. xxxx xxxx xxxx, xxxx, “Facial strain pattern as a soft forensic evidence,” in Proceedings of WACV-2007, Austin, Texas, Feb. 2007 (to appear).
  7. F. Kallel and M. Bertrand, “Tissue elasticity reconstruc- tion using linear perturbation method,” IEEE Trans. Medical Imaging, vol. 15, pp. 299-313, 1996.
  8. B.K.P. Horn and B.G. Schunck, “Determining optical flow,” AI Memo 572. MIT, 1980.
  9. D. J. Fleet J. L. Barron and S. S. Beauchemin, “Perfor- mance of optical ?ow techniques,” International Jour- nal of Computer Vision, vol. 12, no. 1, pp. 43-77, 1994.
  10. M. J. Black and P. Anandan, “The robust estimation of multiple motions: Parametric and piecewise-smooth flow flelds,” Computer Vision and Image Understand- ing, vol. 63, no. 1, pp. 75-104, 1996.