HoloFace: Augmenting Human – to Human Interactions on HoloLens

Marek Kowalski , Zbigniew Nasarzewski , Grzegorz Galiński , Piotr Garbat

Abstract

We present HoloFace, an open-source framework for face alignment, head pose estimation and facial attribute retrieval for Microsoft HoloLens. HoloFace implements two state-of-the-art face alignment methods which can be used interchangeably: one running locally and one running on a remote backend. Head pose estimation is accomplished by fitting a deformable 3D model to the landmarks localized using face alignment. The head pose provides both the rotation of the head and a position in the world space. The parameters of the fitted 3D face model provide estimates of facial attributes such as mouth opening or smile. Together the above information can be used to augment the faces of people seen by the HoloLens user, and thus their interaction. Potential usage scenarios include facial recognition, emotion recognition, eye gaze tracking and many others. We demonstrate the capabilities of our framework by augmenting the faces of people seen through the HoloLens with various objects and animations.
Author Marek Kowalski (FEIT / IRMT)
Marek Kowalski,,
- The Institute of Radioelectronics and Multimedia Technology
, Zbigniew Nasarzewski (FEIT / IRMT)
Zbigniew Nasarzewski,,
- The Institute of Radioelectronics and Multimedia Technology
, Grzegorz Galiński (FEIT / IRMT)
Grzegorz Galiński,,
- The Institute of Radioelectronics and Multimedia Technology
, Piotr Garbat (FEIT / MO)
Piotr Garbat,,
- The Institute of Microelectronics and Optoelectronics
Pages141-149
Publication size in sheets0.5
Book Proceedings of 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018, vol. CFP18082-ART, 2018, IEEE, ISBN 978-1-5386-4886-5, 2100 p.
DOIDOI:10.1109/WACV.2018.00022
URL https://ieeexplore.ieee.org/document/8354126/
Languageen angielski
Score (nominal)15
ScoreMinisterial score = 15.0, 09-08-2018, BookChapterMatConf
Ministerial score (2013-2016) = 15.0, 09-08-2018, BookChapterMatConf
Citation count*
Cite
Share Share

Get link to the record


* presented citation count is obtained through Internet information analysis and it is close to the number calculated by the Publish or Perish system.
Back