High‐Resolution Neural Face Swapping for Visual Effects

Jacek Naruniec , L. Helminger , C. Schroers , Robert Weber

Abstract

In this paper, we propose an algorithm for fully automatic neural face swapping in images and videos. To the best of our knowledge, this is the first method capable of rendering photo-realistic and temporally coherent results at megapixel resolution. To this end, we introduce a progressively trained multi-way comb network and a light- and contrast-preserving blending method. We also show that while progressive training enables generation of high-resolution images, extending the architecture and training data beyond two people allows us to achieve higher fidelity in generated expressions. When compositing the generated expression onto the target face, we show how to adapt the blending strategy to preserve contrast and low-frequency lighting. Finally, we incorporate a refinement strategy into the face landmark stabilization algorithm to achieve temporal stability, which is crucial for working with high-resolution videos. We conduct an extensive ablation study to show the influence of our design choices on the quality of the swap and compare our work with popular state-of-the-art methods.
Author Jacek Naruniec (FEIT / IRMT)
Jacek Naruniec,,
- The Institute of Radioelectronics and Multimedia Technology
, L. Helminger
L. Helminger,,
-
, C. Schroers
C. Schroers,,
-
, Robert Weber
Robert Weber,,
-
Journal seriesComputer Graphics Forum, ISSN 0167-7055, e-ISSN 1467-8659
Issue year2020
Vol39
No4
Pages173-184
Publication size in sheets0.55
ASJC Classification1705 Computer Networks and Communications
DOIDOI:10.1111/cgf.14062
URL https://onlinelibrary.wiley.com/doi/full/10.1111/cgf.14062
Languageen angielski
Score (nominal)100
Score sourcejournalList
ScoreMinisterial score = 100.0, 19-08-2020, ArticleFromJournal
Publication indicators Scopus SNIP (Source Normalised Impact per Paper): 2018 = 1.567; WoS Impact Factor: 2018 = 2.373 (2) - 2018=2.46 (5)
Citation count*1 (2020-09-12)
Cite
Share Share

Get link to the record


* presented citation count is obtained through Internet information analysis and it is close to the number calculated by the Publish or Perish system.
Back
Confirmation
Are you sure?