Efficient Multi-image Correspondences for On-line Light Field Video Processing

Łukasz Dąbała , Matthias Ziegle , Piotr Didyk , Frederik Zilly , Joachim Keinert , Karol Myszkowski , Hans-Peter Seide , Przemysław Rokita , Tobias Ritschel


Light field videos express the entire visual information of an animated scene, but their shear size typically makes capture, processing and display an off-line process, i. e., time between initial capture and final display is far from real-time. In this paper we propose a solution for one of the key bottlenecks in such a processing pipeline, which is a reliable depth reconstruction possibly for many views. This is enabled by a novel correspondence algorithm converting the video streams from a sparse array of off-the-shelf cameras into an array of animated depth maps. The algorithm is based on a generalization of the classic multi-resolution Lucas-Kanade correspondence algorithm from a pair of images to an entire array. Special inter-image confidence consolidation allows recovery from unreliable matching in some locations and some views. It can be implemented efficiently in massively parallel hardware, allowing for interactive computations. The resulting depth quality as well as the computation performance compares favorably to other state-of-the art light field-to-depth approaches, as well as stereo matching techniques. Another outcome of this work is a data set of light field videos that are captured with multiple variants of sparse camera arrays.
Author Łukasz Dąbała (FEIT / IN)
Łukasz Dąbała,,
- The Institute of Computer Science
, Matthias Ziegle
Matthias Ziegle,,
, Piotr Didyk - [Max Planck Institut für Informatik]
Piotr Didyk,,
, Frederik Zilly - [Fraunhofer Institute for Integrated Circuits IIS]
Frederik Zilly,,
, Joachim Keinert - [Fraunhofer Institute for Integrated Circuits IIS]
Joachim Keinert,,
, Karol Myszkowski - [Max Planck Institut für Informatik]
Karol Myszkowski,,
, Hans-Peter Seide
Hans-Peter Seide,,
, Przemysław Rokita (FEIT / IN)
Przemysław Rokita,,
- The Institute of Computer Science
, Tobias Ritschel - [UCL]
Tobias Ritschel,,
Journal seriesComputer Graphics Forum, ISSN 0167-7055
Issue year2016
Publication size in sheets0.5
ASJC Classification1705 Computer Networks and Communications
URL https://diglib.eg.org:443/handle/10.1111/cgf13037
ProjectDevelopment of new algorithms in the areas of software and computer architecture, artificial intelligence and information systems and computer graphics . Project leader: Rybiński Henryk, , Phone: +48 22 234 7731, start date 18-05-2015, end date 30-11-2016, II/2015/DS/1, Completed
WEiTI Działalność statutowa
Languageen angielski
paper_CGF_vol.35_2016_1112_PRO-1.pdf 9.24 MB
Score (nominal)35
Score sourcejournalList
ScoreMinisterial score = 35.0, 24-09-2020, ArticleFromJournal
Ministerial score (2013-2016) = 35.0, 24-09-2020, ArticleFromJournal
Publication indicators Scopus Citations = 12; WoS Citations = 9; GS Citations = 26.0; Scopus SNIP (Source Normalised Impact per Paper): 2016 = 1.265; WoS Impact Factor: 2016 = 1.611 (2) - 2016=2.065 (5)
Citation count*26 (2020-09-17)
Share Share

Get link to the record

* presented citation count is obtained through Internet information analysis and it is close to the number calculated by the Publish or Perish system.
Are you sure?