Does fragile co-adaptation occur in small datasets?

Akbar Gumbira , Rajmund Kożuszek

Abstract

Recent study using AlexNet architecture and ImageNet dataset has shown that the transferability of each layer in Convolutional Neural Network (CNN) can be quantified. One interesting finding from this study is that performance degradation when performing transfer learning without finetuning is not only caused by the specificity of the features, but also due to fragile co-adapted neurons between neighboring layers. This raises a question whether this phenomenon also occurs on smaller datasets and simpler network architectures. Using CIFAR-10 and three different CNN architectures, we reported that we have not seen this effect. The drop in performance is solely due to the specificity of the features learned for the source task.
Author Akbar Gumbira (FEIT / ICS)
Akbar Gumbira,,
- The Institute of Computer Science
, Rajmund Kożuszek (FEIT / IN)
Rajmund Kożuszek,,
- The Institute of Computer Science
Pages278-282
Publication size in sheets0.5
Book Proceedings of the Baltic URSI Symposium supported by National Committees of the Baltic Countries, vol. CFP18N89-ART, 2018, IEEE, ISBN 978-83-949421-3-7, 300 p.
Keywords in Englishtransfer learning, fragile co-adaptation, convolutional neural network
DOIDOI:10.23919/URSI.2018.8406745
projectDevelopment of new algorithms in the areas of software and computer architecture, artificial intelligence and information systems and computer graphics . Project leader: Arabas Jarosław, , Phone: +48 22 234 7432, start date 01-06-2017, end date 31-10-2018, II/2017/DS/1, Completed
WEiTI Działalność statutowa
Languageen angielski
File
08406745.pdf 545.2 KB
Score (nominal)15
ScoreMinisterial score = 15.0, BookChapterMatConf
Ministerial score (2013-2016) = 15.0, BookChapterMatConf
Citation count*
Cite
Share Share

Get link to the record


* presented citation count is obtained through Internet information analysis and it is close to the number calculated by the Publish or Perish system.
Back