Does fragile co-adaptation occur in small datasets?

Akbar Gumbira , Rajmund Kożuszek


Recent study using AlexNet architecture and ImageNet dataset has shown that the transferability of each layer in Convolutional Neural Network (CNN) can be quantified. One interesting finding from this study is that performance degradation when performing transfer learning without finetuning is not only caused by the specificity of the features, but also due to fragile co-adapted neurons between neighboring layers. This raises a question whether this phenomenon also occurs on smaller datasets and simpler network architectures. Using CIFAR-10 and three different CNN architectures, we reported that we have not seen this effect. The drop in performance is solely due to the specificity of the features learned for the source task.
Author Akbar Gumbira (FEIT / ICS)
Akbar Gumbira,,
- The Institute of Computer Science
, Rajmund Kożuszek (FEIT / IN)
Rajmund Kożuszek,,
- The Institute of Computer Science
Publication size in sheets0.5
Book Proceedings of the 2018 Baltic URSI Symposium supported by National Committees of the Baltic Countries, vol. CFP18N89-ART, 2018, Institute of Electrical and Electronics Engineers, ISBN 978-83-949421-3-7, 300 p.
Keywords in Englishtransfer learning, fragile co-adaptation, convolutional neural network
ProjectDevelopment of new algorithms in the areas of software and computer architecture, artificial intelligence and information systems and computer graphics . Project leader: Arabas Jarosław, , Phone: +48 22 234 7432, start date 01-06-2017, end date 31-10-2018, II/2017/DS/1, Completed
WEiTI Działalność statutowa
Languageen angielski
08406745.pdf 545.2 KB
Score (nominal)20
Score sourcepublisherList
ScoreMinisterial score = 20.0, 01-02-2020, ChapterFromConference
Citation count*
Share Share

Get link to the record

* presented citation count is obtained through Internet information analysis and it is close to the number calculated by the Publish or Perish system.
Are you sure?