ICBO_2018_70: Improving Convergence Rates of Deep Learning for Very Small Image Training Sets


Typical visualization tasks in the domain of big image datasets include: image retrieval, clustering, and segmentation. Recently, there has been a tremendous progress in solving these tasks using deep convolutional neural networks (CNNs). CNNs typically require large training sets of manually annotated images for deep learning, which, however, is often impossible to provide in various applications, including image-based biological and medical research. This talk will address one way to address this critical issue – a new CNN learning approach, based on second-order methods, aimed at improving: a) Convergence rates of existing gradient-based methods, and b) Robustness to the choice of learning hyper-parameters (e.g., learn- ing rate). Our approach simultaneously computes both gradients and second derivatives of the CNN's learning objective, and performs second-order back-propagation. In comparison with standard gradient-based deep learning, our evaluation demonstrates that we achieve faster convergences rates, and converge to better optima leading to better performance under a budgeted time for learning.

Year of Publication
Conference Name
International Conference on Biomedical Ontology (ICBO 2018)
Date Published
International Conference on Biological Ontology
Download citation