ICBO_2018_11: Planteome & BisQue: Automating Image Annotation with Ontologies using Deep-Learning Networks

TitleICBO_2018_11: Planteome & BisQue: Automating Image Annotation with Ontologies using Deep-Learning Networks
Publication TypeConference Paper
Year of Publication2018
AuthorsTrigkakis, D, Preece, J, Meier, A, Elser, J, Kvilekval, K, Fedorov, D, Manjunath, BS, Jaiswal, P, Todorovic, S
Conference NameInternational Conference on Biomedical Ontology (ICBO 2018)
Date Published08/06/2018
PublisherInternational Conference on Biological Ontology
Keywordsannotation, convolutional neural networks, deep learning, image analysis, machine learning, Ontology, segmentation

The field of computer vision has recently experienced tremendous progress due to advances in deep learning. This development holds particular promise in applications for plant research, due to a significant increase in the scale of image data harvesting and a strong field-driven interest in the automated processing of observable phenotypes and visible traits within agronomically important species. Parallel developments have occurred in semantic computing; for example, new ontologies have been initiated to capture plant traits and disease indicators. When combined with existing segmentation capabilities, it is possible to conceptualize software applications that give researchers the ability to analyze large quantities of plant phenotype image data, and to auto-annotate that data with meaningful, computable semantic terminology. We have previously reported on a software application that integrates segmentation and ontologies, but lacked the ability to manage very high-resolution images, and also lacked a database platform to allow for high-volume storage requirements. We have also previously reported our migration of the AISO user-guided segmentation feature to a BisQue (Bio-Image Semantic Query User Environment) module to take advantage of its increased power, ability to scale, secure data management environment, and collaborative software ecosystem. Neither AISO nor our initial BisQue implementation possessed a machine-learning component for interpreting (parts of) images. Plant researchers could benefit greatly from a trained classification model that predicts image annotations with a high degree of accuracy. We have therefore implemented two deep-learning prototypes: a coarse classification module for plant object identification (i.e. flower, fruit) and a fine-grained classification module that focuses on plant traits (e.g. reticulate vs. parallel venation, tip shape). Both classification models return results mapped to ontology terms as a form of annotation enrichment. This current version of the Planteome Deep Segmenter module combines image classification with optional guided segmentation and ontology annotation. We have most recently run the module on local Planteome BisQue client services, and are currently working with CyVerse to install a hosted version on their BisQue client service.