ICBO_2018_11: Planteome & BisQue: Automating Image Annotation with Ontologies using Deep-Learning Networks


<p>The field of computer vision has recently experienced tremendous progress due to advances in deep learning. This development holds particular promise in applications for plant research, due to a significant increase in the scale of image data harvesting and a strong field-driven interest in the automated processing of observable phenotypes and visible traits within agronomically important species. Parallel developments have occurred in semantic computing; for example, new ontologies have been initiated to capture plant traits and disease indicators. When combined with existing segmentation capabilities, it is possible to conceptualize software applications that give researchers the ability to analyze large quantities of plant phenotype image data, and to auto-annotate that data with meaningful, computable semantic terminology. We have previously reported on a software application that integrates segmentation and ontologies, but lacked the ability to manage very high-resolution images, and also lacked a database platform to allow for high-volume storage requirements. We have also previously reported our migration of the AISO user-guided segmentation feature to a BisQue (Bio-Image Semantic Query User Environment) module to take advantage of its increased power, ability to scale, secure data management environment, and collaborative software ecosystem. Neither AISO nor our initial BisQue implementation possessed a machine-learning component for interpreting (parts of) images. Plant researchers could benefit greatly from a trained classification model that predicts image annotations with a high degree of accuracy. We have therefore implemented two deep-learning prototypes: a coarse classification module for plant object identification (i.e. flower, fruit) and a fine-grained classification module that focuses on plant traits (e.g. reticulate vs. parallel venation, tip shape). Both classification models return results mapped to ontology terms as a form of annotation enrichment. This current version of the Planteome Deep Segmenter module combines image classification with optional guided segmentation and ontology annotation. We have most recently run the module on local Planteome BisQue client services, and are currently working with CyVerse to install a hosted version on their BisQue client service.</p>

Year of Publication
Conference Name
International Conference on Biomedical Ontology (ICBO 2018)
Date Published
International Conference on Biological Ontology
Download citation