Mobilization of trait data from digital image files by deep learning approaches

• Applicants: Dr. Stefan Dressler (†); Professor Dr. Thomas Hickler; Dr. Jens Kattge; Professor Dr. Georg Zizka
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing ; Evolution and Systematics of Plants and Fungi ; Ecology and Biodiversity of Plants and Ecosystems
• Term: from 2016 to 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 316452578

Based upon digital images of plants (photos, specimen scans) and related taxon- and trait data we will develop and train deep-learning algorithms to recognize (a) traits in other image files and (b) use the identified traits as a step equivalent to a conventional determination process and using deep learning again to recognize taxa in image files. This approach will be used for massive mobilization of trait data for use in ecology and taxonomy, as well as for the automatic identification of species in digitized collections and citizen science portals. As part of the project will be demonstrated how (1) photos from citizen science portals and photo communities can be used by automated detection of flower and fruit traits to document the impact of climate change on phenology, (2) feature recognition can be used as a first step towards taxon recognition in citizen science portals, (3) the taxon recognition can be used for quality control and extended documentation in digitized collection data and (4) may serve to track the spread of invasive species through civil science portals and photo communities. The newly mobilized trait data will be made available to the science community via the contributing information infrastructures and via long term data repositories in the frame of GFBio.

• DFG Programme: Research data and software (Scientific Library Services and Information Systems)
• International Connection: Saudi Arabia
• Co-Investigators: Dr. Marco Schmidt; Dr. Claus Weiland
• Cooperation Partner: Professor Dr. Robert Hoehndorf