Skip to main content
ARS Home » Northeast Area » Beltsville, Maryland (BHNRC) » Beltsville Human Nutrition Research Center » Methods and Application of Food Composition Laboratory » Research » Publications at this Location » Publication #417313
Research Project: USDA National Nutrient Databank for Food Composition

Location: Methods and Application of Food Composition Laboratory

Title: Dynamic retrieval augmented generation of ontologies using artificial intelligence (DRAGON-AI)

Author
item TORO, SABRINA - University Of North Carolina
item ANAGNOSTOPOULOS, ANNA - The Jackson Laboratory
item BELLO, SUSAN - The Jackson Laboratory
item Blumberg, Kai
item CAMERON, RHIANNON - Simon Fraser University
item CARMODY, LEIGH - The Jackson Laboratory
item DIEHL, ALEXANDER - University At Buffalo
item DOOLEY, DAMION - Simon Fraser University
item DUNCAN, WILLIAM - University Of Florida
item FEY, PETRA - Northwestern University
item GAUDET, PASCALE - Swiss Institute Of Bioinformatics
item HARRIS, NOMI - Lawrence Berkeley National Laboratory
item JOACHIMIAK, MARCIN - Lawrence Berkeley National Laboratory
item KIANI, LEILA - Collaborator
item LUBIANA, TIAGO - University Of São Paulo
item MUNOZ-TORRES, MONICA - University Of Colorado
item O'NEIL, SHAWN - University Of North Carolina
item OSUMI-SUTHERLAND, DAVID - University Of Colorado
item PUIG-BARBE, ALEIX - European Bioinformatics Institute
item REESE, JUSTIN - Lawrence Berkeley National Laboratory
item REISER, LEONORE - Phoenix Bioinformatics
item ROBB, SOFIA - Stowers Institute For Medical Research
item RUEMPING, TROY - International Center For Food Ontology Operability Data And Semantics (IC-FOODS)
item SEAGER, JAMES - Rothamsted Research
item SID, ERIC - National Center For Advancing Translational Sciences
item STEFANCSIK, RAY - European Bioinformatics Institute
item WEBER, MAGALIE - Inrae
item WOOD, VALERIE - University Of Cambridge
item HAENDEL, MELISSA - University Of North Carolina
item MUNGALL, CHRISTOPHER - Lawrence Berkeley National Laboratory

Submitted to: Journal of Biomedical Semantics
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 9/8/2024
Publication Date: 10/17/2024
Citation: Toro, S., Anagnostopoulos, A.V., Bello, S.M., Duncan, W.D., Blumberg, K.L., Mungall, C.J., Diehl, A.D., Cameron, R., Dooley, D.M., Fey, P., Harris, N.L., Joachimiak, M.P., Lubiana, T., O'Neil, S., Puig-Barbe, A., Reese, J.T., Seager, J., Weber, M., Stefancsik, R., Wood, V., Haendel, M.A., Carmody, L., Gaudet, P., Kiani, L., Munoz-Torres, M.C., Osumi-Sutherland, D., Reiser, L., Robb, S.M., Ruemping, T., Sid, E. 2024. Dynamic retrieval augmented generation of ontologies using artificial intelligence (DRAGON-AI). Journal of Biomedical Semantics. 15. Article 19. https://doi.org//10.1186/s13326-024-00320-3.
DOI: https://doi.org/10.1186/s13326-024-00320-3

Interpretive Summary: In this work we developmed and tested the efficacy of a new method to use Large Languate Models (LLMs) to help generate text definitions for terminology from open source life science vocabularies. This potentially useful as a starting point for the creation of new defintions but should always be manually checked to ensure high quality data. In our review the more confident expert reviewers were with the vocabulary in question the less often the picked a LLM generated definition.

Technical Abstract: Background Ontologies are fundamental components of informatics infrastructure in domains such as biomedical, environmental, and food sciences, representing consensus knowledge in an accurate and computable form. However, their construction and maintenance demand substantial resources and necessitate substantial collaboration between domain experts, curators, and ontology experts. We present Dynamic Retrieval Augmented Generation of Ontologies using AI (DRAGON-AI), an ontology generation method employing Large Language Models (LLMs) and Retrieval Augmented Generation (RAG). DRAGON-AI can generate textual and logical ontology components, drawing from existing knowledge in multiple ontologies and unstructured text sources. Results We assessed performance of DRAGON-AI on de novo term construction across ten diverse ontologies, making use of extensive manual evaluation of results. Our method has high precision for relationship generation, but has slightly lower precision than from logic-based reasoning. Our method is also able to generate definitions deemed acceptable by expert evaluators, but these scored worse than human-authored definitions. Notably, evaluators with the highest level of confidence in a domain were better able to discern flaws in AI-generated definitions. We also demonstrated the ability of DRAGON-AI to incorporate natural language instructions in the form of GitHub issues. Conclusions These findings suggest DRAGON-AI's potential to substantially aid the manual ontology construction process. However, our results also underscore the importance of having expert curators and ontology editors drive the ontology generation process.