Grant Number:	1R01CA190779-01 Interpret this number
Primary Investigator:	Perl, Yehoshua
Organization:	New Jersey Institute Of Technology
Project Title:	A Family-Based Framework of Quality Assurance for Biomedical Ontologies
Fiscal Year:	2015

DESCRIPTION (provided by applicant): We will develop a family-based Quality Assurance (QA) framework for biomedical ontologies. Ontology QA is critical for increasing the use of ontologies in interdisciplinary research and in electronic health records (EHRs). We will develop computational techniques for identifying concepts with high probability of errors to improve efficiency and effectiveness of ontology QA. Biomedical ontologies are large, complex knowledge representation systems that enable the integration of knowledge from different fields. The largest, best-known ontology repository is the Bioportal of the National Center for Biomedical Ontologies, containing more than 300 ontologies and tools for editing, browsing, and visualizing these ontologies. However, many errors have been discovered in BioPortal's ontologies. QA in BioPortal has been mostly focused on use-cases and ad hoc techniques. Our computational techniques will automatically identify sets of concepts with a high likelihood of errors to empower ontology QA. In past research, we have designed many QA techniques for single ontologies and have shown that sets of complex and uncommonly classified concepts have significantly higher percentages of errors. The theoretical bases for our QA are Abstraction Networks (AbNs), which summarize ontologies in a compact way. Using AbNs, we identified many error-prone concepts. In this project, we will perform QA for whole families of ontologies. We have already identified seven preliminary families, based on structural properties. If a classification of concepts yields higher than usual error rates in several ontologies of a family F then we hypothesize that this will be true for such classifications for most ontologies of F. We will build a prototype software tool (BLUOWL) for determining AbNs for each family, to support QA of its ontologies. Our primary test beds will be seven cancer-related ontologies, e.g., the National Cancer Institute thesaurus (NCIt), with different properties and purposes. Some non-cancer ontologies will also be included. We have published preliminary QA results for four such ontologies. In evaluation studies, we will formulate and test hypotheses, statistically expressing the error expectations for various kinds of concepts. Ontologies' curators were recruited to review the suspicious concepts we will identify as part of their regular QA efforts (outside of our budget). In summary, we will: Identify families of BioPortal ontologies based on ontology structure and design a unified methodology for deriving their abstraction networks; Build a software tool (BLUOWL) for QA of each family; Investigate concept classifications more likely to be erroneous in each family; Perform evaluation of our QA methodologies and usability studies for BLUOWL.




Missing lateral relationships in top-level concepts of an ontology.
Authors: Zheng L. , Chen Y. , Min H. , Hildebrand P.L. , Liu H. , Halper M. , Geller J. , de Coronado S. , Perl Y. .
Source: Bmc Medical Informatics And Decision Making, 2020-12-15 00:00:00.0; 20(Suppl 10), p. 305.
EPub date: 2020-12-15 00:00:00.0.
PMID: 33319709
Related Citations

Alternative classification of identical concepts in different terminologies: Different ways to view the world.
Authors: Keloth V.K. , He Z. , Elhanan G. , Geller J. .
Source: Journal Of Biomedical Informatics, 2019 Jun; 94, p. 103193.
EPub date: 2019-05-07 00:00:00.0.
PMID: 31048072
Related Citations

Training a Convolutional Neural Network with Terminology Summarization Data Improves SNOMED CT Enrichment.
Authors: Zheng L. , Liu H. , Perl Y. , Geller J. .
Source: Amia ... Annual Symposium Proceedings. Amia Symposium, 2019; 2019, p. 972-981.
EPub date: 2020-03-04 00:00:00.0.
PMID: 32308894
Related Citations

Transfer Learning from BERT to Support Insertion of New Concepts into SNOMED CT.
Authors: Liu H. , Perl Y. , Geller J. .
Source: Amia ... Annual Symposium Proceedings. Amia Symposium, 2019; 2019, p. 1129-1138.
EPub date: 2020-03-04 00:00:00.0.
PMID: 32308910
Related Citations

Extended Analysis of Topological-Pattern-Based Ontology Enrichment.
Authors: He Z. , Keloth V.K. , Chen Y. , Geller J. .
Source: Proceedings. Ieee International Conference On Bioinformatics And Biomedicine, 2018 Dec; 2018, p. 1641-1648.
EPub date: 2019-01-24 00:00:00.0.
PMID: 30854243
Related Citations

Complex overlapping concepts: An effective auditing methodology for families of similarly structured BioPortal ontologies.
Authors: Zheng L. , Chen Y. , Elhanan G. , Perl Y. , Geller J. , Ochs C. .
Source: Journal Of Biomedical Informatics, 2018 Jul; 83, p. 135-149.
EPub date: 2018-05-28 00:00:00.0.
PMID: 29852316
Related Citations

Statin Use and Breast Cancer Prognosis in Black and White Women.
Authors: Leiter A. , Bickell N.A. , LeRoith D. , Nayak A. , Feldman S.M. , Friedman N.B. , Estabrook A. , King T.A. , Fei K. , Franco R. , et al. .
Source: Hormones & Cancer, 2018 Feb; 9(1), p. 55-61.
EPub date: 2017-10-19 00:00:00.0.
PMID: 29052171
Related Citations

Validating UMLS Semantic Type Assignments Using SNOMED CT Semantic Tags.
Authors: Gu H. , He Z. , Wei D. , Elhanan G. , Chen Y. .
Source: Methods Of Information In Medicine, 2018 02; 57(1), p. 43-53.
EPub date: 2018-04-05 00:00:00.0.
PMID: 29621830
Related Citations

How Sustainable are Biomedical Ontologies?
Authors: Geller J. , Keloth V.K. , Musen M.A. .
Source: Amia ... Annual Symposium Proceedings. Amia Symposium, 2018; 2018, p. 470-479.
EPub date: 2018-12-05 00:00:00.0.
PMID: 30815087
Related Citations

Leveraging Horizontal Density Differences between Ontologies to Identify Missing Child Concepts: A Proof of Concept.
Authors: Keloth V.K. , He Z. , Chen Y. , Geller J. .
Source: Amia ... Annual Symposium Proceedings. Amia Symposium, 2018; 2018, p. 644-653.
EPub date: 2018-12-05 00:00:00.0.
PMID: 30815106
Related Citations

Using Convolutional Neural Networks to Support Insertion of New Concepts into SNOMED CT.
Authors: Liu H. , Geller J. , Halper M. , Perl Y. .
Source: Amia ... Annual Symposium Proceedings. Amia Symposium, 2018; 2018, p. 750-759.
EPub date: 2018-12-05 00:00:00.0.
PMID: 30815117
Related Citations

Overlapping Complex Concepts Have More Commission Errors, Especially in Intensive Terminology Auditing.
Authors: Zheng L. , Liu H. , Perl Y. , Geller J. , Ochs C. , Case J.T. .
Source: Amia ... Annual Symposium Proceedings. Amia Symposium, 2018; 2018, p. 1157-1166.
EPub date: 2018-12-05 00:00:00.0.
PMID: 30815158
Related Citations

Elevated tumor LDLR expression accelerates LDL cholesterol-mediated breast cancer growth in mouse models of hyperlipidemia.
Authors: Gallagher E.J. , Zelenko Z. , Neel B.A. , Antoniou I.M. , Rajan L. , Kase N. , LeRoith D. .
Source: Oncogene, 2017-11-16 00:00:00.0; 36(46), p. 6462-6471.
EPub date: 2017-07-31 00:00:00.0.
PMID: 28759039
Related Citations

Auditing the Assignments of Top-Level Semantic Types in the UMLS Semantic Network to UMLS Concepts.
Authors: He Z. , Perl Y. , Elhanan G. , Chen Y. , Geller J. , Bian J. .
Source: Proceedings. Ieee International Conference On Bioinformatics And Biomedicine, 2017 Nov; 2017, p. 1262-1269.
EPub date: 2017-12-18 00:00:00.0.
PMID: 29375930
Related Citations

Quality assurance of chemical ingredient classification for the National Drug File - Reference Terminology.
Authors: Zheng L. , Yumak H. , Chen L. , Ochs C. , Geller J. , Kapusnik-Uner J. , Perl Y. .
Source: Journal Of Biomedical Informatics, 2017 Sep; 73, p. 30-42.
EPub date: 2017-07-16 00:00:00.0.
PMID: 28723580
Related Citations

An empirical analysis of ontology reuse in BioPortal.
Authors: Ochs C. , Perl Y. , Geller J. , Arabandi S. , Tudorache T. , Musen M.A. .
Source: Journal Of Biomedical Informatics, 2017 Jul; 71, p. 165-177.
EPub date: 2017-06-02 00:00:00.0.
PMID: 28583809
Related Citations

From SNOMED CT to Uberon: Transferability of evaluation methodology between similarly structured ontologies.
Authors: Elhanan G. , Ochs C. , Mejino J.L.V. , Liu H. , Mungall C.J. , Perl Y. .
Source: Artificial Intelligence In Medicine, 2017-05-19 00:00:00.0; , .
EPub date: 2017-05-19 00:00:00.0.
PMID: 28532962
Related Citations

Relating Complexity and Error Rates of Ontology Concepts. More Complex NCIt Concepts Have More Errors.
Authors: Min H. , Zheng L. , Perl Y. , Halper M. , De Coronado S. , Ochs C. .
Source: Methods Of Information In Medicine, 2017-05-18 00:00:00.0; 56(3), p. 200-208.
EPub date: 2017-02-28 00:00:00.0.
PMID: 28244549
Related Citations

Analyzing Structural Changes In Snomed Ct's Bacterial Infectious Diseases Using A Visual Semantic Delta
Authors: Ochs C. , Case J.T. , Perl Y. .
Source: Journal Of Biomedical Informatics, 2017 Mar; 67, p. 101-116.
PMID: 28215561
Related Citations

Taxonomy-Based Approaches to Quality Assurance of Ontologies.
Authors: Halper M. , Perl Y. , Ochs C. , Zheng L. .
Source: Journal Of Healthcare Engineering, 2017; 2017, p. 3495723.
EPub date: 2017-10-11 00:00:00.0.
PMID: 29158885
Related Citations

Perceiving the Usefulness of the National Cancer Institute Metathesaurus for Enriching NCIt with Topological Patterns.
Authors: He Z. , Chen Y. , Geller J. .
Source: Studies In Health Technology And Informatics, 2017; 245, p. 863-867.
PMID: 29295222
Related Citations

Summarizing an Ontology: A "Big Knowledge" Coverage Approach.
Authors: Zheng L. , Perl Y. , Elhanan G. , Ochs C. , Geller J. , Halper M. .
Source: Studies In Health Technology And Informatics, 2017; 245, p. 978-982.
PMID: 29295246
Related Citations

Correcting Ontology Errors Simplifies Visual Complexity.
Authors: Liu H. , Zheng L. , Perl Y. , Chen Y. , Elhanan G. .
Source: Studies In Health Technology And Informatics, 2017; 245, p. 1330.
PMID: 29295411
Related Citations

Introducing The Big Knowledge To Use (bk2u) Challenge
Authors: Perl Y. , Geller J. , Halper M. , Ochs C. , Zheng L. , Kapusnik-Uner J. .
Source: Annals Of The New York Academy Of Sciences, 2016-10-17 00:00:00.0; , .
PMID: 27750400
Related Citations

A unified software framework for deriving, visualizing, and exploring abstraction networks for ontologies.
Authors: Ochs C. , Geller J. , Perl Y. , Musen M.A. .
Source: Journal Of Biomedical Informatics, 2016 Aug; 62, p. 90-105.
PMID: 27345947
Related Citations

Utilizing a structural meta-ontology for family-based quality assurance of the BioPortal ontologies.
Authors: Ochs C. , He Z. , Zheng L. , Geller J. , Perl Y. , Hripcsak G. , Musen M.A. .
Source: Journal Of Biomedical Informatics, 2016 Jun; 61, p. 63-76.
PMID: 26988001
Related Citations

Quality Assurance Of The Gene Ontology Using Abstraction Networks
Authors: Ochs C. , Perl Y. , Halper M. , Geller J. , Lomax J. .
Source: Journal Of Bioinformatics And Computational Biology, 2016 06; 14(3), p. 1642001.
PMID: 27301779
Related Citations

Quality Assurance of UMLS Semantic Type Assignments Using SNOMED CT Hierarchies.
Authors: Gu H. , Chen Y. , He Z. , Halper M. , Chen L. .
Source: Methods Of Information In Medicine, 2016; 55(2), p. 158-65.
EPub date: 2015-04-30 00:00:00.0.
PMID: 25925776
Related Citations

Preliminary Analysis of Difficulty of Importing Pattern-Based Concepts into the National Cancer Institute Thesaurus.
Authors: He Z. , Geller J. .
Source: Studies In Health Technology And Informatics, 2016; 228, p. 389-93.
PMID: 27577410
Related Citations

Topological-Pattern-Based Recommendation of UMLS Concepts for National Cancer Institute Thesaurus.
Authors: He Z. , Chen Y. , de Coronado S. , Piskorski K. , Geller J. .
Source: Amia ... Annual Symposium Proceedings. Amia Symposium, 2016; 2016, p. 618-627.
EPub date: 2017-02-10 00:00:00.0.
PMID: 28269858
Related Citations

Tracking the Remodeling of SNOMED CT's Bacterial Infectious Diseases.
Authors: Ochs C. , Case J.T. , Perl Y. .
Source: Amia ... Annual Symposium Proceedings. Amia Symposium, 2016; 2016, p. 974-983.
EPub date: 2017-02-10 00:00:00.0.
PMID: 28269894
Related Citations

Structural measures to track the evolution of SNOMED CT hierarchies.
Authors: Wei D. , Helen Gu H. , Perl Y. , Halper M. , Ochs C. , Elhanan G. , Chen Y. .
Source: Journal Of Biomedical Informatics, 2015 Oct; 57, p. 278-87.
PMID: 26260003
Related Citations

Summarizing and visualizing structural changes during the evolution of biomedical ontologies using a Diff Abstraction Network.
Authors: Ochs C. , Perl Y. , Geller J. , Haendel M. , Brush M. , Arabandi S. , Tu S. .
Source: Journal Of Biomedical Informatics, 2015 Aug; 56, p. 127-44.
PMID: 26048076
Related Citations

A tribal abstraction network for SNOMED CT target hierarchies without attribute relationships.
Authors: Ochs C. , Geller J. , Perl Y. , Chen Y. , Agrawal A. , Case J.T. , Hripcsak G. .
Source: Journal Of The American Medical Informatics Association : Jamia, 2015 May; 22(3), p. 628-39.
PMID: 25332354
Related Citations

Scalable quality assurance for large SNOMED CT hierarchies using subject-based subtaxonomies.
Authors: Ochs C. , Geller J. , Perl Y. , Chen Y. , Xu J. , Min H. , Case J.T. , Wei Z. .
Source: Journal Of The American Medical Informatics Association : Jamia, 2015 May; 22(3), p. 507-18.
PMID: 25336594
Related Citations

Abstraction networks for terminologies: Supporting management of "big knowledge".
Authors: Halper M. , Gu H. , Perl Y. , Ochs C. .
Source: Artificial Intelligence In Medicine, 2015 May; 64(1), p. 1-16.
PMID: 25890687
Related Citations

A comparative analysis of the density of the SNOMED CT conceptual content for semantic harmonization.
Authors: He Z. , Geller J. , Chen Y. .
Source: Artificial Intelligence In Medicine, 2015 May; 64(1), p. 29-40.
PMID: 25890688
Related Citations

Drug-drug Interaction Discovery Using Abstraction Networks for "National Drug File - Reference Terminology" Chemical Ingredients.
Authors: Ochs C. , Zheng L. , Gu H. , Perl Y. , Geller J. , Kapusnik-Uner J. , Zakharchenko A. .
Source: Amia ... Annual Symposium Proceedings. Amia Symposium, 2015; 2015, p. 973-82.
PMID: 26958234
Related Citations

Categorizing the Relationships between Structurally Congruent Concepts from Pairs of Terminologies for Semantic Harmonization.
Authors: He Z. , Geller J. , Elhanan G. .
Source: Amia Joint Summits On Translational Science Proceedings. Amia Joint Summits On Translational Science, 2014; 2014, p. 48-53.
EPub date: 2014-04-07 00:00:00.0.
PMID: 25717400
Related Citations