Grant Number:	5U24HG007346-05 Interpret this number
Primary Investigator:	Brenner, Steven
Organization:	University Of California Berkeley
Project Title:	Center for Critical Assessment of Genome Interpretation
Fiscal Year:	2021

Genomic data hold the promise of revolutionizing our understanding and treatment of human disease. Multiple barriers stand between the acquisition of the data and realizing these and other benefits. Rapid accumulation of genomic data far exceeds our capacity to reliably interpret genomic variation. New developments in artificial intelligence and machine learning, combined with increased computing power and domain knowledge, provide hope for the deployment of enhanced computational tools in both basic research and clinical practice. Use of these methods critically depends upon reliable characterization of their performance. The Center for Critical Assessment of Genome Interpretation (C-CAGI) will address these needs, through objective evaluation of the state of the art in relating human genetic variation and health. CAGI has had five editions since 2010 with 50 challenges posed to the community taken on by hundreds of predictors, leading to scores of publications about prediction methods and their assessment. We propose for C-CAGI to continue to advance the field of variant interpretation through the following Specific Aims: 1. Develop community experiments to evaluate the quality of computational methods for interpreting genomic variation data. C-CAGI will conduct community experiments in which participants make bona fide blinded predictions of disease related phenotypes on the basis of genomic data. We will engage a diverse predictor community to spur innovation. The CAGI Ethics Forum will vet studies to ensure that privacy and sharing maintain the highest standards and will educate the community. 2. Assess the quality of current computational methods for interpreting genomic variation data; highlight innovations and progress at interactive conferences. Predictions will be evaluated by independent assessors, who will be supported by new assessment approaches from C-CAGI. Results will be presented at CAGI experiment conferences with deep technical engagement, which will be interleaved with reflective CAGIâ meetings that create an environment for a comprehensive evaluation of the field, facilitating identification of major bottlenecks and problems faced by the current genome interpretation approaches. 3. Broadly disseminate the results and conclusions from the CAGI experiments and analysis. C-CAGI will outreach to the broader scientific and clinical community through its publications, and the creation of a calibrated reference integrated into the most common workflows for ready adoption. CAGI will also be represented at international meetings with presentations and workshops. 4. Operate effectively and responsively. C-CAGI will operate efficiently as it closely interacts with hundreds of participants. CAGI will build upon a robust information infrastructure that securely facilitates data dissemination, prediction submission, and assessment.




Critical assessment of missense variant effect predictors on disease-relevant variant data.
Authors: Rastogi R. , Chung R. , Li S. , Li C. , Lee K. , Woo J. , Kim D.W. , Keum C. , Babbi G. , Martelli P.L. , et al. .
Source: Human Genetics, 2025-03-21 00:00:00.0; , .
EPub date: 2025-03-21 00:00:00.0.
PMID: 40113603
Related Citations

Evaluation of enzyme activity predictions for variants of unknown significance in Arylsulfatase A.
Authors: Jain S. , Trinidad M. , Nguyen T.B. , Jones K. , Neto S.D. , Ge F. , Glagovsky A. , Jones C. , Moran G. , Wang B. , et al. .
Source: Human Genetics, 2025-03-08 00:00:00.0; , .
EPub date: 2025-03-08 00:00:00.0.
PMID: 40055237
Related Citations

Assessing the predicted impact of single amino acid substitutions in MAPK proteins for CAGI6 challenges.
Authors: Turina P. , Petrosino M. , Enriquez Sandoval C.A. , Novak L. , Pasquo A. , Alexov E. , Alladin M.A. , Ascher D.B. , Babbi G. , Bakolitsa C. , et al. .
Source: Human Genetics, 2025-02-20 00:00:00.0; , .
EPub date: 2025-02-20 00:00:00.0.
PMID: 39976676
Related Citations

Evaluating predictors of kinase activity of STK11 variants identified in primary human non-small cell lung cancers.
Authors: Chen Y. , Lee K. , Woo J. , Kim D.W. , Keum C. , Babbi G. , Casadio R. , Martelli P.L. , Savojardo C. , Manfredi M. , et al. .
Source: Human Genetics, 2025-02-12 00:00:00.0; , .
EPub date: 2025-02-12 00:00:00.0.
PMID: 39934475
Related Citations

An augmented transformer model trained on protein family specific variant data leads to improved prediction of variants of uncertain significance.
Authors: Joshi D. , Pradhan S. , Sajeed R. , Srinivasan R. , Rana S. .
Source: Human Genetics, 2025-01-27 00:00:00.0; , .
EPub date: 2025-01-27 00:00:00.0.
PMID: 39869148
Related Citations

CAGI6 ID panel challenge: assessment of phenotype and variant predictions in 415 children with neurodevelopmental disorders (NDDs).
Authors: Aspromonte M.C. , Del Conte A. , Zhu S. , Tan W. , Shen Y. , Zhang Y. , Li Q. , Wang M.H. , Babbi G. , Bovo S. , et al. .
Source: Human Genetics, 2025-01-09 00:00:00.0; , .
EPub date: 2025-01-09 00:00:00.0.
PMID: 39786577
Related Citations

Assessing the predicted impact of single amino acid substitutions in calmodulin for CAGI6 challenges.
Authors: Turina P. , Dal Cortivo G. , Enriquez Sandoval C.A. , Alexov E. , Ascher D.B. , Babbi G. , Bakolitsa C. , Casadio R. , Fariselli P. , Folkman L. , et al. .
Source: Human Genetics, 2024-12-23 00:00:00.0; , .
EPub date: 2024-12-23 00:00:00.0.
PMID: 39714488
Related Citations

Variant Impact Predictor database (VIPdb), version 2: trends from three decades of genetic variant impact predictors.
Authors: Lin Y.J. , Menon A.S. , Hu Z. , Brenner S.E. .
Source: Human Genomics, 2024-08-28 00:00:00.0; 18(1), p. 90.
EPub date: 2024-08-28 00:00:00.0.
PMID: 39198917
Related Citations

Rapid discrimination between deleterious and benign missense mutations in the CAGI 6 experiment.
Authors: Faraggi E. , Jernigan R.L. , Kloczkowski A. .
Source: Human Genomics, 2024-08-27 00:00:00.0; 18(1), p. 89.
EPub date: 2024-08-27 00:00:00.0.
PMID: 39192324
Related Citations

Structure-informed protein language models are robust predictors for variant effects.
Authors: Sun Y. , Shen Y. .
Source: Human Genetics, 2024-08-08 00:00:00.0; , .
EPub date: 2024-08-08 00:00:00.0.
PMID: 39117802
Related Citations

Assessing predictions on fitness effects of missense variants in HMBS in CAGI6.
Authors: Zhang J. , Kinch L. , Katsonis P. , Lichtarge O. , Jagota M. , Song Y.S. , Sun Y. , Shen Y. , Kuru N. , Dereli O. , et al. .
Source: Human Genetics, 2024-08-07 00:00:00.0; , .
EPub date: 2024-08-07 00:00:00.0.
PMID: 39110250
Related Citations

Evaluating predictors of kinase activity of STK11 variants identified in primary human non-small cell lung cancers.
Authors: Chen Y. , Lee K. , Woo J. , Kim D.W. , Keum C. , Babbi G. , Casadio R. , Martelli P.L. , Savojardo C. , Manfredi M. , et al. .
Source: Research Square, 2024-07-02 00:00:00.0; , .
EPub date: 2024-07-02 00:00:00.0.
PMID: 39011112
Related Citations

Variant Impact Predictor database (VIPdb), version 2: Trends from 25 years of genetic variant impact predictors.
Authors: Lin Y.J. , Menon A.S. , Hu Z. , Brenner S.E. .
Source: Biorxiv : The Preprint Server For Biology, 2024-06-28 00:00:00.0; , .
EPub date: 2024-06-28 00:00:00.0.
PMID: 38979289
Related Citations

Critical assessment of missense variant effect predictors on disease-relevant variant data.
Authors: Rastogi R. , Chung R. , Li S. , Li C. , Lee K. , Woo J. , Kim D.W. , Keum C. , Babbi G. , Martelli P.L. , et al. .
Source: Biorxiv : The Preprint Server For Biology, 2024-06-08 00:00:00.0; , .
EPub date: 2024-06-08 00:00:00.0.
PMID: 38895200
Related Citations

Evaluation of enzyme activity predictions for variants of unknown significance in Arylsulfatase A.
Authors: Jain S. , Trinidad M. , Nguyen T.B. , Jones K. , Neto S.D. , Ge F. , Glagovsky A. , Jones C. , Moran G. , Wang B. , et al. .
Source: Biorxiv : The Preprint Server For Biology, 2024-05-19 00:00:00.0; , .
EPub date: 2024-05-19 00:00:00.0.
PMID: 38798479
Related Citations

Critical assessment of variant prioritization methods for rare disease diagnosis within the rare genomes project.
Authors: Stenton S.L. , O'Leary M.C. , Lemire G. , VanNoy G.E. , DiTroia S. , Ganesh V.S. , Groopman E. , O'Heir E. , Mangilog B. , Osei-Owusu I. , et al. .
Source: Human Genomics, 2024-04-29 00:00:00.0; 18(1), p. 44.
EPub date: 2024-04-29 00:00:00.0.
PMID: 38685113
Related Citations

Prioritizing genomic variants pathogenicity via DNA, RNA, and protein-level features based on extreme gradient boosting.
Authors: Ding M. , Chen K. , Yang Y. , Zhao H. .
Source: Human Genetics, 2024-04-04 00:00:00.0; , .
EPub date: 2024-04-04 00:00:00.0.
PMID: 38575818
Related Citations

An AI-based approach driven by genotypes and phenotypes to uplift the diagnostic yield of genetic diseases.
Authors: Zucca S. , Nicora G. , De Paoli F. , Carta M.G. , Bellazzi R. , Magni P. , Rizzo E. , Limongelli I. .
Source: Human Genetics, 2024-03-23 00:00:00.0; , .
EPub date: 2024-03-23 00:00:00.0.
PMID: 38520562
Related Citations

Explicable prioritization of genetic variants by integration of rule-based and machine learning algorithms for diagnosis of rare Mendelian disorders.
Authors: Kim H.H. , Kim D.W. , Woo J. , Lee K. .
Source: Human Genomics, 2024-03-21 00:00:00.0; 18(1), p. 28.
EPub date: 2024-03-21 00:00:00.0.
PMID: 38509596
Related Citations

CAGI, the Critical Assessment of Genome Interpretation, establishes progress and prospects for computational genetic variant interpretation methods.
Authors: Critical Assessment of Genome Interpretation Consortium .
Source: Genome Biology, 2024-02-22 00:00:00.0; 25(1), p. 53.
EPub date: 2024-02-22 00:00:00.0.
PMID: 38389099
Related Citations

Predicting the impact of rare variants on RNA splicing in CAGI6.
Authors: Lord J. , Oquendo C.J. , Wai H.A. , Douglas A.G.L. , Bunyan D.J. , Wang Y. , Hu Z. , Zeng Z. , Danis D. , Katsonis P. , et al. .
Source: Human Genetics, 2024-01-03 00:00:00.0; , .
EPub date: 2024-01-03 00:00:00.0.
PMID: 38170232
Related Citations

Impact of calmodulin missense variants associated with congenital arrhythmia on the thermal stability and the degree of unfolding.
Authors: Dal Cortivo G. , Marino V. , Zamboni D. , Dell'Orco D. .
Source: Human Genetics, 2023-12-28 00:00:00.0; , .
EPub date: 2023-12-28 00:00:00.0.
PMID: 38153589
Related Citations

Critical assessment of variant prioritization methods for rare disease diagnosis within the Rare Genomes Project.
Authors: Stenton S.L. , O'Leary M. , Lemire G. , VanNoy G.E. , DiTroia S. , Ganesh V.S. , Groopman E. , O'Heir E. , Mangilog B. , Osei-Owusu I. , et al. .
Source: Medrxiv : The Preprint Server For Health Sciences, 2023-08-04 00:00:00.0; , .
EPub date: 2023-08-04 00:00:00.0.
PMID: 37577678
Related Citations

CAGI6 ID-Challenge: Assessment of phenotype and variant predictions in 415 children with Neurodevelopmental Disorders (NDDs).
Authors: Aspromonte M.C. , Conte A.D. , Zhu S. , Tan W. , Shen Y. , Zhang Y. , Li Q. , Wang M.H. , Babbi G. , Bovo S. , et al. .
Source: Research Square, 2023-08-02 00:00:00.0; , .
EPub date: 2023-08-02 00:00:00.0.
PMID: 37577579
Related Citations

ClinVar and HGMD genomic variant classification accuracy has improved over time, as measured by implied disease burden.
Authors: Sharo A.G. , Zou Y. , Adhikari A.N. , Brenner S.E. .
Source: Genome Medicine, 2023-07-13 00:00:00.0; 15(1), p. 51.
EPub date: 2023-07-13 00:00:00.0.
PMID: 37443081
Related Citations

Calibration of computational tools for missense variant pathogenicity classification and ClinGen recommendations for PP3/BP4 criteria.
Authors: Pejaver V. , Byrne A.B. , Feng B.J. , Pagel K.A. , Mooney S.D. , Karchin R. , O'Donnell-Luria A. , Harrison S.M. , Tavtigian S.V. , Greenblatt M.S. , et al. .
Source: American Journal Of Human Genetics, 2022-12-01 00:00:00.0; 109(12), p. 2163-2177.
EPub date: 2022-11-21 00:00:00.0.
PMID: 36413997
Related Citations

Massively parallel reporter assays discover de novo exonic splicing mutants in paralogs of Autism genes.
Authors: Rhine C.L. , Neil C. , Wang J. , Maguire S. , Buerer L. , Salomon M. , Meremikwu I.C. , Kim J. , Strande N.T. , Fairbrother W.G. .
Source: Plos Genetics, 2022 01; 18(1), p. e1009884.
EPub date: 2022-01-20 00:00:00.0.
PMID: 35051175
Related Citations

LEAP: Using machine learning to support variant classification in a clinical setting.
Authors: Lai C. , Zimmer A.D. , O'Connor R. , Kim S. , Chan R. , van den Akker J. , Zhou A.Y. , Topper S. , Mishne G. .
Source: Human Mutation, 2020-03-16 00:00:00.0; , .
EPub date: 2020-03-16 00:00:00.0.
PMID: 32176384
Related Citations

Navigating ethical quandaries with the privacy dilemma of biomedical datasets.
Authors: Gürsoy G. , Doerr M. , Wilbanks J. , Wagner J.K. , Tang H. , Brenner S.E. .
Source: Pacific Symposium On Biocomputing. Pacific Symposium On Biocomputing, 2020; 25, p. 736-738.
PMID: 31797643
Related Citations

Matching whole genomes to rare genetic disorders: Identification of potential causative variants using phenotype-weighted knowledge in the CAGI SickKids5 clinical genomes challenge.
Authors: Pal L.R. , Kundu K. , Yin Y. , Moult J. .
Source: Human Mutation, 2019-11-03 00:00:00.0; , .
EPub date: 2019-11-03 00:00:00.0.
PMID: 31680375
Related Citations

CAGI 5 splicing challenge: Improved exon skipping and intron retention predictions with MMSplice.
Authors: Cheng J. , Çelik M.H. , Nguyen T.Y.D. , Avsec Ž. , Gagneur J. .
Source: Human Mutation, 2019 09; 40(9), p. 1243-1251.
EPub date: 2019-07-29 00:00:00.0.
PMID: 31070280
Related Citations

Assessment of predicted enzymatic activity of α-N-acetylglucosaminidase variants of unknown significance for CAGI 2016.
Authors: Clark W.T. , Kasak L. , Bakolitsa C. , Hu Z. , Andreoletti G. , Babbi G. , Bromberg Y. , Casadio R. , Dunbrack R. , Folkman L. , et al. .
Source: Human Mutation, 2019 Sep; 40(9), p. 1519-1529.
PMID: 31342580
Related Citations

Reports from the fifth edition of CAGI: The Critical Assessment of Genome Interpretation.
Authors: Andreoletti G. , Pal L.R. , Moult J. , Brenner S.E. .
Source: Human Mutation, 2019-07-23 00:00:00.0; , .
EPub date: 2019-07-23 00:00:00.0.
PMID: 31334884
Related Citations

CAGI SickKids challenges: Assessment of phenotype and variant predictions derived from clinical and genomic data of children with undiagnosed diseases.
Authors: Kasak L. , Hunter J.M. , Udani R. , Bakolitsa C. , Hu Z. , Adhikari A.N. , Babbi G. , Casadio R. , Gough J. , Guerrero R.F. , et al. .
Source: Human Mutation, 2019-07-19 00:00:00.0; , .
EPub date: 2019-07-19 00:00:00.0.
PMID: 31322791
Related Citations

CAGI5: Objective performance assessments of predictions based on the evolutionary action equation.
Authors: Katsonis P. , Lichtarge O. .
Source: Human Mutation, 2019-07-18 00:00:00.0; , .
EPub date: 2019-07-18 00:00:00.0.
PMID: 31317604
Related Citations

Assessing predictions of the impact of variants on splicing in CAGI5.
Authors: Mount S.M. , Avsec Ž. , Carmel L. , Casadio R. , Çelik M.H. , Chen K. , Cheng J. , Cohen N.E. , Fairbrother W.G. , Fenesh T. , et al. .
Source: Human Mutation, 2019-07-13 00:00:00.0; , .
EPub date: 2019-07-13 00:00:00.0.
PMID: 31301154
Related Citations

Assessing computational predictions of the phenotypic effect of cystathionine-beta-synthase variants.
Authors: Kasak L. , Bakolitsa C. , Hu Z. , Yu C. , Rine J. , Dimster-Denk D.F. , Pandey G. , De Baets G. , Bromberg Y. , Cao C. , et al. .
Source: Human Mutation, 2019-07-13 00:00:00.0; , .
EPub date: 2019-07-13 00:00:00.0.
PMID: 31301157
Related Citations

Future directions for high-throughput splicing assays in precision medicine.
Authors: Rhine C.L. , Neil C. , Glidden D.T. , Cygan K.J. , Fredericks A.M. , Wang J. , Walton N.A. , Fairbrother W.G. .
Source: Human Mutation, 2019-07-12 00:00:00.0; , .
EPub date: 2019-07-12 00:00:00.0.
PMID: 31297895
Related Citations

Assessment of blind predictions of the clinical significance of BRCA1 and BRCA2 variants.
Authors: Cline M.S. , Babbi G. , Bonache S. , Cao Y. , Casadio R. , de la Cruz X. , Díez O. , Gutiérrez-Enríquez S. , Katsonis P. , Lai C. , et al. .
Source: Human Mutation, 2019-07-11 00:00:00.0; , .
EPub date: 2019-07-11 00:00:00.0.
PMID: 31294896
Related Citations

VIPdb, a genetic Variant Impact Predictor Database.
Authors: Hu Z. , Yu C. , Furutsuki M. , Andreoletti G. , Ly M. , Hoskins R. , Adhikari A.N. , Brenner S.E. .
Source: Human Mutation, 2019-07-08 00:00:00.0; , .
EPub date: 2019-07-08 00:00:00.0.
PMID: 31283070
Related Citations

Assessing predictions on fitness effects of missense variants in calmodulin.
Authors: Zhang J. , Kinch L.N. , Cong Q. , Katsonis P. , Lichtarge O. , Savojardo C. , Babbi G. , Martelli P.L. , Capriotti E. , Casadio R. , et al. .
Source: Human Mutation, 2019-07-08 00:00:00.0; , .
EPub date: 2019-07-08 00:00:00.0.
PMID: 31283071
Related Citations

What went wrong with variant effect predictor performance for the PCM1 challenge.
Authors: Miller M. , Wang Y. , Bromberg Y. .
Source: Human Mutation, 2019-07-03 00:00:00.0; , .
EPub date: 2019-07-03 00:00:00.0.
PMID: 31268618
Related Citations

Performance of computational methods for the evaluation of pericentriolar material 1 missense variants in CAGI-5.
Authors: Monzon A.M. , Carraro M. , Chiricosta L. , Reggiani F. , Han J. , Ozturk K. , Wang Y. , Miller M. , Bromberg Y. , Capriotti E. , et al. .
Source: Human Mutation, 2019-07-01 00:00:00.0; , .
EPub date: 2019-07-01 00:00:00.0.
PMID: 31260570
Related Citations

Predicting changes in protein stability caused by mutation using sequence- and structure-based methods in a CAGI5 blind challenge.
Authors: Strokach A. , Corbi-Verge C. , Kim P.M. .
Source: Human Mutation, 2019-06-27 00:00:00.0; , .
EPub date: 2019-06-27 00:00:00.0.
PMID: 31243847
Related Citations

Assessing the performance of in-silico methods for predicting the pathogenicity of variants in the gene CHEK2, among Hispanic females with breast cancer.
Authors: Voskanian A. , Katsonis P. , Lichtarge O. , Pejaver V. , Radivojac P. , Mooney S.D. , Capriotti E. , Bromberg Y. , Wang Y. , Miller M. , et al. .
Source: Human Mutation, 2019-06-26 00:00:00.0; , .
EPub date: 2019-06-26 00:00:00.0.
PMID: 31241222
Related Citations

Gene-specific features enhance interpretation of mutational impact on acid alpha-glucosidase enzyme activity.
Authors: Adhikari A.N. .
Source: Human Mutation, 2019-06-22 00:00:00.0; , .
EPub date: 2019-06-22 00:00:00.0.
PMID: 31228295
Related Citations

Predicting functional variants in enhancer and promoter elements using RegulomeDB.
Authors: Dong S. , Boyle A.P. .
Source: Human Mutation, 2019-06-22 00:00:00.0; , .
EPub date: 2019-06-22 00:00:00.0.
PMID: 31228310
Related Citations

Evaluating the predictions of the protein stability change upon single amino acid substitutions for the FXN CAGI5 challenge.
Authors: Savojardo C. , Petrosino M. , Babbi G. , Bovo S. , Corbi-Verge C. , Casadio R. , Fariselli P. , Folkman L. , Garg A. , Karimi M. , et al. .
Source: Human Mutation, 2019-06-17 00:00:00.0; , .
EPub date: 2019-06-17 00:00:00.0.
PMID: 31209948
Related Citations

Characterization of intellectual disability and autism comorbidity through gene panel sequencing.
Authors: Aspromonte M.C. , Bellini M. , Gasparini A. , Carraro M. , Bettella E. , Polli R. , Cesca F. , Bigoni S. , Boni S. , Carlet O. , et al. .
Source: Human Mutation, 2019-06-17 00:00:00.0; , .
EPub date: 2019-06-17 00:00:00.0.
PMID: 31209962
Related Citations

Assessment of methods for predicting the effects of PTEN and TPMT protein variants.
Authors: Pejaver V. , Babbi G. , Casadio R. , Folkman L. , Katsonis P. , Kundu K. , Lichtarge O. , Martelli P.L. , Miller M. , Moult J. , et al. .
Source: Human Mutation, 2019-06-11 00:00:00.0; , .
EPub date: 2019-06-11 00:00:00.0.
PMID: 31184403
Related Citations