Skip to main content
An official website of the United States government
Grant Details

Grant Number: 5R21CA185704-02 Interpret this number
Primary Investigator: Uno, Hajime
Organization: Dana-Farber Cancer Inst
Project Title: Statistical Analysis Methods for Validating Risk Prediction Models
Fiscal Year: 2015


Abstract

DESCRIPTION (provided by applicant): Risk prediction models play an important role in selecting prevention and treatment strategies for many cancers. While extensive effort in both clinical and methodological research is spent on building accurate and reliable models, the science and practice of validating these models is less well-developed. We have identified a potentially critical pitfall in the standard analytic approach to validating a prediction model. Whle it is common to observe poorer performance in a validation set compared to a training set, this difference is generally attributed to optimistic bias in measuring performance in the training set. However, the difference might be rather due to differences in distribution of the predictors in the validation set, which can strongly affect predictive performance. Currently, no analytic tools exis to address this possibility, which leaves investigators unable to reliably interpret results of validation studies. For example, if a validation study includes a higher proportion of cases for which prognostication is more difficult because of clinical characteristics, a standard validation analysis might erroneously give a low rating to a useful risk prediction model. Ultimately, because validation studies determine which prediction models are adopted for research and clinical use, it is critical that their methods be grounded in rigorous cross-study comparisons. Development of methods to enhance validation of prediction models is thus both timely and important. We propose to develop a practical and effective statistical method that will enable investigators to systematically adjust for differences in distribution of predictors among multiple datasets or multiple target populations in validation studies (Aim 1). This proposed approach and resulting estimates will provide a better understanding of how risk prediction models perform in specific target populations, and will make interpretation of results of cross-study validation with multiple validation sets much more reliable. We will also develop software to implement our methods on three very commonly used statistical platforms (R, Stata, and SAS), making immediate public availability possible (Aim 2). Our proposal is relevant to the mission of the NCI, because the methods we will develop are innovative, and have broad applicability to developers of cancer-related prediction models. Because of the emergence of large amount of observational data from electronic medical records (EMR), there is an increased opportunity to harness this information to develop prediction tools for use at the bedside. Moreover, clinical decision support tools are increasingly being built into EMRs and so implementation of prediction models at the point of care has become more common; it stands to reason that the opportunity to exploit the EMR for this purpose will only be successful if the models that inform i are statistically sound.



Publications

Detecting Lung and Colorectal Cancer Recurrence Using Structured Clinical/Administrative Data to Enable Outcomes Research and Population Health Management.
Authors: Hassett M.J. , Uno H. , Cronin A.M. , Carroll N.M. , Hornbrook M.C. , Ritzwoller D. .
Source: Medical Care, 2017 12; 55(12), p. e88-e98.
PMID: 29135771
Related Citations

Efficiency of two sample tests via the restricted mean survival time for analyzing event time observations.
Authors: Tian L. , Fu H. , Ruberg S.J. , Uno H. , Wei L.J. .
Source: Biometrics, 2017-09-12 00:00:00.0; , .
EPub date: 2017-09-12 00:00:00.0.
PMID: 28901017
Related Citations

Survival after recurrence of stage I-III breast, colorectal, or lung cancer.
Authors: Hassett M.J. , Uno H. , Cronin A.M. , Carroll N.M. , Hornbrook M.C. , Fishman P. , Ritzwoller D.P. .
Source: Cancer Epidemiology, 2017-07-12 00:00:00.0; 49, p. 186-194.
EPub date: 2017-07-12 00:00:00.0.
PMID: 28710943
Related Citations

Development and Validation of the PREMM5 Model for Comprehensive Risk Assessment of Lynch Syndrome.
Authors: Kastrinos F. , Uno H. , Ukaegbu C. , Alvero C. , McFarland A. , Yurgelun M.B. , Kulke M.H. , Schrag D. , Meyerhardt J.A. , Fuchs C.S. , et al. .
Source: Journal Of Clinical Oncology : Official Journal Of The American Society Of Clinical Oncology, 2017-07-01 00:00:00.0; 35(19), p. 2165-2172.
EPub date: 2017-05-10 00:00:00.0.
PMID: 28489507
Related Citations

Cancer Susceptibility Gene Mutations in Individuals With Colorectal Cancer.
Authors: Yurgelun M.B. , Kulke M.H. , Fuchs C.S. , Allen B.A. , Uno H. , Hornick J.L. , Ukaegbu C.I. , Brais L.K. , McNamara P.G. , Mayer R.J. , et al. .
Source: Journal Of Clinical Oncology : Official Journal Of The American Society Of Clinical Oncology, 2017-04-01 00:00:00.0; 35(10), p. 1086-1095.
EPub date: 2017-01-30 00:00:00.0.
PMID: 28135145
Related Citations

Alternatives to Hazard Ratios for Comparing the Efficacy or Safety of Therapies in Noninferiority Studies.
Authors: Uno H. , Wittes J. , Fu H. , Solomon S.D. , Claggett B. , Tian L. , Cai T. , Pfeffer M.A. , Evans S.R. , Wei L.J. .
Source: Annals Of Internal Medicine, 2015-07-21 00:00:00.0; 163(2), p. 127-34.
PMID: 26054047
Related Citations

Financial hardship and the intensity of medical care received near death.
Authors: Tucker-Seeley R.D. , Abel G.A. , Uno H. , Prigerson H. .
Source: Psycho-oncology, 2015 May; 24(5), p. 572-8.
PMID: 25052138
Related Citations

Derivation and validation of the SEER-Medicare myelodysplastic syndromes risk score (SMMRS).
Authors: Uno H. , Cronin A.M. , Wadleigh M. , Schrag D. , Abel G.A. .
Source: Leukemia Research, 2014 Dec; 38(12), p. 1420-4.
PMID: 25443886
Related Citations

Acute hospital care is the chief driver of regional spending variation in Medicare patients with advanced cancer.
Authors: Brooks G.A. , Li L. , Uno H. , Hassett M.J. , Landon B.E. , Schrag D. .
Source: Health Affairs (project Hope), 2014 Oct; 33(10), p. 1793-800.
PMID: 25288424
Related Citations




Back to Top