Welcome

 

Welcome to Nancy Cook’s page in the Division of Preventive Medicine! This page focuses on reclassification and other methods of assessing and comparing risk prediction models, and provides software in SAS and links to R programs.

 

This work and the development of programs listed here were supported by grants HL113080 from the NHLBI, BAA award HHSN268200960011C from the NHLBI, and from the Donald W. Reynolds Foundation. The Women’s Health Study (WHS) cohort is supported by grants HL43851 and CA47988 from the National Heart, Lung, and Blood Institute and the National Cancer Institute.

 

Overview of Reclassification Statistics

Risk prediction equations are used in a variety of fields for risk stratification and to determine cost-effective and appropriate courses of treatment. The Framingham risk score for cardiovascular disease (CVD) has been widely used for establishing guidelines for treatment with cholesterol-lowering medications. Whether new risk predictors can add clinical utility is an important question in cardiology as well as in other fields of research.

 

Traditionally, risk prediction models have been evaluated using the area under the receiver operating characteristic (ROC) curve, but this can be an insensitive measure for model comparison, and has little direct clinical relevance. Since its description in 2006, much interest has been generated in reclassification, which assesses the ability of new models to more accurately classify individuals into higher or lower risk strata. This has led to new methods of evaluating and comparing risk prediction models, including the reclassification calibration test and the net reclassification index (NRI). Extensions of the NRI to survival settings have now been introduced, as well as a continuous version that eliminates the need for categories. Debate about these methods continues to evolve, and research on their interpretation and use remains ongoing.

 

Chronology

We first described using reclassification tables to compare models in an applied paper looking at the influence of CRP in predicting risk in women in the WHS:

 

Cook NR, Buring JE, Ridker PM. The effect of including C-reactive protein in cardiovascular risk prediction models for women. Ann Intern Med 2006; 145:21-29.

 

The use of reclassification as well as a discussion of receiver operating characteristic (ROC) curves was further described here:

 

Cook NR. Use and misuse of the ROC curve in risk prediction. Circ 2007; 115:928-35.

 

Pencina at al developed the net reclassification improvement (NRI) and the integrated discrimination improvement here:

 

Pencina MJ, D’Agostino RB Sr, D’Agostino RB Jr, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27(2):157–72.

 

And we described these methods and their performance in the WHS data:

 

Cook NR, Ridker PM. Advances in measuring the effect of individual predictors of cardiovascular risk: the role of reclassification measures. Ann Intern Med 2009; 150:795-802.

 

We described the statistical performance of the new metrics in simulations in:

 

Cook NR, Paynter NP. Performance of reclassification statistics in comparing risk prediction models. Biom J 2011; 53(2):237-58.

 

We have also published on the use of the continuous version of the NRI, on the NRI for clinical subgroups, and on measures of fit using survival data. Please see the publications list for more references.