SAS Macros

 

These macros are provided for general use as is. Although they have been debugged and validated, they are provided with no guarantee of performance in other data.

 

Binary Outcomes

All macros assume that predicted probabilities have been saved for each model of interest, such as through logistic regression or some other method. Note that the predicted probabilities can be from any type of model and do not need to be nested. The bootstrap program that adjusts for optimism is the only exception since it compares nested models.

 

The following inputs are required in all macros:

DSNAME = data set name

DETAIL = 2 for detailed output, 1 for limited output, 0 for no output report (such as for simulations or bootstrapping)

STATVAR or OUTC = binary outcome variable

PROB = predicted probability

Macros that compare two models replace the latter with

PROB1 = predicted probability for Model 1

PROB2 = predicted probability for Model 2

 

Hosmer-Lemeshow goodness of fit test

hlcalibmacs.v1.sas – Includes macros:

%calibhl – For Hosmer-Lemeshow goodness-of-fit test across deciles of risk

%calhlcat – For Hosmer-Lemeshow goodness-of-fit test across user-specified categories of risk

 

C-statistics or area under the ROC curve

cstatmacs.v1.sas – Includes macros:

%cstat – C-statistic from Proc Freq

%ctest – Tests difference in c-statistics (based on Rosner & Glynn, Biometrics 2009)

%corrs – Computes SE of difference in c-statistics, used within %ctest

%delong – Computes DeLong test of difference in c’s (Biometrics 1988) using Proc Logistic

(Note that the tests of difference in c’s should not be used to test nested models in the model development data – see Demler, StatMed 2012)

 

Reclassification Statistics

reclass.macros.v4.sas – Includes macros:

%reclass – for reclassification calibration test

%nricat – for categorical NRI

%idimacro – for IDI

%contnri – for continuous NRI

 

Conditional NRI for subcategories of intermediate risk

(Includes correction for null – see Paynter & Cook, MDM 2013, and Paynter & Cook, BMJ 2016)

cnrimacro.v1.sas – Includes macro:

%cnricat

 

Weighted Reclassification Statistics

reclass.wtmacs.v2.sas – Includes macros:

%reclasswt – for weighted reclassification calibration test

%nriwt – for weighted categorical NRI

%idiwt – for weighted IDI

%contnriwt – for weighted continuous NRI

 

Bootstrap programs to compute standard error

bootitmacro.v2.sas – Includes macro:

%bootit – to compute SE and CI for the reclassification statistics using bootstraps (calls macros in reclass.macros.v4.sas)

 

Bootstrap programs to adjust for optimism

bootoptmacro.v3.sas – Includes macro:

%bootopt – to adjust for optimism in estimates of fit using bootstraps when adding new variables to a base model (calls macros in reclass.macros.v4.sas). Uses Cox model to get estimate at time T and compares to binary outcome at time T.

 

bootbinopt.v2.sas  – Includes macro:

%bootbinopt – to adjust for optimism in estimates of fit using bootstraps when adding new variables to a base model (calls macros in reclass.macros.v4.sas) using logistic regression for binary classification.

 

 

Survival Outcomes

All macros assume that predicted probabilities at time T have been saved for each model of interest, such as through Cox regression or some other method. For program code to do this in SAS, see the program savesurv_T.sas below.  Note that the predicted probabilities can be from any type of model and do not need to be nested. Observed probabilities are based on nonparametric estimates. The bootstrap program that adjusts for optimism is the only exception since it compares nested models.

 

The following inputs are required in all macros:

DSNAME = data set name

DETAIL = 2 for detailed output, 1 for limited output, 0 for no output report (such as for simulations or bootstrapping)

STATVAR or OUTC = outcome variable (1=event, 0=censored)

PYRS = person-time to event or censoring

T = Time of interest for prediction

PROB = predicted probability at time T

Macros that compare two models replace the latter with

PROB1 = predicted probability at time T for Model 1

PROB2 = predicted probability at time T for Model 2

 

D’Agostino-Nam test of goodness-of-fit

From D’Agostino and Nam, Handbook of Statistics, 2004. Note that this should only be used in low-censoring situations.

surv_calibmacs.v3.sas – Includes macros:

%surv_calibhl – for decile categories

%surv_calibr2 – for user-specified categories of risk

 

Greenwood-D’Agostino-Nam test of calibration

From Demler, Paynter, Cook, Statistics in Medicine, 2015. This test is appropriate with censoring.

GND_test.v1.sas – Includes macro:

%gnd_calib – for user-specified categories of risk

 

C-statistics for survival data

Based on Pencina et al, StatMed 2004. Note that these can take a long time to run.

survcmacs.v3.sas – Includes macros:

%predc – computes Noether and wide confidence intervals

%predc_simp – computes wide CIs only (eg., useful for bootstrapping)

%predcwt – computes Noether and wide CIs, including weights

%predcwt_simp – computes wide CIs only with weights

%predc_big – computes wide CIs only; use for large datasets (subsets for efficiency)

%predcwt_big – computes wide CIs only with weights; use for large datasets

 

Reclassification Statistics for survival data

survmacs.v4.sas – Includes macros:

%surv_reclass – for reclassification calibration test (for low-censoring situations)

%surv_nri – for categorical NRI

%surv_idi – for IDI

%surv_contnricat – for continuous NRI

 

Weighted Reclassification Statistics for survival data

survwtmacs.v3.sas – Includes macros:

%surv_reclasswt – for reclassification calibration test

%surv_nriwt – for categorical NRI

%surv_idiwt – for IDI

%surv_contnriwt – for continuous NRI

 

Bootstrap programs to compute standard errors in survival data

(Note: Bootstrap programs are not available for weighted samples, since the sampling can depend on the weighting structure.)

bootitsmacro.v3.sas – Includes macros for reclassification statistics:

%bootits – to compute SE and CI for the survival reclassification statistics using bootstraps (calls macros in survmacs.sas)

 

bootscmacro.v3.sas – Includes macro for c-index for survival data:

%bootitsc – to compute SE and CI for the survival c-index using bootstraps (calls macros in survcmacs.sas). Note: takes a long time to run.

 

bootsurvcmac.v4.sas – Includes macros for both reclassification statistics and c-index:

%bootalls – to compute SE and CI for the survival reclassification statistics and c-index using bootstraps (calls macros in survmacs.sas and survcmacs.sas). Note: takes a long time to run.

 

Bootstrap programs to adjust for optimism for survival data

bootsurvoptmac.v4.sas – Includes macro:

%bootsurvopt – to adjust for optimism in estimates of fit using bootstraps when adding new variables to a base model for survival data (calls macros in survmacs.sas).

 

bootsurvcopt.v3.sas – Includes macro:

%bootsurvcopt – to adjust for optimism in c-index (only) using bootstraps when adding new variables to a base model for survival data (calls macros in survcmacs.sas).

 

 

Other programs

savesurv_T.sas – Code to save predicted probabilities at time T from a Cox model

risk_score_code.sas – Includes code to compute the AHA/ACC, ATP III, and Reynolds risk scores