The CHARGE-AF model was a poor predictor of AF risk when calibrated with electronic medical records, according to a data published in JAMA Cardiology.
“Although the CHARGE-AF models had satisfactory discrimination in our cohort, calibration was poor,” Matthew J. Kolek,MD, MSci, and colleagues wrote. “One potential reason for the poor calibration is important differences in the characteristics of CHARGE-AF discovery cohorts and our cohort.”
The 33,494 patient prediction model study deidentified EMRs of those aged 40 years or older who had no history of AF. A follow up took place in the internal medicine outpatient clinics at Vanderbilt University Medical Center for incident AD between December 2005 and December 2010.
The study was adjusted for differences in baseline hazard and the CHARGE_AF Cox proportional hazards model regression coefficients were applied to the EMR cohort. A simplified version of the risk prediction model without echocardiographic variables was evaluated and the data were analyzed from October 2013 to January 2014.
The median age of the 33,494 patients was 57 years with an interquartile range of 49-67.
Women made up 57% of the patient population, while men made up the remaining 43%. 85.7% of the patients included in the study were while and 14.3% were African American.
A follow-up of 4.9 years showed a development of AF in 2,455 patients (7.3%).
Both models had difficulty applying risk models to an EMR cohort and often under predicted the risk of AF among low-risk individuals, while AF among high-risk individuals was over predicted (10th and 90th percentiles for predicted probability of incident AF, 0.005 and 0.179, respectively).
The C index in the full CHARGE-AF model was 0.708 (95% CI, 0.699-0.718) in our cohort.
There was similar discrimination in the simplified model (C index, 0.709; 95% CI, 0.699-0.718; P =.70 for differences between models).
There were several important limitations to the study including inaccuracies in data collection and ascertainment, indication bias and influence by loss of patient followup,
“The emergence of EHRs greatly increases the interest in and potential applications of predictive models in health systems,” Benjamin A. Goldstein, PhD, of the Duke University School of Medicine and colleagues wrote in a related editorial. “However, as the work of Kokel et al illustrates, before models can be recommended for health system-wide applications, validation work needs to be conducted to better understand the strengths and weaknesses of the selected model in the context of the intended clinical use.” – by Dave Quaile
Disclosure: Kolek and Goldstein reports no relevant financial disclosures.