Development and validation of models to predict the risk of major cardiovascular events and death for people with kidney failure having non-cardiac surgery
Loading...
Date
2024-08-27
Authors
Pabla, Gurpreet
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Abstract
Introduction: Patients with kidney failure undergoing non-cardiac surgery have a significantly higher risk of adverse cardiovascular events and mortality than the average population. Existing risk prediction tools are not valid for patients with kidney failure. Harrison et al. developed three risk prediction models to predict the risk of major post-operative events in individuals with kidney failure. We externally validated the Alberta models and developed and validated a ML model for MACE and mortality in kidney failure patients within 30 days of undergoing outpatient or inpatient non-cardiac surgery in Alberta and Manitoba, Canada.
Methods: Data was sourced from Manitoba Health, including adults (≥ 18 years) with kidney failure (eGFR < 15 mL/min/1.73m2 or on maintenance dialysis) undergoing non-cardiac surgery, 2007-2019. The primary outcome was a composite of acute myocardial infarction, cardiac arrest, ventricular arrhythmia, and all-cause mortality. The performance of the models was evaluated through AUC-ROC, AUC-PR, calibration, and other metrics. We externally validated Alberta models using two approaches: 1) Model deployment: Used coefficients from the Alberta models to predict outcomes on Manitoba data. 2) Model refitting: Re-estimated model coefficients using logistic regression on Manitoba data while maintaining the same variables as the AKDN models.
To develop a machine learning model, data was split into 70% (training), 15% (validation), and 15% (testing). The training set was used to tune the hyperparameters and train the models; the validation dataset for feature selection and evaluate model performance, while the testing set evaluated the model’s final performance. We used XGBoost and Random Forest, selecting a model with reasonable and balanced AUC-ROC and AUC-PR. The final model was externally tested using Alberta data.
Results: We identified 12,082 surgeries and 569 outcomes (5%). Model deployment performed well, with AUC-ROC ranging from 0.82 (model 1) to 0.87 (model 3) and good calibration. Once refit, discrimination remained strong with C-statistics ranging from 0.83 (model 1) to 0.86 (model 3) and calibration slope of 1. The XGBoost model (8 features) showed an AUC-ROC of 0.861 and AUC-PR of 0.304, and the random forest model (20 features) estimated an AUC-ROC of 0.863 and AUC-PR of 0.332 in the Manitoba cohort. External testing in Alberta showed similar performance. Calibration plots demonstrated good calibration.
Conclusion: Our study confirms the Alberta models' robustness in a geographically distinct Canadian population. Machine learning models demonstrated good performance, with improved parsimony compared to existing tools. Future work should compare these tools and test the impact of risk-guided approaches to perioperative care.
Description
Keywords
kidney failure, machine learning