Table 2 Accuracy of AI-based Risk Prediction Models
From: Application and risk prediction of thrombolytic therapy in cardio-cerebrovascular diseases: a review
First author, year | Risk score/ prediction model/ predictors | Method | Dataset | Observation time endpoint | Results |
|---|---|---|---|---|---|
Source size | |||||
Aziz F, 2021 [79] | New prediction models (Important variables: age, race, smoking status, hypertension, DM, family history of premature CVD, CRD, HR, BP, Killip class, blood glucose, administrations including cardiac catheterization, PCI, ASA, beta-blocker, ACEI, statins, diuretics, oral hypoglycemic agent and insulin) | RF | NCVD registry 12,368 | 1 year All-cause mortality: 1) In-hospital 2) at 30-day 3) at 1-year | 1) AUC = 0.86; Se = 34.7%; Sp = 96.8%; Acc = 93.5% 2) AUC = 0.83; Se = 37.3%; Sp = 94.9%; Acc = 90.3% 3) AUC = 0.78; Se = 37.3%; Sp = 94.9%; Acc = 82.7% |
RFvarImp-SBE-RF | 1) AUC = 0.87; Se = 33.7%; Sp = 97.7%; Acc = 94.2% 2) AUC = 0.85; Se = 41.3%; Sp = 93.0%; Acc = 88.9% 3) AUC = 0.80; Se = 46.0%; Sp = 90.1%; Acc = 83.8% | ||||
RFE-RF | 1) AUC = 0.86; Se = 34.7%; Sp = 96.4%; Acc = 93.1% 2) AUC = 0.82; Se = 38.7%; Sp = 95.2%; Acc = 90.7% 3) AUC = 0.79; Se = 49.2%; Sp = 88.9%; Acc = 83.2% | ||||
SVM | 1) AUC = 0.86; Se = 61.4%; Sp = 89.2%; Acc = 87.7% 2) AUC = 0.87; Se = 73.3%; Sp = 81.7%; Acc = 81.0% 3) AUC = 0.84; Se = 74.6%; Sp = 79.8%; Acc = 79.1% | ||||
SVMvarImp-SBE-SVM | 1) AUC = 0.88; Se = 69.3%; Sp = 86.1%; Acc = 85.2% 2) AUC = 0.90; Se = 84.0%; Sp = 79.0%; Acc = 79.4% 3) AUC = 0.84; Se = 75.4%; Sp = 77.3%; Acc = 77.1% | ||||
RFE-SVM | 1) AUC = 0.85; Se = 72.3%; Sp = 83.8%; Acc = 83.2% 2) AUC = 0.88; Se = 81.3%; Sp = 80.0%; Acc = 80.1% 3) AUC = 0.84; Se = 76.2%; Sp = 79.8%; Acc = 79.3% | ||||
LR | 1) AUC = 0.88; Se = 71.3%; Sp = 85.0%; Acc = 84.2% 2) AUC = 0.85; Se = 74.7%; Sp = 80.3%; Acc = 79.9% 3) AUC = 0.76; Se = 61.1%; Sp = 79.2%; Acc = 76.6% | ||||
LRstepwise-SBE-LR | 1) AUC = 0.89; Se = 75.2%; Sp = 84.1%; Acc = 83.6% 2) AUC = 0.85; Se = 74.7%; Sp = 83.4%; Acc = 82.7% 3) AUC = 0.80; Se = 65.9%; Sp = 81.4%; Acc = 79.2% | ||||
RFE-LR | 1) AUC = 0.87; Se = 66.3%; Sp = 83.4%; Acc = 82.5% 2) AUC = 0.83; Se = 72.0%; Sp = 81.0%; Acc = 80.3% 3) AUC = 0.78; Se = 61.9%; Sp = 80.2%; Acc = 77.6% | ||||
TIMI | - | 1) AUC = 0.81; Se = 64.4%; Sp = 83.4%; Acc = 82.4% 2) AUC = 0.80; Se = 62.7%; Sp = 83.2%; Acc = 81.6% 3) AUC = 0.76; Se = 48.4%; Sp = 83.7%; Acc = 78.6% | |||
Xu Y, 2022 [80] | New prediction models (Important variables: triglycerides, Lpa, baseline NIHSS score, hemoglobin, BP, INR, WBC, etc.) | RF | Primary data 345 | - Hemorrhagic transformation | AUC = 0.795; Se = 66.7%; Sp = 80.7% |
LR | AUC = 0.703; Se = 60.0%; Sp = 78.0% | ||||
SITS-ICH | - | AUC = 0.660 | |||
MSS | - | AUC = 0.657 | |||
SEDAN | - | AUC = 0.655 | |||
Meng Y, 2022 [81] | New clinical model | RF | Primary data 71 | - Hemorrhagic transformation | AUC = 0.556; Se = 33.3%; Sp = 55.6%; Acc = 54.5%; F1 score = 0.067 |
New radiomics models | Abnormal ROIs: AUC = 0.831; Se = 60.0%; Sp = 88.2%; Acc = 81.8%; F1 score = 0.600 All ROIs: AUC = 0.871; Se = 73.3%; Sp = 88.2%; Acc = 84.8%; F1 score = 0.687 | ||||
Combined model | AUC = 0.911; Se = 81.0%; Sp = 93.3%; Acc = 89.4%; F1 score = 0.830 | ||||
Weng ZA, 2022 [82] | New nomogram (Smoking, NIHSS, BUN/Cr, and NLR) | LR | Primary data Training group: 387 Testing group: 166 | During hospitalization Any ICH | Training group: AUC = 0.887; calibration plots mean absolute error = 0.025; DCA threshold probabilities = 2.5–57.8% Testing group: AUC = 0.776; calibration plots mean absolute error = 0.036; DCA threshold probabilities = 5.4–38.2% |
MSS | - | Training group: AUC = 0.723 Testing group: AUC = 0.647 | |||
GRASPS | - | Training group: AUC = 0.738 Testing group: AUC = 0.671 | |||
SPAN-100 | - | Training group: AUC = 0.538 Testing group: AUC = 0.552 | |||
Zhang K, 2022 [83] | New nomogram (DM, AF, total cholesterol, fibrous protein, cerebral infarction area, NIHSS score, onset-to-treatment) | LR | Primary data Training group: 392 Testing group: 178 | During hospitalization SICH: ECASS II | Training group: AUC = 0.831 Testing group: AUC = 0.889; Sp = 82.3%; Se = 83.3% Hosmer–Lemeshow test calibration = 7.466 |
Zhang KJ, 2021 [84] | NT-proBNP | - | Primary data 404 | 3 months 1) Hemorrhagic transformation 2) 3-month mortality | 1) p < 0.001; OR = 1.341 2) p < 0.001; OR = 1.788 |
New nomogram (NT-proBNP, NIHSS scores and baseline glucose levels) | LR | Primary data Training group: 283 Testing group: 121 | 3 months Poor 3-month functional outcomes | Training group: AUC = 0.710; Se = 72.3%; Sp = 60.8% Testing group: AUC = 0.764; Se = 80.3%; Sp = 64.9% | |
Guo H, 2021 [85] | New nomogram (AF, baseline glucose level, NLR, and baseline NIHSS) | LR | Primary data 1200 | During hospitalization SICH: ECASS II | AUC = 0.788 |
Soni M, 2021 [86] | New risk score (Age > 75 years, BP, severity of stroke, pre-treatment antithrombotic and history of hypertension and hyperlipidemia) | K-means clustering | Primary data 890 | During hospitalization SICH: ECASS II | AUC = 0.75 (continuous risk scoring), 0.71 (risk stratification) |
Liu J, 2021 [87] | New radiomic model | LASSO regression | Primary data 109 | 90 days 1) Hemorrhage expansion (a 33% increase in the hematoma volume) 2) 3-month mortality 3) 3-month mortality/ disability (mRS score 3–6) | 1) AUC = 0.85; Se = 82%; Sp = 77%; Acc = 78% 2) AUC = 0.67; OR = 5.17 3) AUC = 0.63; OR = 4.70 |
After SMOTE data Training group: 96 Testing group: 23 | Training group: 1) AUC = 0.91; Se = 83%; Sp = 89%; Acc = 87% Testing group: 1) AUC = 0.87; Se = 60%; Sp = 85%; Acc = 74% | ||||
Chen Z, 2021 [88] | New radiomic models | LR | Primary data 40 | - 1) No recanalization 2) Full recanalization | 1) AUC = 0.751; MCC = 0.730 2) AUC = 0.797; MCC = 0.762 |
ALEM | 1) AUC = 0.804; MCC = 765 2) AUC = 0.866; MCC = 0.841 | ||||
CNN | 1) AUC = 0.781; MCC = 0.758 2) AUC = 0.814; MCC = 792 | ||||
U-NL-NET | 1) AUC = 0.844; MCC = 0.827 2) AUC = 0.875; MCC = 0.853 | ||||
AUNet | 1) AUC = 0.875; MCC = 0.851 2) AUC = 0.898; MCC = 0.863 | ||||
Wang F, 2020 [89] | New prediction models (Important variables: age, AF, glucose level, NIHSS score, and DNT) | SVM | Primary data 2237 | - SICH | AUC = 0.79 |
Neural Network | AUC = 0.82 | ||||
LR | AUC = 0.77 | ||||
AdaBoost | AUC = 0.77 | ||||
RF | AUC = 0.76 | ||||
Bacchi S, 2020 [90] | New prediction models | CNN + ANN CTB + clinical data | Primary data 204 | 90 days 1) 24-h outcome (based on NIHSS) 2) 90-day outcome (based on mRS) | 1) AUC = 0.70; Se = 93%; Sp = 53%; Acc = 71% 2) AUC = 0.75; Se = 56%; Sp = 93%; Acc = 74% |
ANN | 1) AUC = 0.68; Se = 43%; Sp = 88%; Acc = 68% 2) AUC = 0.61; Se = 81%; Sp = 47%; Acc = 65% | ||||
CNN CTB | 1) AUC = 0.63; Se = 71%; Sp = 65%; Acc = 68% 2) AUC = 0.54; Se = 88%; Sp = 40%; Acc = 65% | ||||
THRIVE | - | 1) AUC = 0.63; Se = 43%; Sp = 76%; Acc = 61% 2) AUC = 0.69; Se = 81%; Sp = 47%; Acc = 65% | |||
HAT | - | 1) AUC = 0.67; Se = 71%; Sp = 59%; Acc = 65% 2) AUC = 0.63; Se = 56%; Sp = 67%; Acc = 61% | |||
SPAN 100 | - | 1) AUC = N/A; Se = 21%; Sp = 88%; Acc = 58% 2) AUC = N/A; Se = 100%; Sp = 33%; Acc = 67% | |||
Chung CC, 2020 [91] | New prediction models (Important variables: BP, HR, glucose level, consciousness level, NIHSS score, and history of DM) | ANN | Primary data Training group: 157 Testing group: 39 | 24 h major neurologic improvement (based on mRS) | Training group: AUC = 0.950; Acc = 97.5% Testing group: AUC = 0.944; Se = 89.8%; Sp = 95.9%; Acc = 94.6% |
ANN | 3-month outcome (based on mRS) | Training group: AUC = 0.992; Acc = 98.2% Testing group: AUC = 0.933; Se = 94.3%; Sp = 86.5%; Acc = 88.8% | |||
Bentley P, 2014 [92] | New radiomic models | Automated SVM | Primary data 116 | During hospitalization SICH: NINDS | AUC = 0.744 |
Manual SVM | AUC = 0.671 | ||||
SEDAN | - | AUC = 0.626 | |||
HAT | - | AUC = 0.629 | |||
SEDAN-NIHSS, CT only | - | AUC = 0.720 | |||
HAT-NIHSS, CT only | - | AUC = 0.648 |