A Recall-Oriented Comparative Evaluation of Machine Learning Models for Heart Disease Detection on the Cleveland Heart Disease Dataset
Main Article Content
Abstract
Heart disease screening models are usually tuned for accuracy, although a missed diagnosis costs far more than a false alarm. This study compared six classifiers (Decision Tree, Random Forest, Support Vector Machine, XGBoost, Logistic Regression and a Multilayer Perceptron) on the 303-patient Cleveland Heart Disease dataset, in which 139 patients had angiographic disease. A widely shared copy of the dataset was found to carry a reversed outcome label, so all experiments used the original repository file. Hyperparameters were tuned by repeated nested cross-validation, and each model's decision threshold was fixed on training data to reach a recall of at least 0.90. Across 50 outer test folds, every model met the recall target (0.89 to 0.91). Logistic Regression achieved the highest mean specificity (0.68; 95% CI 0.60 to 0.77) and precision-recall AUC (0.91), although its advantage over Random Forest and Support Vector Machine was not statistically significant. The Decision Tree ruled out far fewer disease-free patients (specificity 0.43). Restricting the inputs to five pre-test variables reduced Logistic Regression specificity to 0.57. Logistic Regression is recommended as an interpretable, recall-oriented baseline, subject to external validation in unselected primary-care populations.
Article Details
Copyright (c) 2026 Reginald Ekene Ogu, Juliana Ihuefo, Precious Iheanacho (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
Ahmad, A., & Polat, H. (2023). Prediction of heart disease based on machine learning using jellyfish optimization algorithm. Diagnostics, 13(14), 2392. https://doi.org/10.3390/diagnostics13142392 DOI: https://doi.org/10.3390/diagnostics13142392
Altman, D. G., & Bland, J. M. (1994). Diagnostic tests 2: Predictive values. BMJ, 309(6947), 102. https://doi.org/10.1136/bmj.309.6947.102 DOI: https://doi.org/10.1136/bmj.309.6947.102a
Banerjee, T., & Paçal, İ. (2025). A systematic review of machine learning in heart disease prediction. Turkish Journal of Biology, 49(5), 600–634. https://doi.org/10.55730/1300-0152.2766 DOI: https://doi.org/10.55730/1300-0152.2766
Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32. https://doi.org/10.1023/A:1010933404324 DOI: https://doi.org/10.1023/A:1010933404324
Chakraborty, A., Das, U. K., Sazzad, S., Das, P., & Khan, M. M. H. (2026). Explainable machine learning for early heart disease risk prediction: Insights from a clinical dataset in Bangladesh. Intelligence-Based Medicine, 13, 100352. https://doi.org/10.1016/j.ibmed.2026.100352 DOI: https://doi.org/10.1016/j.ibmed.2026.100352
Chen, T., & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 785–794). Association for Computing Machinery. https://doi.org/10.1145/2939672.2939785 DOI: https://doi.org/10.1145/2939672.2939785
Detrano, R., Janosi, A., Steinbrunn, W., Pfisterer, M., Schmid, J.-J., Sandhu, S., Guppy, K. H., Lee, S., & Froelicher, V. (1989). International application of a new probability algorithm for the diagnosis of coronary artery disease. The American Journal of Cardiology, 64(5), 304–310. https://doi.org/10.1016/0002-9149(89)90524-9 DOI: https://doi.org/10.1016/0002-9149(89)90524-9
Dietterich, T. G. (1998). Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation, 10(7), 1895–1923. https://doi.org/10.1162/089976698300017197 DOI: https://doi.org/10.1162/089976698300017197
Gul, G., Korejo, I. A., Hakro, D. N., Alqahtani, H., Abbasi, A., Babar, M., Al Rahbi, O., & Ali, N. I. (2026). Machine learning and ensemble methods for cardiovascular disease prediction: A systematic review of approaches, performance trends, and research challenges. Computers, 15(1), 25. https://doi.org/10.3390/computers15010025 DOI: https://doi.org/10.3390/computers15010025
He, H., & Garcia, E. A. (2009). Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering, 21(9), 1263–1284. https://doi.org/10.1109/TKDE.2008.239 DOI: https://doi.org/10.1109/TKDE.2008.239
Ilyas, A. R., Javaid, S., & Kharisma, I. L. (2025). Heart disease prediction using ML. Engineering Proceedings, 107(1), 124. https://doi.org/10.3390/engproc2025107124 DOI: https://doi.org/10.3390/engproc2025107124
Janosi, A., Steinbrunn, W., Pfisterer, M., & Detrano, R. (1989). Heart disease [Dataset]. UCI Machine Learning Repository. https://doi.org/10.24432/C52P4X
Khan, S. S., Bhave, N., Blumenthal, R. S., Coresh, J., Huang, X., Joseph, J. J., Khera, A., Ho, J. E., Lloyd-Jones, D. M., Low Wang, C. C., Lu, Y., Morris, P. B., Nasir, K., Natarajan, P., Rangaswami, J., Rodriguez, F., Sperling, L. S., Virani, S. S., Zhang, S., & Ndumele, C. E. (2026). Use of predicted risk and expected benefit to guide decision-making in cardiovascular-kidney-metabolic syndrome for the primary prevention of cardiovascular disease: A scientific statement from the American Heart Association and American College of Cardiology. Circulation, 154(4), e159–e180. https://doi.org/10.1161/CIR.0000000000001447 DOI: https://doi.org/10.1161/CIR.0000000000001447
Kumar, R., Garg, S., Kaur, R., Johar, M. G. M., Singh, S., Menon, S. V., Kumar, P., Hadi, A. M., Hasson, S. A., & Lozanović, J. (2025). A comprehensive review of machine learning for heart disease prediction: Challenges, trends, ethical considerations, and future directions. Frontiers in Artificial Intelligence, 8, 1583459. https://doi.org/10.3389/frai.2025.1583459 DOI: https://doi.org/10.3389/frai.2025.1583459
Latha, C. B. C., & Jeeva, S. C. (2019). Improving the accuracy of prediction of heart disease risk based on ensemble classification techniques. Informatics in Medicine Unlocked, 16, 100203. https://doi.org/10.1016/j.imu.2019.100203 DOI: https://doi.org/10.1016/j.imu.2019.100203
Mohan, S., Thirumalai, C., & Srivastava, G. (2019). Effective heart disease prediction using hybrid machine learning techniques. IEEE Access, 7, 81542–81554. https://doi.org/10.1109/ACCESS.2019.2923707 DOI: https://doi.org/10.1109/ACCESS.2019.2923707
Mohapatra, S., Maneesha, S., Mohanty, S., Patra, P. K., Bhoi, S. K., Sahoo, K. S., & Gandomi, A. H. (2023). A stacking classifiers model for detecting heart irregularities and predicting cardiovascular disease. Healthcare Analytics, 3, 100133. https://doi.org/10.1016/j.health.2022.100133 DOI: https://doi.org/10.1016/j.health.2022.100133
Nadeau, C., & Bengio, Y. (2003). Inference for the generalization error. Machine Learning, 52(3), 239–281. https://doi.org/10.1023/A:1024068626366 DOI: https://doi.org/10.1023/A:1024068626366
Nagavelli, U., Samanta, D., & Chakraborty, P. (2022). Machine learning technology-based heart disease detection models. Journal of Healthcare Engineering, 2022, 7351061. https://doi.org/10.1155/2022/7351061 DOI: https://doi.org/10.1155/2022/7351061
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, É. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.
Pedroso, A. F., Nascimento, B. R., Dhingra, L. S., Shankar, S. V., Vinhal, W. C., e Reges, R. B., Cardoso, C. S., Sable, C., Ribeiro, A. L. P., & Khera, R. (2026). Artificial intelligence-enabled electrocardiography to triage echocardiography for structural heart disease diagnosis in a low-resource setting. American Journal of Preventive Cardiology, 27, 101539. https://doi.org/10.1016/j.ajpc.2026.101539 DOI: https://doi.org/10.1016/j.ajpc.2026.101539
Rehman, M. U., Naseem, S., Butt, A., Mahmood, T., Khan, A. R., Khan, I., Khan, J., & Jung, Y. (2025). Predicting coronary heart disease with advanced machine learning classifiers for improved cardiovascular risk assessment. Scientific Reports, 15, 13361. https://doi.org/10.1038/s41598-025-96437-1 DOI: https://doi.org/10.1038/s41598-025-96437-1
Rhee, T.-M., Kang, S., Lee, M. S., Han, G. I., Yoo, A.-H., Jang, J.-H., Jo, Y.-Y., Son, J. M., Kwon, J., Choi, S.-Y., Lee, H. S., & Lee, H. (2026). Artificial intelligence–driven electrocardiogram screening for asymptomatic left ventricular systolic dysfunction in the general population. JACC: Advances, 5(4), 102660. https://doi.org/10.1016/j.jacadv.2026.102660 DOI: https://doi.org/10.1016/j.jacadv.2026.102660
Safdar, S., Zafar, S., Zafar, N., & Khan, N. F. (2018). Machine learning based decision support systems (DSS) for heart disease diagnosis: A review. Artificial Intelligence Review, 50(4), 597–623. https://doi.org/10.1007/s10462-017-9552-8 DOI: https://doi.org/10.1007/s10462-017-9552-8
Saito, T., & Rehmsmeier, M. (2015). The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS ONE, 10(3), e0118432. https://doi.org/10.1371/journal.pone.0118432 DOI: https://doi.org/10.1371/journal.pone.0118432
Suleiman, A. B., Luka, S., & Ibrahim, M. (2023). Cardiovascular disease prediction using random forest machine learning algorithm. FUDMA Journal of Sciences, 7(6). https://doi.org/10.33003/fjs-2023-0706-2128 DOI: https://doi.org/10.33003/fjs-2023-0706-2128
Ullah, I., Inayat, T., Ullah, N., Alzahrani, F., & Khan, M. I. (2023). Clinical decision support system (CDSS) for heart disease diagnosis and prediction by machine learning algorithms: A systematic literature review. Journal of Mechanics in Medicine and Biology, 23(10), 2330001. https://doi.org/10.1142/S0219519423300016 DOI: https://doi.org/10.1142/S0219519423300016
Van Calster, B., McLernon, D. J., van Smeden, M., Wynants, L., Steyerberg, E. W., Bossuyt, P., Collins, G. S., Macaskill, P., Moons, K. G. M., & Vickers, A. J. (2019). Calibration: The Achilles heel of predictive analytics. BMC Medicine, 17(1), 230. https://doi.org/10.1186/s12916-019-1466-7 DOI: https://doi.org/10.1186/s12916-019-1466-7
Varma, S., & Simon, R. (2006). Bias in error estimation when using cross-validation for model selection. BMC Bioinformatics, 7, 91. https://doi.org/10.1186/1471-2105-7-91 DOI: https://doi.org/10.1186/1471-2105-7-91
Wan, S., Wan, F., & Dai, X. (2025). Machine learning approaches for cardiovascular disease prediction: A review. Archives of Cardiovascular Diseases, 118(10), 554–562. https://doi.org/10.1016/j.acvd.2025.04.055 DOI: https://doi.org/10.1016/j.acvd.2025.04.055
Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 22(158), 209–212. https://doi.org/10.1080/01621459.1927.10502953 DOI: https://doi.org/10.1080/01621459.1927.10502953