Comparative evaluation of resampling techniques for improving the performance of classification algorithms on a breast cancer dataset
DOI:
https://doi.org/10.21303/2461-4262.2026.004289Keywords:
artificial intelligence, machine learning, classification algorithms, resampling techniques, performance metricsAbstract
Class imbalance is a common problem in machine learning classification tasks and may negatively affect the reliability and predictive performance of classification models. The research object is the comprehensive evaluation of different resampling techniques applied to a publicly available breast cancer dataset consisting of 569 records categorized as benign and malignant. To address class imbalance, four resampling techniques available in the WEKA (Waikato Environment for Knowledge Analysis) environment, namely Resample, SMOTE (Synthetic Minority Over-sampling Technique), SpreadSubSample, and StratifiedRemoveFolds, were applied to the dataset. The resulting datasets were evaluated using NB (Naive Bayes), KNN (K-Nearest Neighbors), and DT (Decision Tree) classification algorithms. Algorithm performance was assessed using accuracy, precision, recall, and F1-score metrics. The experimental results indicate that resampling techniques have a significant impact on classification performance. Among the evaluated methods, StratifiedRemoveFolds produced the highest accuracy values, achieving 0.982 with Naive Bayes and 0.964 with Decision Tree. In contrast, SpreadSubSample generally reduced the performance of all evaluated algorithms. Furthermore, the findings revealed that different resampling techniques influence algorithms in different ways. These results demonstrate that the effectiveness of a resampling method depends on the characteristics of both the dataset and the classification algorithm used. The research highlights the practical application of appropriate resampling strategies in medical decision-support systems and other machine learning applications involving imbalanced datasets, where reliable classification performance is required.
Downloads
References
Chen, W., Yang, K., Yu, Z., Shi, Y., Chen, C. L. P. (2024). A survey on imbalanced learning: latest research, applications and future directions. Artificial Intelligence Review, 57 (6). https://doi.org/10.1007/s10462-024-10759-6
Altalhan, M., Algarni, A., Turki-Hadj Alouane, M. (2025). Imbalanced Data Problem in Machine Learning: A Review. IEEE Access, 13, 13686–13699. https://doi.org/10.1109/access.2025.3531662
Salmi, M., Atif, D., Oliva, D., Abraham, A., Ventura, S. (2024). Handling imbalanced medical datasets: review of a decade of research. Artificial Intelligence Review, 57 (10). https://doi.org/10.1007/s10462-024-10884-2
Kim, A., Jung, I. (2023). Optimal selection of resampling methods for imbalanced data with high complexity. Plos One, 18 (7), e0288540. https://doi.org/10.1371/journal.pone.0288540
Mujahid, M., Kına, E., Rustam, F., Villar, M. G., Alvarado, E. S., De La Torre Diez, I., Ashraf, I. (2024). Data oversampling and imbalanced datasets: an investigation of performance for machine learning and feature engineering. Journal of Big Data, 11 (1). https://doi.org/10.1186/s40537-024-00943-4
Carvalho, M., Pinho, A. J., Brás, S. (2025). Resampling approaches to handle class imbalance: a review from a data perspective. Journal of Big Data, 12 (1). https://doi.org/10.1186/s40537-025-01119-4
Ghorbani, R., Ghousi, R. (2020). Comparing Different Resampling Methods in Predicting Students’ Performance Using Machine Learning Techniques. IEEE Access, 8, 67899–67911. https://doi.org/10.1109/access.2020.2986809
Kubus, M. (2020). Evaluation of resampling methods in the class unbalance problem. Econometrics, 24 (1), 39–50. https://doi.org/10.15611/eada.2020.1.04
Alahmari, F. (2020). A Comparison of Resampling Techniques for Medical Data Using Machine Learning. Journal of Information & Knowledge Management, 19 (01), 2040016. https://doi.org/10.1142/s021964922040016x
Türkmenoğlu, B. K., Yildiz, O. (2021). Predicting the Survival of Heart Failure Patients in Unbalanced Data Sets. 2021 29th Signal Processing and Communications Applications Conference (SIU), 1–4. https://doi.org/10.1109/siu53274.2021.9477806
Harman, G. (2021). Destek Vektör Makineleri ve Naive Bayes Sınıflandırma Algoritmalarını Kullanarak Diabetes Mellitus Tahmini. European Journal of Science and Technology. https://doi.org/10.31590/ejosat.1041186
Nakatsu, R. T. (2021). An Evaluation of Four Resampling Methods Used in Machine Learning Classification. IEEE Intelligent Systems, 36 (3), 51–57. https://doi.org/10.1109/mis.2020.2978066
Breast Cancer Wisconsin (Diagnostic). Available at: https://archive.ics.uci.edu/dataset/17/breast+cancer+wisconsin+diagnostic
Caruana, R., Niculescu-Mizil, A. (2006). An empirical comparison of supervised learning algorithms. Proceedings of the 23rd International Conference on Machine Learning - ICML ’06, 161–168. https://doi.org/10.1145/1143844.1143865
Burhaeın, E., Fadjerı, A., Widiyono, I. P. (2024). Application of Naive Bayes Algorithm for Physical Fitness Level Classification. International Journal of Disabilities Sports and Health Sciences, 7 (1), 178–187. https://doi.org/10.33438/ijdshs.1330745
Karataş, A. (2024). Improving Automatic Migraine Classification Performance with Naive Bayes. Sakarya Üniversitesi Fen Bilimleri Enstitüsü Dergisi, 28 (4), 816–823. https://doi.org/10.16984/saufenbilder.1332882
Avuçlu, E., Elen, A. (2019). Classification of Cardiotocography Records with Naïve Bayes. International Scientific and Vocational Studies Journal, 3 (2), 105–110. Available at: https://izlik.org/JA78UA77KY
Güvenç, E., Sakal, M., Çetin, G., Özkaraca, O. (2022). Classification of Students’ Course Qualifications Using Machine Learning Techniques. Düzce Üniversitesi Bilim Ve Teknoloji Dergisi, 10 (3), 1359–1371. https://doi.org/10.29130/dubited.1017202
Uludağ, O., Gürsoy, A. (2020). KNN ve Naive Bayes Sınıflandırma Algoritmaları ile Finansal Durum Analizi Üzerine. Iğdır Üniversitesi Fen Bilimleri Enstitüsü Dergisi, 10 (4), 2881–2888. https://doi.org/10.21597/jist.703004
Paşa, M., Özer, Z. (2025). Makine Öğrenmesi Yöntemleri ile Banka Müşteri Kaybı Tahmini. İstanbul Ticaret Üniversitesi Sosyal Bilimler Dergisi, 24 (54), 874–906. https://doi.org/10.46928/iticusbe.1656859
Sullivan, W. (2017). Machine Learning For Beginners Guide Algorithms: Supervised & Unsupervsied Learning. Decision Tree & Random Forest Introduction (Artificial Intelligence Book 1). Kindle Edition.
Uğurlu, B., Karasulu, B. (2025). Zaman Serisi Tabanlı Makine Öğrenmesi Modelleri ile GitHub Projelerindeki Programlama Dili Popülerliğinin Tahmini (2011–2021). Kırklareli Üniversitesi Mühendislik Ve Fen Bilimleri Dergisi, 11 (2), 351–365. https://doi.org/10.34186/klujes.1790613
Ateş, İ., Bilgin, T. T. (2021). Meme Kanseri Tespitinde Farklı Makine Öğrenmesi Yöntemleri Başarısının İncelenmesi. Konuralp Tıp Dergisi, 13 (2), 347–356. https://doi.org/10.18521/ktd.912462
Goy, G., Gezer, C., Gungor, V. C. (2019). Credit Card Fraud Detection with Machine Learning Methods. 2019 4th International Conference on Computer Science and Engineering (UBMK), 350–354. https://doi.org/10.1109/ubmk.2019.8906995
Estabrooks, A., Jo, T., Japkowicz, N. (2004). A Multiple Resampling Method for Learning from Imbalanced Data Sets. Computational Intelligence, 20 (1), 18–36. Portico. https://doi.org/10.1111/j.0824-7935.2004.t01-1-00228.x
Katore, L. S., Umale, J. S. ((2015). Comparative Study of Recommendation Algorithms and Systems using WEKA. International Journal of Computer Applications, 110 (3), 14–17. https://doi.org/10.5120/19295-0731
Fernandez, A., Garcia, S., Herrera, F., Chawla, N. V. (2018). SMOTE for Learning from Imbalanced Data: Progress and Challenges, Marking the 15-year Anniversary. Journal of Artificial Intelligence Research, 61, 863–905. https://doi.org/10.1613/jair.1.11192
Mirmozaffari, M., Golilarz, N. A., Band, S. S. (2020). Machine Learning Algorithms Based on an Optimization Model. https://doi.org/10.20944/preprints202009.0729.v1
Downloads
Published
Versions
- 2026-09-30 (Version of Record 1.0)
Data Availability Statement
Manuscript has associated data in a data repository.
Funding Statement
The research was performed without financial support.
Issue
Section
License
Copyright (c) 2026 Ogtay Safaraliyev (Author); Khifayat Mammadova (Author)

This work is licensed under a Creative Commons Attribution 4.0 International License.
Our journal abides by the Creative Commons CC BY copyright rights and permissions for open access journals.
Authors, who are published in this journal, agree to the following conditions:
1. The authors reserve the right to authorship of the work and pass the first publication right of this work to the journal under the terms of a Creative Commons CC BY, which allows others to freely distribute the published research with the obligatory reference to the authors of the original work and the first publication of the work in this journal.
2. The authors have the right to conclude separate supplement agreements that relate to non-exclusive work distribution in the form in which it has been published by the journal (for example, to upload the work to the online storage of the journal or publish it as part of a monograph), provided that the reference to the first publication of the work in this journal is included.
