Principal component analysis as tool for data reduction with an application

Keywords: Kernel Functions, Principal Component Analysis, Renal failure, Support Vector Regression


The recent trends in collecting huge datasets have posed a great challenge that is brought by the high dimensionality and aggravated by the presence of irrelevant dimensions. Machine learning models for regression is recognized as a convenient way of improving the estimation for empirical models. Popular machine learning models is support vector regression (SVR). However, the usage of principal component analysis (PCA) as a variable reduction method along with SVR is suggested. The principal component analysis helps in building a predictive model that is simple as it contains the smallest number of variables and efficient. In this paper, we investigate the competence of SVR with PCA to explore its performance for a more accurate estimation. Simulation study and Renal Failure (RF) data of SVR optimized by four different kernel functions; linear, polynomial, radial basis, and sigmoid functions using R software, version (R x64 3.2.5) to compare the behavior of ε SVR and v-SVR models for different sample sizes ranges from small, moderate to large such as; 50, 100, and 150. The performance criteria are root mean squared error (RMSE) and coefficient of determination R2 showed the superiority of ε-SVR over v- SVR. Furthermore, the implementation of SVR after employing PCA improves the results. Also, the simulation results showed that the best performing kernel function is the linear kernel. For real data the results showed that the best kernels are linear and radial basis function. It is also clear that, with ε-SVR and v-SVR, the RMSE values for almost kernel functions decreased with increasing sample size. Therefore, the performance of ε-SVR improved after applying PCA. In addition sample size n=50 gave good results for linear and radial kernel


Download data is not yet available.

Author Biographies

Shereen Hamdy Abdel Latif, Cairo University

Department of Applied Statistics and Econometrics

Asraa Sadoon Alwan, Cairo University

Department of Applied Statistics and Econometrics

Amany Mousa Mohamed, Cairo University

Department of Applied Statistics and Econometrics


Chowdhury, U. N., Chakravarty, S. K., Hossain, Md. T. (2018). Short-Term Financial Time Series Forecasting Integrating Principal Component Analysis and Independent Component Analysis with Support Vector Regression. Journal of Computer and Communications, 06 (03), 51–67. doi:

Yu, H., Chen, R., Zhang, G. (2014). A SVM Stock Selection Model within PCA. Procedia Computer Science, 31, 406–412. doi:

‏Glaser, J. I., Benjamin, A. S., Farhoodi, R., Kording, K. P. (2019). The roles of supervised machine learning in systems neuroscience. Progress in Neurobiology, 175, 126–137. doi:

Lee, J. A., Verleysen, M. (2009). Quality assessment of dimensionality reduction: Rank-based criteria. Neurocomputing, 72 (7-9), 1431–1443. doi:

Jolliffe, I. T., Cadima, J. (2016). Principal component analysis: a review and recent developments. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 374 (2065), 20150202. doi:

Rosipal, R., Girolami, M., Trejo, L. J., Cichocki, A. (2001). Kernel PCA for Feature Extraction and De-Noising in Nonlinear Regression. Neural Computing & Applications, 10 (3), 231–243. doi:

Shi, S., Li, G., Chen, H., Hu, Y., Wang, X., Guo, Y., Sun, S. (2018). An efficient VRF system fault diagnosis strategy for refrigerant charge amount based on PCA and dual neural network model. Applied Thermal Engineering, 129, 1252–1262. doi:

Chao, D., Zhou, W., Ye, C., Zhang, Q., Chen, Y., Gu, L. et. al. (2019). An electrolytic Zn–MnO2 battery for high‐voltage and scalable energy storage. Angewandte Chemie International Edition, 58 (23), 7823–7828.‏ doi:

Vapnik, V. (2000). The nature of statistical learning theory. Springer, 314.‏ doi:

Mechelli, A., Vieira, S. (Eds.) (2019). Machine learning: methods and applications to brain disorders. Academic Press.‏ doi:

Blanco, V., Puerto, J., Rodriguez-Chia, A. M. (2020). On lp-Support Vector Machines and Multidimensional Kernels. Journal of Machine Learning Research, 21 (14).‏ Available at:

Astuti, W., Adiwijaya (2018). Support vector machine and principal component analysis for microarray data classification. Journal of Physics: Conference Series, 971, 012003. doi:

Drucker, H., Burges, C. J., Kaufman, L., Smola, A., Vapnik, V. (1996). Support vector regression machines. Advances in Neural Information Processing Systems 9 (NIPS 1996), 155–161. Available at:

Jiang, M., Zhu, L., Wang, Y., Xia, L., Shou, G., Liu, F., Crozier, S. (2011). Application of kernel principal component analysis and support vector regression for reconstruction of cardiac transmembrane potentials. Physics in Medicine and Biology, 56 (6), 1727–1742. doi:

Shokri, S., Sadeghi, M., Marvast, M., Narasimhan, S. (2015). Integrating principal component analysis and vector quantization with support vector regression for sulfur content prediction in HDS process. Chemical Industry and Chemical Engineering Quarterly, 21 (3), 379–390. doi:

Chowdhury, U. N., Rayhan, M. A., Chakravarty, S. K., Hossain, M. T. (2017). Integration of principal component analysis and support vector regression for financial time series forecasting. International Journal of Computer Science and Information Security (IJCSIS), 15 (8), 28–32.‏

Naik, G. R. (Ed.) (2018). Advances in Principal Component Analysis. Springer. doi:

Vapnik, V. N. (1998). Statistical Learning Theory. Adaptive and Learning Systems for Signal Processing, Communications, and Control. Signal Processing, Communications, and Control.

Scholkopf, B., Bartlett, P. L., Smola, A. J., Williamson, R. (1999). Shrinking the tube: a new support vector regression algorithm. Advances in Neural Information Processing Systems, 330–336.‏

Higuchi, I., Eguchi, S. (2004). Robust principal component analysis with adaptive selection for tuning parameters. Journal of Machine Learning Research, 453–471.‏

Root mean squared error of ε-SVR after applying pincipal component analysis for n (real data results)

👁 83
⬇ 56
How to Cite
Latif, S. H. A., Alwan, A. S., & Mohamed, A. M. (2022). Principal component analysis as tool for data reduction with an application . EUREKA: Physics and Engineering, (5), 184-198.