Principal Component Optimization for Random Forest-Based Classification of Palm Oil Leaf Disease Images

Abstract

Principal Component Analysis (PCA) is a widely used dimensionality reduction technique for mitigating high-dimensional feature spaces, while Random Forest is a robust ensemble classifier that can naturally handle many input variables. However, the effect of the number of PCA components on the predictive performance and generalization ability of Random Forest models is still not well quantified, especially in terms of its trade-off between information preservation and noise reduction. This study investigates how varying the number of PCA components from 2 to 10 influences the performance of a Random Forest classifier on a multiclass dataset. The experimental design employs k-fold cross-validation and multiple values of the number of trees (n_estimators), and evaluates models using Accuracy, Precision, Recall, F1-score, and training time. The results exhibit an inverted U-shaped relationship, where 6–7 PCA components yield the highest and most stable performance, with average Accuracy around 0.96 and F1-score around 0.97, while very low (2–3) and high (?8) numbers of components lead to underfitting and structural overfitting, respectively. These findings suggest that PCA-based dimensionality reduction should be tuned with respect to discriminative performance rather than solely maximizing explained variance, and that a moderate number of components can best exploit the synergy between PCA and Random Forest.