PREDIKSI FRAUD TRANSAKSI DOMPET DIGITAL MENGGUNAKAN ALGORITMA CATBOOST

Abstract

The growth of digital transactions via digital wallet and mobile money services has heightened the need for fraud detection systems capable of accurately identifying suspicious transactions. A major challenge in fraud detection is class imbalance, where the volume of fraudulent transactions is significantly lower than that of legitimate ones. This study aims to analyze and compare the performance of Random Forest and CatBoost algorithms in detecting fraud within digital transactions using the PaySim dataset. Random Forest served as the benchmark model, while CatBoost was the primary model evaluated through both baseline configurations and hyperparameter tuning. Class imbalance was addressed using class weighting, and the dataset was partitioned using an 80:20 train-test split combined with stratified sampling. Model performance was evaluated using Accuracy, Precision, Recall, F1-Score, ROC-AUC, and PR-AUC, alongside an analysis of False Positives and False Negatives. The results indicate that while Random Forest achieved a Recall of 98.78%, it produced 18,284 False Positives, resulting in a Precision of 8.15%. The CatBoost Baseline model achieved a Precision of 79.19%, Recall of 85.27%, F1-Score of 82.12%, ROC-AUC of 99.92%, and PR-AUC of 90.93%. Following hyperparameter tuning, the CatBoost Tuned model improved its Recall to 92.08%, ROC-AUC to 99.96%, and PR-AUC to 94.46%, while reducing the number of False Negatives from 242 to 130 transactions; however, Precision dropped to 71.46% and the F1-Score to 80.48%. These findings demonstrate that CatBoost Tuned possesses superior fraud detection capabilities—particularly in maximizing Recall and minimizing False Negatives—whereas CatBoost Baseline offers a better balance between Precision and Recall at the classification threshold used in this study.