Deep Reinforcement Learning Based Cyberattack Detection in Supervisory Control and Data Acquisition System
DOI:
https://doi.org/10.21467/proceedings.7.4.6Keywords:
Industrial Control System, SARSA, Deep Reinforcement LearningAbstract
The cybersecurity landscape is continuously evolving and dynamic, as new attackers develop increasingly sophisticated methods to attack the targeted organizations. Traditional cybersecurity strategies focused mainly on safeguarding the information using the core principles of the Confidentiality, Integrity, and Availability i.e. CIA triad. However, in Supervisory Control and Data Acquisition (SCADA) systems, existing intrusion detection mechanisms have certain limitations when it comes to identifying the abnormalities effectively. Researchers have widely explored different types of Machine Learning (ML) techniques and Deep Learning (DL) algorithms to detect the threats faced by Industrial Control Systems (ICS). Although these techniques have provided some level of protection, they have proven to be insufficient in fully securing these systems against evolving cyber threats. To tackle this problem, we propose a novel approach based on Deep Reinforcement Learning (DRL) to amplify the identification of cyber-attacks in SCADA networks. Our model proposes the “SARSA algorithm,” a model-free reinforcement learning technique designed to evaluate the state-action value pairs. SARSA employs an on-policy strategy, it learns from the actions currently taken according to the ongoing policy, allowing for proactive and adaptive intrusion detection. It updates the value in regards to the action selected by the ongoing policy. This approach allows for immediate updates, enabling our model to adapt and respond to intrusions more efficiently. For validation, we use the WUSTL-IIOT- 2021 dataset, a publicly available dataset that includes twenty-five number of networking features representing both attack traffic and benign. Experimental results illustrate that our proposed algorithm achieved a good accuracy in detecting the cyber threats and highlights the hypothetical SARSA-based techniques to strengthen the security of critical infrastructure.
References
[1] F. Mesadieu, D. Torre, and A. Chennameneni, “Leveraging Deep Reinforcement Learning Technique for Intrusion Detection in SCADA Infrastructure,” IEEE Access, vol. 12, no. May, pp. 63381–63399, 2024, doi: 10.1109/ACCESS.2024.3390722.
[2] K. Shaukat, S. Luo, V. Varadharajan, I. A. Hameed, and M. Xu, “A Survey on Machine Learning Techniques for Cyber Security in the Last Decade,” IEEE Access, vol. 8, pp. 222310– 222354, 2020, doi: 10.1109/ACCESS.2020.3041951.
[3] A. L. Buczak and E. Guven, “A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection,” IEEE Commun. Surv. Tutorials, vol. 18, no. 2, pp. 1153– 1176, Apr. 2016, doi: 10.1109/COMST.2015.2494502.
[4] S. Mohamed and R. Ejbali, “Deep SARSA-based reinforcement learning approach for anomaly,” Int. J. Inf. Secur., vol. 22, no. 1, pp. 235–247, 2023, doi: 10.1007/s10207-022-00634- 2.
[5] C. M. Moreira, “QL vs . SARSA : Performance Evaluation for Intrusion Prevention Systems in Software-Defined IoT Networks,” 2023 Int. Wirel. Commun. Mob. Comput., pp. 500–504, 2023, doi: 10.1109/IWCMC58020.2023.10183144.
[6] D. Wang et al., “Peer-to-peer Electricity Transaction Decisions of the User-side Smart Energy System Based on the SARSA Reinforcement Learning,” vol. 8, no. 3, pp. 826–837, 2022, doi: 10.17775/CSEEJPES.2020.03290.
[7] H. Moradimaryamnegari and M. Frego, “Model Predictive Control-Based Reinforcement Learning Using Expected Sarsa,” IEEE Access, vol. 10, no. August, pp. 81177–81191, 2022, doi: 10.1109/ACCESS.2022.3195530.
[8] Z. Jin, M. Ma, S. Zhang, Y. Hu, Y. Zhang, and C. Sun, “Secure State Estimation of Cyber- Physical System under Cyber Attacks: Q- Learning vs. SARSA,” Electron., vol. 11, no. 19, pp. 1–19, 2022, doi: 10.3390/electronics11193161.
[9] M. H. Olyaei, H. Jalali, A. Olyaei, and A. Noori, Implement Deep SARSA in Grid World with Changing Obstacles and Testing Against New Environment : The Selected Papers of The First International Conference on Fundamental Researchin ... Implement Deep SARSA in Grid World with Changing Obstacles and Testin, no. January. Springer Singapore, 2019. doi: 10.1007/978-981-10-8672-4.
[10] T. Alfakih, M. M. Hassan, A. Gumaei, C. Savaglio, and G. Fortino, “Task Offloading and Resource Allocation for Mobile Edge Computing by Deep Reinforcement Learning Based on SARSA,” IEEE Access, vol. 8, pp. 54074–54084, 2020, doi: 10.1109/ACCESS.2020.2981434.
[11] T. T. Nguyen and V. J. Reddi, “Deep Reinforcement Learning for Cyber Security,” IEEE Trans. Neural Networks Learn. Syst., vol. 34, no. 8, pp. 3779–3795, 2023, doi: 10.1109/TNNLS.2021.3121870.
[12] M. Zolanvari, M. A. Teixeira, L. Gupta, K. M. Khan, and R. Jain, “Machine Learning- Based Network Vulnerability Analysis of Industrial Internet of Things,” IEEE Internet Things J., vol. 6, no. 4, pp. 6822–6834, Aug. 2019, doi: 10.1109/JIOT.2019.2912022.
[13] F. Khan, R. Alturki, M. A. Rahman, S. Mastorakis, I. Razzak, and S. T. Shah, “Trustworthy and Reliable Deep-Learning-Based Cyberattack Detection in Industrial IoT,” IEEE Trans. Ind. Informatics, vol. 19, no. 1, pp. 1030–1038, Jan. 2023, doi: 10.1109/TII.2022.3190352.
[14] R. S. Tiwari, D. Lakshmi, T. K. Das, A. K. Tripathy, and K. C. Li, “A lightweight optimized intrusion detection system using machine learning for edge-based IIoT security,” Telecommun. Syst., vol. 87, no. 3, pp. 605–624, 2024, doi: 10.1007/s11235-024-01200-y.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 K K Harisha, N Sivakumaran, T K Radhakrishnan (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
