An Agent-Based Reinforcement Learning Framework for Dynamic Cryptographic Security
DOI:
https://doi.org/10.65204/djes.v3i1.536Keywords:
Intelligent Agent, Reinforcement Learning, Encryption System, Information Security, Machine LearningAbstract
The increasing sophistication of cyber threats has exposed critical vulnerabilities in conventional cryptographic systems, necessitating innovative and adaptive security measures. In response, a multi-agent reinforcement learning framework is proposed to optimize cryptographic operations dynamically through intelligent, adaptive agents. The system integrates a Defense Agent, which selects appropriate encryption algorithms and key sizes, and a Key Management Agent, which governs adaptive key rotation strategies. Additionally, an Attacker Agent is employed to simulate realistic adversarial tactics, facilitating a co-evolutionary learning environment. The approach is grounded in an asynchronous Actor-Critic architecture, which continuously adjusts policy parameters based on the advantage function to reinforce effective defense strategies. Experimental evaluations across escalating threat scenarios reveal a significant enhancement in the system’s resilience, as demonstrated by prolonged time-to-compromise, reduced damage impact, and a high threat interception rate while maintaining acceptable resource overhead and encryption latency. These results underscore the potential of the proposed adaptive framework to substantially mitigate risks and elevate the robustness of cryptographic systems, thus providing a promising avenue for next-generation cybersecurity solutions.
References
T. T. Nguyen and V. J. Reddi, “Deep reinforcement learning for cyber security,” IEEE Transactions on Neural Networks and Learning Systems, vol. 34, no. 8, pp. 3779–3795, 2021.
G. Borrageiro, N. Firoozye, and P. Barucca, “The recurrent reinforcement learning crypto agent,” IEEE Access, vol. 10, pp. 38590–38599, 2022.
A. M. K. Adawadkar and N. Kulkarni, “Cyber-security and reinforcement learning—a brief survey,” Engineering Applications of Artificial Intelligence, vol. 114, p. 105116, 2022.
A. Z. Al-Marridi, A. Mohamed, and A. Erbad, “Reinforcement learning approaches for efficient and secure blockchain-powered smart health systems,” Computer Networks, vol. 197, p. 108279, 2021.
S. K. Mousavi, A. Ghaffari, S. Besharat, and H. Afshari, “Improving the security of Internet of Things using cryptographic algorithms: a case of smart irrigation systems,” Journal of Ambient Intelligence and Humanized Computing, vol. 12, no. 2, pp. 2033–2051, 2021.
M. A. M. Abu-Faraj and Z. A. Alqadi, “Improving the efficiency and scalability of standard methods for data cryptography,” International Journal of Computer Science & Network Security, vol. 21, no. 12spc, pp. 451–458, 2021.
H. R. Shakir, “Secure selective image encryption based on wavelet domain, 3D-chaotic map, and discrete fractional random transform,” International Journal of Intelligent Engineering & Systems, vol. 16, no. 6, 2023.
S. Zhou, H. Zhang, Y. Zhang, and H. Zhang, “Novel hyperchaotic image encryption method using machine learning-RBF,” Nonlinear Dynamics, vol. 112, no. 20, pp. 18527–18550, 2024.
M. S. Mahdi, S. N. Alsaad, and H. S. Abdullah, “An innovative deep learning model for image splice forgery detection using ResNet50 and advanced optimization techniques,” in AIP Conference Proceedings, vol. 2025, AIP Publishing, 2025.
Y. Lei et al., “New challenges in reinforcement learning: a survey of security and privacy,” Artificial Intelligence Review, vol. 56, no. 7, pp. 7195–7236, 2023.
J. Park, D. S. Kim, and H. Lim, “Privacy-preserving reinforcement learning using homomorphic encryption in cloud computing infrastructures,” IEEE Access, vol. 8, pp. 203564–203579, 2020.
S. A. Mehdi and Z. L. Ali, “Image encryption algorithm based on a novel six-dimensional hyper-chaotic system,” Al-Mustansiriyah Journal of Science, vol. 31, no. 1, pp. 54–63, 2020.
A. Hafsa et al., “Image encryption method based on improved ECC and modified AES algorithm,” Multimedia Tools and Applications, vol. 80, pp. 19769–19801, 2021.
P. Mishra et al., “Delphi: A cryptographic inference system for neural networks,” in Proceedings of the 2020 Workshop on Privacy-Preserving Machine Learning in Practice, 2020, pp. 27–30.
N. T. Ahmed, Y. M. Mohialden, and D. R. Abdulrazzaq, “A new method for self-adaptation of genetic algorithms operators,” International Journal of Civil Engineering and Technology, vol. 9, no. 11, pp. 1279–1285, 2018.
A. A. Jamal et al., “A review on security analysis of cyber-physical systems using machine learning,” Materials Today: Proceedings, vol. 80, pp. 2302–2306, 2023.
M. S. Mahdi and Z. L. Ali, “A lightweight algorithm to protect the web of things in IoT,” in International Conference on Emerging Technology Trends in Internet of Things and Computing, 2021, pp. 46–60.
Y. M. Mohialden et al., “An Internet of Things-based medication validity monitoring system,” Indonesian Journal of Electrical Engineering and Computer Science, vol. 26, no. 2, pp. 932–938, 2022.
M. S. Mahdi, S. N. Alsaad, and H. S. Abdullah, “Hybrid deep learning models for robust image splice detection: an ensemble-based strategy,” in AIP Conference Proceedings, vol. 2025, AIP Publishing, 2025.
Z. W. Salman, H. I. Mohammed, and A. M. Enad, “SMS security by elliptic curve and chaotic encryption algorithms,” Al-Mustansiriyah Journal of Science, vol. 34, no. 3, pp. 56–63, 2023.
J. Natarajan, “Cyber secure man-in-the-middle attack intrusion detection using machine learning algorithms,” in Research Anthology on Machine Learning Techniques, Methods, and Applications, IGI Global, 2022, pp. 976–1001.
E. Hato, Z. S. Abduljabbar, and Z. J. Ahmed, “Comparative Analysis for Bag of Features (BoF) Performance,” Iraqi Journal of Science, pp. 4606–4622, 2024.
A. M. Ali et al., “Image encryption using new non-linear stream cipher cryptosystem,” Al-Mustansiriyah Journal of Science, vol. 34, no. 2, pp. 103–112, 2023.
F. Thabit et al., “A novel effective, lightweight homomorphic cryptographic algorithm for data security in cloud computing,” International Journal of Intelligent Networks, vol. 3, pp. 16–30, 2022.
S. Salman et al., “A novel method for Hill cipher encryption and decryption using Gaussian integers implemented in banking systems,” Iraqi Journal for Computer Science and Mathematics, vol. 5, no. 1, pp. 277–284, 2024.
N. M. Hussien, M. A. M. Al-Obaidi, R. A. Abtan, A. H. Al-Saleh, and A. A. D. Al-Zuky, “Smart system for detecting the entry of authority people in the security facilities based on IoT using SURF recognition and Viola-Jones algorithms,” in Journal of Physics: Conference Series, vol. 1963, no. 1, p. 012075, IOP Publishing, July 2021.
Y. M. Mohialden, S. A. Alazawi, and A. M. Elewe, “An improved life cycle for building secure software,” in IOP Conference Series: Materials Science and Engineering, vol. 871, no. 1, p. 012009, IOP Publishing, June 2020.
N. A. Gunathilake, W. J. Buchanan, and R. Asif, “Next generation lightweight cryptography for smart IoT devices: implementation, challenges, and applications,” in 2019 IEEE 5th World Forum on Internet of Things (WF-IoT), April 2019, pp. 707–710.
A. K. Kalusivalingam, A. Sharma, N. Patel, and V. Singh, “Optimizing industrial systems through deep Q-networks and proximal policy optimization in reinforcement learning,” International Journal of AI and ML, vol. 1, no. 3, 2020.
X. Lu, L. Xiao, G. Niu, X. Ji, and Q. Wang, “Safe exploration in wireless security: a safe reinforcement learning algorithm with a hierarchical structure,” IEEE Transactions on Information Forensics and Security, vol. 17, pp. 732–743, 2022.
N. M. Hussien, Y. M. Mohialden, M. M. Akawee, and M. A. Mohammed, “The software requirements process for designing a microcontroller-based voice-controlled system,” Bulletin of Electrical Engineering and Informatics, vol. 12, no. 1, pp. 539–543, 2023.
M. Rana, Q. Mamun, and R. Islam, “Lightweight cryptography in IoT networks: a survey,” Future Generation Computer Systems, vol. 129, pp. 77–89, 2022.
Y. Makki et al., “An internet of things-based medication validity monitoring system,” Indones. J. Electr. Eng. Comput. Sci, vol. 26, no. 2, pp. 932–938, 2022.
J. Yang, S. He, Y. Xu, L. Chen, and J. Ren, “A trusted routing scheme using blockchain and reinforcement learning for wireless sensor networks,” Sensors, vol. 19, no. 4, p. 970, 2019, doi: 10.3390/s19040970.
G. Kim, H. Kim, Y. Heo, Y. Jeon, and J. Kim, “Generating cryptographic S-Boxes using reinforcement learning,” IEEE Access, vol. 9, pp. 83092–83104, 2021, doi: 10.1109/access.2021.3085861.
I. Meraouche, S. Dutta, S. Mohanty, I. Agudo, and K. Sakurai, “Learning multi-party adversarial encryption and its application to secret sharing,” IEEE Access, vol. 10, pp. 121329–121339, 2022, doi: 10.1109/access.2022.3223430.
A. Badr, “Instant-hybrid neural-cryptography (IHNC) based on fast machine learning,” Neural Computing and Applications, vol. 34, no. 22, pp. 19953–19972, 2022, doi: 10.1007/s00521-022-07539-0.
D. Kundi et al., “AxR-LWE: a multilevel approximate Ring-LWE co-processor for lightweight IoT applications,” IEEE Internet of Things Journal, vol. 9, no. 13, pp. 10492–10501, 2022, doi: 10.1109/jiot.2021.3122276.
J. Liu, C. Wei, S. Wen, and A. Wang, “Design and implementation of a physical security evaluation system for cryptographic chips based on machine learning,” 2023, doi: 10.1117/12.2655942.
Z. Xu and S. Cao, “Multi-source data privacy protection method based on homomorphic encryption and blockchain,” Computer Modeling in Engineering & Sciences, vol. 136, no. 1, pp. 861–881, 2023, doi: 10.32604/cmes.2023.025159.
J. Zhao, “From learning with errors (LWE) problem to CLWE problem,” Theoretical and Natural Science, vol. 26, no. 1, pp. 286–298, 2023, doi: 10.54254/2753-8818/26/20241119.
Y. Li, “Accelerate the promotion and application of commercial cryptography technology in the automotive industry,” Frontiers in Business Economics and Management, vol. 14, no. 3, pp. 249–254, 2024, doi: 10.54097/1wt51486.
