Integrando o Machine Learning a um sistema de simulação baseada em agentes

Abstract

Industrial competitiveness has been increasing more and more in recent years and one of the alternatives to face this competitiveness is the use of Industry 4.0 technologies, and among them are Simulation and Big Data. Big Data involves a large generation of data that needs interpretation, which can be interpreted using machine learning algorithms from reinforcement learning, which can be in conjunction with simulation. Computer simulation is the incorporation of the real world into a virtual system, absorbing the fundamental characteristics, and one of the simulation methods is Agent-Based Simulation, in which the agent is the focus of the system. In this context, this work proposes to explain how it is possible to integrate machine learning to an agent-based simulation system. Being a tool to aid the modeler, showing two ways to carry out this implementation in AnyLogic® software. The first way will be using an external tool, the Pathmind, for that, a system will be created that generates boxes of three different colors (red, green and blue), represented by vectors, at random. The system must be able to identify the color of the box, focusing on the description of the steps to be followed to carry out the implementation using this tool. The tool efficiency test was given based on the number of adjustments that the machine is capable of performing, and the result found showed a high efficiency by this tool. Since before the implementation of machine learning, the system acted randomly, matching the colors following the statistical probability of randomness predicted for this problem, which was 12.5%, and after the implementation, the system reached a rate 100% hit. The second way will be directly in the AnyLogic® software, using Java programming language through the Q-learning reinforcement learning algorithm, which was developed in this research. For this, the same basis as the previous computational model will be used, however, for this application, boxes of five different colors will be created (red, green, blue, white and black), and will be represented through strings, in which the system seeks to hit the right box color from machine learning using the Q-learning algorithm and using the Q result matrix. And as with the form using the external tool, the emphasis will be on demonstrating all the steps to be followed to complete this implementation. The system again proved to be efficient, being able to correctly identify in all attempts. So this work was able to show two efficient ways to implement reinforcement learning in AnyLogic® software, using an external tool and in a direct way, in which the first one needs a lower level of knowledge of machine learning and programming, proving to be simpler, however, it is black box, while the second way is the opposite, requiring a high level of knowledge of machine learning and programming, but with open source.