Navegação Autônoma em Ambientes Dinâmicos com Interação com Humanos baseada em Aprendizado por Reforço Profundo e Visão Computacional

Abstract

Recent research demonstrates that autonomous navigation in dynamic environments shared with humans remains an ongoing challenge. In such environments, the movement of people can create obstacles, impeding sensing and hindering the robot’s perception of its position. Furthermore, the uncertainty of human behavior can lead to unsafe situations for both the robot and the people around it. Given this scenario, the study of learning-based methods has gained prominence, along with the implementation of hybrid structures that combine different architectures and algorithms in pursuit of an efficient solution. This work proposes the integration of deep reinforcement learning techniques with computer vision methods to develop a solution capable of enabling the robot to navigate autonomously and safely in indoor environments shared with humans, considering the specific characteristics associated with the problem at hand. Thus, the navigation must take into account safety concerns, such as the distancing between the robot and the people in its vicinity. To achieve this, an innovative approach based on deep reinforcement learning has been developed, utilizing the Deep Deterministic Policy Gradient (DDPG) algorithm, combined with computer vision techniques. Comparative tests between the DDPG and Deep Q-Network (DQN) algorithms were conducted, addressing four distinct stages, each representing two different training environment scenarios and with complexity levels higher than what the robot was trained on. The DDPG algorithm demonstrated greater efficiency and stability than the DQN, with higher average success rates in all analyzed stages, showcasing excellent generalization capacity and consistently better results in different environments than the training setting. On the other hand, the DQN struggled to avoid collisions and achieved significantly lower average success rates. These findings underscore the superiority of DDPG and demonstrate the promise of the proposed solution, contributing to the advancement of research in the field. This allows for the analysis of experiments in simulated environments and testing for the subsequent deployment of robotic systems in real-world scenarios.