User-friendly virtual reality learning from demonstration setup
Basic Information
Students: Nico Renlund, Eetu Suominen, Sampo Vänninen, Emrehan Öz
Project manager: Eetu Suominen
Instructor: Fares Abu-Dakka
Other advisors: Oliver Struckmeier
Starting date: 11.1.2022
Completion date: 3.6.2022
Abstract
Learning from demonstration (LfD) provides a user-friendly setup for non-roboticists to teach robots how to perform new tasks. This is done by physically demonstrating the robots how to execute tasks without directly programming them. However, in some situations, there might not be a direct physical access to the robot and therefore preventing the use of learning from demonstration approach. This can be circumvented by creating a virtual reality (VR) setup for the robot and the environment where it operates.
The goal of the project was to create a virtual reality setup which gives the ability to remotely control and teach the Franka Emika Panda robot. The setup would then give the ability in the future to do research about VR-based LfD and teleoperation. This goal was achieved during the project by creating a system which consisted of virtual reality environment created in Unity3D and joint impedance controllers made on Robot Operating System (ROS). Two types of controllers were created; a trajectory controller which accomplishes a "record and replay"- functionality and a continuous controller which gives the ability to teleoperate the robot in real-time.
The virtual reality environment includes a replica of the Franka Emika Panda robot and an user interface which allows to execute different features in VR and send desired joint positions to the real robot. The interaction with the virtual robot is done by simply grabbing the robot with VR controllers. The VR setup allows the user to experiment different robot trajectories by recording and replaying them in VR before sending them to the real robot. In addition to the advantage of teleoperation, VR also offers the possibility to implement tools and visualizations that would not be possible when interacting with the robot physically. The VR setup includes for example trajectory and joint stress visualizations.
Project software repositories:
https://version.aalto.fi/gitlab/projectwork/vrlfd
https://version.aalto.fi/gitlab/projectwork/franka_control
