Workspace Navigation

User-friendly virtual reality learning from demonstration setup

Basic Information

Students: Nico Renlund, Eetu Suominen, Sampo Vänninen, Emrehan Öz

Project manager: Eetu Suominen

Instructor: Fares Abu-Dakka

Other advisors: Oliver Struckmeier

Starting date: 11.1.2022

Completion date: 3.6.2022


Learning from demonstration (LfD) provides a user-friendly setup for non-roboticists to teach robots how to perform new tasks. This is done by physically demonstrating the robots how to execute tasks without directly programming them. However, in some situations, there might not be a direct physical access to the robot and therefore preventing the use of learning from demonstration approach. This can be circumvented by creating a virtual reality (VR) setup for the robot and the environment where it operates. 

The goal of the project was to create a virtual reality setup which gives the ability to remotely control and teach the Franka Emika Panda robot. The setup would then give the ability in the future to do research about VR-based LfD and teleoperation. This goal was achieved during the project by creating a system which consisted of virtual reality environment created in Unity3D and joint impedance controllers made on Robot Operating System (ROS). Two types of controllers were created; a trajectory controller which accomplishes a "record and replay"- functionality and a continuous controller which gives the ability to teleoperate the robot in real-time.

The virtual reality environment includes a replica of the Franka Emika Panda robot and an user interface which allows to execute different features in VR and send desired joint positions to the real robot. The interaction with the virtual robot is done by simply grabbing the robot with VR controllers. The VR setup allows the user to experiment different robot trajectories by recording and replaying them in VR before sending them to the real robot. In addition to the advantage of teleoperation, VR also offers the possibility to implement tools and visualizations that would not be possible when interacting with the robot physically. The VR setup includes for example trajectory and joint stress visualizations.


Project software repositories:

  • No labels
  File Modified
PDF File Final_report_E8004_2022.pdf Jun 08, 2022 by Eetu Suominen
PDF File Project_plan_E8004_2022.pdf Jun 02, 2022 by Eetu Suominen
PNG File simplified_system_diagram.png Jun 02, 2022 by Eetu Suominen
PDF File VRLFD_business_aspects.pdf Jun 02, 2022 by Eetu Suominen
Multimedia File VRLFD_demo_video.mp4 Jun 02, 2022 by Eetu Suominen
PDF File VRLFD_technical_documentation.pdf Jun 02, 2022 by Eetu Suominen
PDF File VRLFD_user_manual.pdf Jun 02, 2022 by Eetu Suominen