This article refers to the agricultural harvesting use case of the Horizon Europe Sestosenso project. The use case aims to improve the working conditions of the human workers by using wearable and collaborative robotic technologies.

The application foresees a mobile manipulator mounted on a robotic platform with enhanced awareness, working outdoors in the presence of obstacles. Such a robotic system collaborates with the human worker wearing an exoskeleton, sensorized with inertial sensors (IMUs). The goal is to achieve efficient operations in partially or totally unstructured scenarios, and to improve human working conditions in terms of physical ergonomics.

In collaboration with Inertia Technology for the inertial sensing technology and CERTH Research Institute for recognition algorithms, University of Bolzano demonstrated the sensorized exoskeleton and mobile manipulator as a collaborative harvesting operation, both in an emulated and a real vineyard. The worker wears an exoskeleton and has a set of ProMove-V IMUs attached to his upper-body (head, lower arms) and the exo-skeleton (upper arms, torso). The motion of the worker is modeled using OpenSim simulation environment. The worker is able to communicate to the mobile manipulator and the robotic platform through specific gestures. The gesture recognition is done in real-time, offering the possibility for the worker to intuitively control the actions of the robot.

The video below shows a set of gestures, as they are performed by the worker and modeled in OpenSim. Subsequently, the whole chain including gesture recognition and robot control is demonstrated, both in the lab environment and in the field.