Conceptualisation of human-on-the-loop haptic teleoperation with fully autonomous self-driving vehicles in the urban environment

Abstract

The automotive industry aims to deploy commercial level-5 fully autonomous self-driving vehicles (FA-SDVs) in a diverse range of benefit-driven concepts on city roads in the years to come. In all future visions of operating networks of FA-SDVs, humans are expected to intervene with some kind of remote supervisory role. Recent advances in cyber-physical systems (CPS) within the concept of Internet of Everything (IoE) using tactile internet (TI) teleport us to teleoperate remote objects within the cyber-world. Human-on-the-loop (HOTL) haptic teleoperation with an extension of human control and sensing capability by coupling with artificial sensors and actuators with an increased sense of real-time driving in the remote vehicle can help overcome the challenging tasks when the new driver - artificial intelligence (AI) agent - encounters an unorthodox situation that can't be addressed by the autonomous capabilities. This paper analyses HOTL real-time haptic delay-sensitive teleoperation with FA-SDVs, in the aspects of human-vehicle teamwork by establishing two similar remote parallel worlds --- real-world vehicle time-varying environment and cyber-world emulation of this environment, i.e., digital twins (DTs) --- in which a human telesupervisor (HTS), as a biological agent, can be immersed within a reasonable timescale with no cybersickness enabling omnipresence and a bidirectional flow of energy and information. The experiments conducted as a proof of concept of HOTL haptic teleoperation shows promising results and the potential of benefiting from the proposed framework.