Australian scientists crack cooperative perception tests showing smart cars can now see hidden pedestrians … even through buildings

Image Credit:
Media Release

Australian scientists have made a substantial breakthrough in showing how connected and automated vehicles (CAVs) can see people around corners and even through buildings, as well as dodge running pedestrians and errant joggers using a promising and emerging technology called cooperative perception, or collective perception messaging (CPM).

CPM allows an Intelligent Transport System (ITS) station – or, for example in this research, a so-called intelligent roadside unit (IRSU) – to share local perception information with others by using vehicle-to-X (V2X) communication technology. This emerging technology not only makes roads safer for motorists but also protects vulnerable users such as joggers, pedestrians, and cyclists.
The two-year study funded by the iMOVE Cooperative Research Centre involved scientists from University of Sydney’s Australian Centre for Field Robotics (ACFR), and engineers at Australian world-leading CAV technology company and V2X pioneers Cohda Wireless.
The study involved three experiments including a controlled live roll-out in Sydney’s CBD.

The Sydney test showed how a connected vehicle using CPM sensory information from an intelligent roadside unit fitted out with high-tech gadgetry including cameras and LiDAR laser sensors was able to provide a CAV with the capability to “see” through buildings and around corners onto side streets to detect pedestrians hidden from its view.

In another test using cutting-edge CARLA autonomous driving simulation software to recreate incredibly detailed virtual worlds, the research team demonstrated how a connected autonomous vehicle using CPM took measures to safely interact with pedestrians crossing the road at a non-designated crossing area.

In the third and final test in a controlled lab traffic environment with a 55m stretch of straight road, the research team showed how a vehicle stopped for a pedestrian running to make a pedestrian crossing “although he had not physically entered the crossing yet”.

The ground-breaking findings are being reviewed for publication in the academic journal Sensors.
The ACFR team has made significant progress in urban vehicle automation during the last few years. The team was also responsible for a number of successful demonstrations of driverless cars with only built-in sensors.

Despite significant advances in sensor technology, the perception capabilities of vehicles fitted with current perception technology is ultimately bounded in range and field of view (FOV) due to sensor’s physical constraints”, Professor Eduardo Nebot from the ACFR said.

“Hidden from view objects in urban traffic environments such as buildings, trees, and other road users impose challenges in perception.

“Unfortunately, failing to maintain sufficient awareness of other road users – vulnerable road users in particular – can cause catastrophic safety consequences.”

Dr Mao Shan, the lead researcher of this project said the study confirmed using CPM could improve awareness of vulnerable road users and safety for CAVs in various traffic scenarios.

“We demonstrate in the experiments that a connected vehicle can ‘see’ a pedestrian around corners,” he said.

“More importantly, we demonstrate how CAVs can autonomously and safely interact with walking and running pedestrians, relying only on the CPM information from the IRSU.

“This is one of the first demonstrations of urban vehicle automation using only CPM information.”

Research co-author and Cohda Wireless Chief Technical Officer Professor Paul Alexander said the use of CPM and V2X communication technology “can be a game changer for both human operated and autonomous vehicles”.

“CPM enables the smart vehicles to break the physical and practical limitations of onboard perception sensors, and in the meantime, to embrace improved perception quality and robustness along with other expected benefits from the CPM service and V2X communication,” Professor Alexander said.

“This could lower per vehicle cost to facilitate the massive deployment of CAV technology,” Professor Alexander said.

“As for manually driven Connected Vehicles, CPM also brings an attractive advantage of enabling perception capability without retrofitting the vehicle with perception sensors and the associated processing unit.”

iMOVE Managing Director Ian Christensen said:

“These successful demonstrations will help engineers and researchers better understand the safety implications of cooperative perception and its impact on current and future transportation systems.

“The researchers from the Australian Centre for Field Robotics at The University of Sydney and the engineers from Cohda Wireless have made a significant and important breakthrough.”