Mission Planning Validation: Parametric Matrix and Their Significance to Validate Path Planning Algorithms
The mission planning validation pipeline is one of the most critical pipelines of an autonomous system. It enables the capability to test the full stack’s [Autoware in this case] end-to-end accuracy by evaluating all internal modules such as sensor integration, object detection, and localization. In this validation pipeline, there is also a need to access logs to get objects, controls, and location information.
Figure -1: High-level Architecture for Mission Planning Pipeline
To run a scenario, user provides start and goal position to Autoware stack. Autoware internally runs path planner node and plans path where vehicle will travel. Once AV vehicle starts motion, object detection node detects objects for collision avoidance and localization nodes to determine the exact position of vehicle. Based on inputs from object detection and localization node, AV vehicle updates its planned path and drives the controlling mechanism. Path planner node decides if there is possible to avoid obstacles or if it is necessary to apply brakes.
Using parametric matrices, PolyVerif Mission planning validation pipeline can test and evaluate AV vehicle’s capability in planning travelling strategy and the impact of various in-road objects. Currently PolyVerif Algorithms process logs and captures the following parameters for every scenario. These can be used to decide the success or failure of the task.
Goal Position Achieved : True / False
Deviation from Goal Position (in meters)
Number of Collisions by AV vehicle
First two parameters are able to detect scenarios where AV vehicle is not able to achieve defined goal position. Same scenario is shown in figure -2.
Figure -2: Deviation from Goal Position
3rd parameter helps in determining whether the AV vehicle is able to avoid obstacles or not.
Figure -3: Obstacle Avoidance Validation
Two more quality checks are planned for future releases:
The vehicle was on a drivable area - True/False
The traveled path was safe (or not) – True/False
These quality checks will ensure that AV vehicle always travels on a safe and drivable path. Designing scenarios with various edge conditions and validation using a mission planning validation pipeline ensures a pleasant travel experience for passengers.
Figure -4: Vehicle on Drivable Area
Currently PolyVerif framework checks the situation of collision during travel, while in future releases, we are planning to design and implement AI based algorithms to predict the possibility of collision in case of any change in motion of on-road pedestrian/vehicle as shown in below figure-5.
Figure -5: Driving Path Validation