Privately Owned Vehicle Work Group Meeting - 2025/02/17 - Slot 2

[m-zain-khawaja]

Agenda

• Update on Scene3D training
• Update on EgoPath dataset curation
• EgoPath network design discussion
• Update on EgoLanes dataset curation

Discussion

@m-zain-khawaja :

The fourth and final phase of Scene3D training using both simulator and real-world data is nearly completed. In this phase, a dedicated DepthContext block has been introduced in order to allow the network to more accurately extract global image features and their relationships for background scene elements such as trees and vegetation - which were not being accurately captured in the prior network version in which the SceneContext block pre-trained for SceneSeg was utilized.

Furthermore, it was found that the MUAD dataset had some anomalous data samples with objects were very close to the virtual camera in the simulated environment, and at times, the sim-camera was clipping into the vehicle in front. These images were filtered out from the MUAD dataset using a minimum depth filter of 1m - resulting in much more stable training results for samples from this dataset.

An important addition to the network in this fourth phase is the introduction of a scale factor whereby the predicted network depth is scaled by the scale factor to achieve the final depth as follow:

metric_inverse_depth = scale_factor * predicted_network_depth_map

The scale_factor is derived from the viewing angle of the camera and is designed to help the network better predict scene depth given prior information about metric scene scale, as different FOV cameras can view the same scene resulting in drastically different images and therefore incorrect metric depth estimates.

For example, the same scene looks very different when captured from a narrow angle camera (left) vs a wide-angle camera (right)

This version of the network is on track to achieve the best performance on validation data with sharp and clear boundaries for both foreground and background objects, including vegetation and poles/infrastructure as seen in snapshots below:

EgoPath Dataset Curation Update

@TranHuuNhatHuy - awaiting update.

@m-zain-khawaja to review and merge PR by @docjag for ROADWorks dataset

Dataset curation tracking

• TuSimple - 6,394 training data samples
• CULane - 15,800 training data samples
• CurveLane - (parsing completed by @TranHuuNhatHuy, data to be uploaded soon)
• BDD100K - 24,500 training samples
• ROADWorks - 4,780 training samples
• Comma2k19 - 31,962 training data samples

Ego Path Network Design

Literature review and network design discussion topics:

EgoLanes Dataset Curation Update

• TuSimple - 6,394 training data samples
• CULane - 15,800 training data samples
• CurveLane - (to be started)
• BDD100K - (in progress by @aadarshkt)

@aadarshkt has begun dataset curation of the BDD100K dataset the EgoLanes Network. He has completed saving of the binary lane masks and is deriving a strategy to determine the left/right ego lane in certain edge case scenarios. @sarun-hub and @devang-marvania have shared guidance on JSON format for the dataset as well as the crop/scale parameters to be applied to ground truth images.

Attendees

TBD

Zoom Meeting Video Recording

Video Meeting Link
Please contact the work group lead (@m-zain-khawaja) to request access to a recording of this meeting.