Upgrade to CUDA 12

[xmfcx]

Checklist

• I've read the contribution guidelines.
• I've searched other issues and no duplicate issues were found.
• I've agreed with the maintainers that I can plan this task.

Description

In the Autoware, currently we are using a CUDA version from Ubuntu 20.04 with a workaround.

Related PR:

• build(CUDA): upgrade CUDA versions #3084

Related Links:

• https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/
• https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/sbsa/
• https://github.com/orgs/autowarefoundation/discussions/3807

We haven't updated the version for a while because NVIDIA didn't release the TensorRT for ARM until 2023 September.

Now that it is released, we can test our CUDA dependent packages and finally update.

Purpose

To use recommended, latest NVIDIA packages.

Possible approaches

We can run each CUDA dependent package on both x86_64 and ARM64 and if they work, we can update the ansible script here to install the latest CUDA, TensorRT and cuDNN:

• https://github.com/autowarefoundation/autoware/tree/b92953c3ded0bcfb0d7f56b686011888547ed43c/ansible/roles/cuda
• https://github.com/autowarefoundation/autoware/tree/b92953c3ded0bcfb0d7f56b686011888547ed43c/ansible/roles/tensorrt (contains both cudnn and tensorrt)

Definition of done

• CUDA dependent packages are listed
  • cuda_utils
  • image_projection_based_fusion
  • lidar_apollo_instance_segmentation
  • lidar_centerpoint
  • tensorrt_classifier
  • tensorrt_common
  • tensorrt_yolo
  • tensorrt_yolox
  • traffic_light_classifier
  • traffic_light_fine_detector
  • traffic_light_ssd_fine_detector
  • trtexec_vendor
• CUDA dependent packages are tested
• Ansible scripts are updated build: update to CUDA 12.3 #3956

[xmfcx]

I followed:

• https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#network-repo-installation-for-ubuntu

cd /tmp

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb

sudo dpkg -i cuda-keyring_1.1-1_all.deb

sudo apt update

sudo apt install cuda tensorrt-dev libcudnn8-dev

to install latest CUDA on my machine and I was able to compile Autoware without errors.

I will test some of the packages (starting from the ones required for AWSIM testing) and report my findings.

$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Fri_Sep__8_19:17:24_PDT_2023
Cuda compilation tools, release 12.3, V12.3.52
Build cuda_12.3.r12.3/compiler.33281558_0

Following dpkg -l | grep cuDNN and dpkg -l | grep TensorRT, some relevant package versions:

libcudnn8                                         8.9.5.29-1+cuda12.2                     amd64
libcudnn8-dev                                     8.9.5.29-1+cuda12.2                     amd64
libnvinfer-dev                                    8.6.1.6-1+cuda12.0                      amd64
libnvinfer8                                       8.6.1.6-1+cuda12.0                      amd64
tensorrt-dev                                      8.6.1.6-1+cuda12.0                      amd64

[xmfcx]

It works!

centerpoint and traffic_light_classifier works.

[xmfcx]

@mitsudome-r do you think we should also test this in ARM or can we start upgrading the Ansible scripts right away?

[xmfcx]

@oguzkaganozt have you tested Autoware on AVA developer kit?
If so, could you test it on it too?
It's not urgent, we can use CI to test if it builds and just update the ansible scripts too.

[esteve]

@xmfcx I'm targeting CUDA 12 for the Debian packages, the process for installing the previous version was a bit clunky and hopefully by the time Autoware works with CUDA 12, the old steps will be obsolete.

[oguzkaganozt]

@oguzkaganozt have you tested Autoware on AVA developer kit? If so, could you test it on it too? It's not urgent, we can use CI to test if it builds and just update the ansible scripts too.

I have tested Autoware on AVA and it works. I think this update should also work on it too. But give me some time to validate.

[esteve]

@xmfcx I've updated the Ansible scripts for CUDA in #3956 , I don't have a GPU powerful enough to test it, though.

[esteve]

I've updated the ticket with the list of packages that depend on CUDA, I'd appreciate if someone can test #3956 with a GPU on x86_64 and ARM, thanks.

[oguzkaganozt]

I have tested on both arm64(AADP-AVA) and amd64 platforms using rosbag replay simulator with GeForce RTX 3070. It is working fine, I didn't notice any difference between older versions of CUDA and TensorRT.

[xmfcx]

CUDA 12.2 downgrade discussion:

• build: update to CUDA 12.3 #3956 (comment)