feat(system_monitor): check UDP network errors

[iwatake2222]

Description

Why this PR is needed

• UDP packet drops make Autoware unstable. Therefore, this is an important indicator for determining whether Autoware is functioning correctly
• There are several causes of UDP packet drops, but the primary reason is drops in the send/receive buffer. This is particularly true in the case of the loopback network

What this PR changs

• Monitor UDP recv buffer errors and UDP send buffer errors

Related links

None

How was this PR tested?

• total UDP rcv buf errors and total UDP snd buf errors are monitored, and they are retrieved from /proc/net/snmp
• When errors per unit time is greater than threshold (>=1), status becomes warning
• The CPU usage of system_monitor doesn't change
  • Without this PR: 13.6% / 1600%
  • With this PR: 12.6% / 1600%
  • (the CPU usage of system_monitor in 16-core ECU)

How to cause UDP buf errors

Run the following scripts

import socket
import time

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

# To cause snd buf error, enable the following line and run the following command
# $> sudo tc qdisc add dev lo root netem delay 100ms
# $> sudo tc qdisc del dev lo root    # fix setting
sock.setsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF, 4)

target_ip = "127.0.0.1"
target_port = 12345

while True:
    sock.sendto(b"a" * 1024, (target_ip, target_port))
    time.sleep(0.00001)

import socket

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 1)
sock.bind(("0.0.0.0", 12345))

while True:
    try:
        data, addr = sock.recvfrom(1024)
    except Exception as e:
        print(f"Error: {e}")

Notes for reviewers

None.

Interface changes

Topic changes

Additions and removals

None

Modifications

A new member named net_monitor: UDP Buf Errors is added into /diagnostics topic

ROS Parameter Changes

Additions and removals

system/system_monitor/config/net_monitor.param.yaml

Version	Parameter Name	Type	Default Value	Description
New	udp_buf_errors_check_duration	int	1	UDP buf errors check duration
New	udp_buf_errors_check_count	int	1	Generates warning when count of UDP buf errors during udp_buf_errors_check_duration reaches a specified value or higher

Modifications

None

Effects on system behavior

None.

[github-actions]

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

• You've checked our contribution guidelines.
• Your PR follows our pull request guidelines.
• All required CI checks pass before marking the PR ready for review.

[iwatake2222]

@ito-san @TetsuKawa
Could you review this PR?
Also, please re-add run-build-and-test-differential label to run worflows (some workflows are stuck for some reasons...)

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 121 lines in your changes missing coverage. Please review.

Project coverage is 29.77%. Comparing base (eef298b) to head (b5e2f63).
Report is 7 commits behind head on main.

Files with missing lines	Patch %	Lines
...tem/system_monitor/src/net_monitor/net_monitor.cpp	0.00%	121 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff            @@
##             main    #9538    +/-   ##
========================================
  Coverage   29.77%   29.77%            
========================================
  Files        1442     1442            
  Lines      108570   108622    +52     
  Branches    42621    42645    +24     
========================================
+ Hits        32324    32344    +20     
- Misses      72940    73100   +160     
+ Partials     3306     3178   -128     

Flag	Coverage Δ		*Carryforward flag
differential	0.00% <0.00%> (?)
total	29.81% <ø> (+0.03%)	⬆️	Carriedforward from 3730113

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[iwatake2222]

@ito-san @TetsuKawa
Thank you for reviewing it. I updated the code as you suggested.
Could you review it again please?

[TetsuKawa]

LGTM
@ito-san
I leave the rest to your judgment.

[ito-san]

LGTM @ito-san I leave the rest to your judgment.

plz let me test on my side.

[iwatake2222]

@ito-san
Friendly reminder...

[ito-san]

@ito-san Friendly reminder...

Sorry, the codebase was different between this branch and my project, so my Autoware was broken after I merged this branch.
I've updated the branch, so please wait a little longer.

[ito-san]

Verified that no errors occur during startup

[ito-san]

It seems to be working well

[ito-san]

LGTM

[ito-san]

@iwatake2222
Please fix this.
/w/autoware.universe/autoware.universe/system/system_monitor/src/net_monitor/net_monitor.cpp:814:26: error: folding type 'unsigned int' into type 'int' might result in loss of precision [bugprone-fold-init-type,-warnings-as-errors]
value_per_unit_time = std::accumulate(queue.begin(), queue_.end(), 0);
^

[iwatake2222]

@ito-san
Could you add label to run workflow?

[iwatake2222]

@ito-san
Thank you for reviewing it !!
I fixed the code and all required checks are passed (pre-commit-optional fails due to link error, but it's not related to this PR)

Could you merge this PR? I'm asking because Only those with write access to this repository can merge pull requests.