Very slow response when there are many vfolder hosts

[jopemachine]

Motivation

Some customers have reported a significant slowdown in response times when there are multiple vfolder hosts.

The cause of this issue needs to be investigated, and the system's response time must be optimized.

How to reproduce

To reproduce the issue, increase the number of vfolder hosts to 30 or more.

Then, execute the ./backend vfolder list-hosts command and measure the time it takes to complete.

Suggestion

The repeated invocation of fetch_exposed_volume_fields within the loop, which repeatedly calls the storage proxy's fs-usage API, is likely to impact performance.

https://github.com/lablup/backend.ai/blob/671843a049820ea1e7eee473c563b8e46fd1325a/src/ai/backend/manager/api/vfolder.py#L826-L842

Instead of repeatedly calling fs-usage, let's optimize by creating a new API, such as fs-usages, to reduce the number of API requests.

Impact

By reducing the number of API calls and communication overhead, it should be possible to achieve shorter response times.

Since there is no need to modify the existing API, there are unlikely to be any side effects.