Skip to content
This repository has been archived by the owner on Sep 27, 2023. It is now read-only.

[Feature] Support simple (as in functional API) visualisation of appropriate (grayscale or RGB image) GPU tensors without transitioning back to host #75

Open
tvercaut opened this issue Mar 3, 2022 · 0 comments

Comments

@tvercaut
Copy link
Member

tvercaut commented Mar 3, 2022

Is your feature request related to a problem? Please describe.
It would be great to have an easy to use means of directly visualising a PyTorch GPU tensor that represents a standard grayscale of RGB image. For efficiency reasons, we would want to avoid trnsfering the data back to host.

Describe the solution you'd like
Something as simple to use as the OpenCV cv.imshow (see here and here) as shown in the tutorial:
https://docs.opencv.org/5.x/dd/d43/tutorial_py_video_display.html

Describe alternatives you've considered
Transfering to host and use numpy to display using OpenCV

Additional context
Snippets of relevant code can be found on the web for this task but it's not trivial to use:
https://gist.github.com/victor-shepardson/5b3d3087dc2b4817b9bffdb8e87a57c4

In terms of visualisation, vulkan would be preferable on the long run but opengl may be good enough for most purposes.

Also, proper GUI work is probably out of scope for MONAIStream. That said providing a simple means of displaying the content of an appropriate PyTorch GPU tensor in any suitable GUI toolkit without going back to host would be a great step forward. MONAILabel did something similar with Slicer. For MONAI Stream, in terms of GUI toolkit, it's unclear to me what makes sense. Slicer is probably not the simplest as it's a full blown app rather than a toolkit. Something like Datoviz / Dear PyGui may be interesting to look at. I guess talking to the VTK devs would also be super helpful.

Some other relevant points:

@tvercaut tvercaut changed the title [Feature] Support simple visualisation of appropriate (grayscale or RGB image) GPU tensors without transitioning back to host [Feature] Support simple (as in functional API) visualisation of appropriate (grayscale or RGB image) GPU tensors without transitioning back to host Mar 29, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant