Any updates in generating dynamic shape images?

[Meshwa428]

Like you mentioned in #44 that in the next version of onnxstream we will be able to generate dynamically shaped images

Any progress so far? Any updates?

Thanks in advance

[vitoplantamura]

hi, yes, OnnxStream (the library!) now supports dynamic shapes for inputs, via the m_support_dynamic_shapes option. This was done to be able to run TinyLlama and Mistral7B; the Stable Diffusion application has not been updated. I plan to do some tests in the future to support custom width and height in the SD application, but I can't determine a release date for this feature at the moment... :-( Vito

[Meshwa428]

No, probs I will try to implement it by my self and let you know when I'm done 😉

[vitoplantamura]

cool! Let me know about your progress here! thanks, Vito

[dylski]

Hi - Any news on custom resolutions?

I've got SD powering my 800x480 7-colour e-ink display and would desperately like to be able to render to an apsect ratio closer to that. Cropping my 512x512 images is suboptimal!

[Meshwa428]

Nope, no progress. Got busy with some personal work. Sorry to say but my job is exhaustive and I don't get time even at weekends 😞

[vitoplantamura]

sorry for the late reply... so, I took a look at the problem: I was hoping for an easy workaround, but unfortunately, in order to support arbitrary resolutions, we must re-export the UNET model and the VAE model... I will post updates here... Vito

[Meshwa428]

Can't we use gguf for this task?
Idk how to split it into small chunks like you have done with the original model, but maybe dynamically loading the small chunks of the gguf model into memory might be efficient 😅.

It might be dumb to say that but I am kinda new to llama.cpp stuff.

[vitoplantamura]

OnnxStream and llama.cpp are two completely different projects, without any dependencies. Models in OnnxStream are converted from onnx to the OnnxStream format using onnx2txt: https://github.com/vitoplantamura/OnnxStream/blob/master/onnx2txt/onnx2txt.ipynb Vito

[vitoplantamura]

Done.

Only SDXL Turbo is supported.

The --res option allows to set the resolution.

@dylski: this is an example with "--res 800x480":

Vito