Actual acceleration on Resnet

[albert-666]

Thank you for your great work! I have a question about the latency. Could the method achieve actual acceleration on Resnet?

[changlin31]

Yes, it can achieve actual acceleration on ResNet. We plan to report detailed latency comparison on ResNet.

Please note that if practical acceleration on batch inference is wanted, the dynamic gate should be disabled, as dynamic mode only support sequential inference, i.e., one input at a time.

[lixinghe1999]

Edited.
Seems the default cfg is "mobilenet", which can't get acceleration on cuda.

Previous question:
@changlin31
Dear author,
I just tried model, however, I don't receive actual acceleration.

I test DSNet with "largest" and "smallest", 30 warm-up and 30 test. The input data is (1, 3, 224, 224)
The latency is almost the same.
My machine: GPU3060 + CUDA 11.2 + torch 1.12.
Note that I find one line in dyn_slim_ops/DSConv2d: self.channel_choice = -1, which can prohibit the inference of mode "smallest", so I temporarily comment it. Otherwise, the smallest model can only run once.
I have tested the intermediate feature shape, my "smallest" model actually gets a smaller feature shape, however, no actual acceleration is found.

Can you give me some hints? Am I misunderstanding?