QuantTensor creation is slow

[Giuseppe5]

Describe the bug

Creating a QT is slow, and the reason is how we handle boolean values.

In particular, when creating a QT, for simplicity, IntQuantTensor, everything is casted to a Tensor.
This casting is always skipped except for training and signed, which are normally bool and then casted to boolean torch.Tensor.

I tried dropping this, making sure that training and signed are only bool, and not boolean tensors, but it breaks export #1140
To fix this, we would need to drop QT during export (the PR used to exist and was re-adapted for something else, but it is trivial).

Reproducibility

• Can be reproduced consistently.
• Difficult to reproduce.
• Unable to reproduce.

To Reproduce

import torch
from brevitas.quant_tensor import IntQuantTensor
import time

value = torch.randn(1, 3, 5, 5)
scale = torch.randn(1, 3)
zp = torch.randn(1)
bit_width = torch.randn(1)
signed = True
training = True
start = time.time()
for _ in range(10000):
    qt = IntQuantTensor(value, scale, zp, bit_width, signed, training)
print(time.time() - start)

# 0.09406900405883789

signed = torch.tensor(signed, dtype=torch.bool)
training = torch.tensor(training, dtype=torch.bool)

start = time.time()
for _ in range(10000):
    qt = IntQuantTensor(value, scale, zp, bit_width, signed, training)
print(time.time() - start)

# 0.009476900100708008