Why are there not bias terms in the CNN model?

[MBleeker]

No description provided.

[da03]

Sorry I didn't quite get it. What do you mean by bias terms?

[MBleeker]

Most convolutions in the literature are implemented as x^T w + b

For example in vgg16 a conv layer is defined as (in TF code):

conv = tf.nn.conv2d(bottom, filt, [1, 1, 1, 1], padding='SAME')
conv_biases = self.get_bias(name)
bias = tf.nn.bias_add(conv, conv_biases)
relu = tf.nn.relu(bias)

Is there a reason that you are not using bias terms in your conv model? I assume that bias terms are not needed when you apply batch norm after a convolution

[da03]

I see. Yes you're right, since we apply batch norm before ReLU's, anyway the features are recentered such that bias terms are not needed.