Regarding 'SAME'
padding, the Convolution
documentation offers some detailed explanations (further details in those notes). Especially, when using 'SAME'
padding, the output shape is defined so:
# for `tf.layers.conv2d` with `SAME` padding:
out_height = ceil(float(in_height) / float(strides[1]))
out_width = ceil(float(in_width) / float(strides[2]))
In this case, the output shape depends only on the input shape and stride. The padding size is computed from there to fill this shape requirement (while, with 'VALID'
padding, it's the output shape which depends on the padding size)
Now for transposed convolutions... As this operation is the backward counterpart of a normal convolution (its gradient), it means that the output shape of a normal convolution corresponds to the input shape to its counterpart transposed operation. In other words, while the output shape of tf.layers.conv2d()
is divided by the stride, the output shape
of tf.layers.conv2d_transpose()
is multiplied by it:
# for `tf.layers.conv2d_transpose()` with `SAME` padding:
out_height = in_height * strides[1]
out_width = in_width * strides[2]
But once again, the padding size is calculated to obtain this output shape, not the other way around (for SAME
padding). Since the normal relation between those values (i.e. the relation you found) is:
# for `tf.layers.conv2d_transpose()` with given padding:
out_height = strides[1] * (in_height - 1) + kernel_size[0] - 2 * padding_height
out_width = strides[2] * (in_width - 1) + kernel_size[1] - 2 * padding_width
Rearranging the equations we get
padding_height = [strides[1] * (in_height - 1) + kernel_size[0] - out_height] / 2
padding_width = [[strides[2] * (in_width - 1) + kernel_size[1] - out_width] / 2
note: if e.g. 2 * padding_height
is an odd number, then padding_height_top = floor(padding_height)
; and padding_height_bottom = ceil(padding_height)
(same for resp. padding_width
, padding_width_left
and padding_width_right)
Replacing out_height
and out_width
with their expressions, and using your values (for the 1st transposed convolution):
padding = [2 * (128 - 1) + 4 - (128 * 2)] / 2 = 1
You thus have a padding of 1
added on every side of your data, in order to obtain the output dim out_dim = in_dim * stride = strides * (in_dim - 1) + kernel_size - 2 * padding = 256