Augmenting both X and Y images with Keras

Question

I know how to use the ImageDataGenerator to augment my data by translating, flipping, rotationg, shearing, etc. The question is let's say that I have both a training image and the corresponding segmentation images and I would like to augment both of these images. For example if I rotated a training image by 45 degrees then I would also like to augment the segmentation image by 45 degrees. In essence I want to perform the identical set of transforms to two data sets. Is that possible to do with ImageDataGenerator, or do I have to write all the augmentation functions from scratch? Thanks very much in advance.

question from:https://stackoverflow.com/questions/65910066/augmenting-both-x-and-y-images-with-keras

Answer 1

You can use augmentations in tf.data.Dataset.map and return the image twice. I don't know of any way to do this with ImageDataGenerator.

import tensorflow as tf
import matplotlib.pyplot as plt
from skimage import data

cats = tf.concat([data.chelsea()[None, ...] for i in range(24)], axis=0)

test = tf.data.Dataset.from_tensor_slices(cats)


def augment(image):
    image = tf.cast(x=image, dtype=tf.float32)
    image = tf.divide(x=image, y=tf.constant(255.))
    image = tf.image.random_hue(image=image, max_delta=5e-1)
    image = tf.image.random_brightness(image=image, max_delta=2e-1)
    return image, image


test = test.batch(1).map(augment)

fig = plt.figure()
plt.subplots_adjust(wspace=.1, hspace=.2)
images = next(iter(test.take(1)))
for index, image in enumerate(images):
    ax = plt.subplot(1, 2, index + 1)
    ax.set_xticks([])
    ax.set_yticks([])
    ax.imshow(tf.clip_by_value(tf.squeeze(image), clip_value_min=0, clip_value_max=1))
plt.show()