python - Autoencoder - Reconstruction lacks color

Question

My goal is to find unsupervised full body landmarks. For that purpose I am using an autoencoder structure to disentangle shape and appearance of full body images (deep fashion dataset). The loss function contains a reconstruction loss, which is needed as a side task.

The detection of landmarks works quite okay, as can be seen in the image below. However, the reconstructed images strongly lack in color diversity.

On this picture, you can see two input images - the original image and an augmented version of the image. Both image should be reconstructed. You can see the images in the following order:

input image - spatial representation - detected keypoints - reconstruction - original image with landmarks - input image augmented - etc..

As you can see, the reconstruction of the augmented image totally lacks color. It has no green parts at all, even though the loss is calculated with respect to the augmented image. Also the reconstruction of the unaugmented image is very "dry".

As loss function for the reconstruction loss, I am using L1-Loss around the detected landmarks. I assume, that this might be suboptimal. Does anyone have ideas, how this could be improved? Are there more suitable loss functions for reconstruction? I have also tried L2-Loss, but there was no improvement.

Thanks a lot!

question from:https://stackoverflow.com/questions/65891780/autoencoder-reconstruction-lacks-color

Answer 1

Waitting for answers