Skip to main content


Learn About Our Meetup

5000+ Members



Join our meetup, learn, connect, share, and get to know your Toronto AI community. 



Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.



Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

[R] Mask Based Unsupervised Content Transfer

Hi All, Author here –

Given two domains where one contains some additional information compared to the other, our method disentangles the common and the seperate parts and transfers the seperate information from one image to another using a mask, while not using any supervision at train time. For example, we can transfer the specific facial hair from an image of a men with a mustache to an image of a shaved person. Using a mask enables state-of-the-art quality (see example here), but also, the generated mask can be used as a semantic segmentation of the seperate part. Thus our method perform weakly-supervised semantic segmentation, using only class lables as supervision, achieving state-of-the-art performance, see example here.png).

In short, our architecture consist of two encoders, two decoders and discriminator. One encoder for encoding the common part and one to encode the separate part. The discriminator used to disentangle the encoding to the separate and common parts correctly. In training, One decoder used to decode only the common part, and the second decoder decodes only the separate part using a mask. In inference, we use only the second decoder which given the relevant encoding, adds the specific content to a new image. We also use novel regularization scheme to encourage to mask to be minimal.

Refer to the full paper for more details.

PyTorch implementation is on GitHub.

Feel free to ask questions.

submitted by /u/RonMokady
[link] [comments]