Denoising autoencoder 2019.

Noise the inputs and put original input as labels.

Noise is unfortunately must be cleverly highly hand-engineered.

Like crop image uncrop image, remove color channels predict colors, remove positions predict positions, rotate image reorient image, remove last word predict last word. Then take the learnt weights as feature extractor.

Now we wait for something like GANs for denoising, where generator make new denoising tasks, and discriminator must solve the new tasks