If, say, we translate from english to german then it appears that the Decoder wants the german text as in Input before it produces an output that could be used for a loss calculation and then training.

https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/models/transformer.py

*The title should read ” It appears that the Tensor2Tensor Transformer Decoder recquires the actual target as an input?”, sorry