One more question:
I trained OpenNMT model for paraphrase task. On the 3d epoch I had Perplexity~25, BLEU=22.
But unfortunately the model learned to generate identity sentences. How to fix this issue?
Is it possible to use MERT? according to this paper http://www.aclweb.org/anthology/E17-1083
MERT gives much more better results then BLEU.