Reproduce the result of IWSLT 14 DE-EN


I want to reproduce the result from pretrained model of IWSLT 14 DE-EN dataset. According to the “Training Parameters”, I use following training command:

python -data $saved_data -save_model $saved_model -keep_checkpoint 1 -encoder_type brnn

Other training parameters are default. I use the specified script to obtain the training, test and valid dataset. In the preprocessing phase, I pass ‘-lower’ to and other parameters are default.

From the result of, I get:
BLEU = 29.21, 63.6/36.8/23.3/15.1 (BP=0.970, ratio=0.970, hyp_len=127268, ref_len=131141)

But the pretrained model gives 30.33 BLEU score.

I wonder where I set the model wrongly or are there anything I’m missing?

Thank you so much in advance.


Sorry for the delay. I’m not sure if the default settings are still comparable. The model is most likely the same but the training duration and schedule might be different.