Reproduce the result of IWSLT 14 DE-EN

Support
1

Hello,

I want to reproduce the result from pretrained model of IWSLT 14 DE-EN dataset. According to the “Training Parameters”, I use following training command:

python train.py -data $saved_data -save_model $saved_model -keep_checkpoint 1 -encoder_type brnn

Other training parameters are default. I use the specified script to obtain the training, test and valid dataset. In the preprocessing phase, I pass ‘-lower’ to preprocess.py and other parameters are default.

From the result of translate.py, I get:
BLEU = 29.21, 63.6/36.8/23.3/15.1 (BP=0.970, ratio=0.970, hyp_len=127268, ref_len=131141)

But the pretrained model gives 30.33 BLEU score.

I wonder where I set the model wrongly or are there anything I’m missing?

Thank you so much in advance.

2

Hi,

Sorry for the delay. I’m not sure if the default settings are still comparable. The model is most likely the same but the training duration and schedule might be different.