I want to reproduce the result from pretrained model of IWSLT 14 DE-EN dataset. According to the “Training Parameters”, I use following training command:
python train.py -data $saved_data -save_model $saved_model -keep_checkpoint 1 -encoder_type brnn
Other training parameters are default. I use the specified script to obtain the training, test and valid dataset. In the preprocessing phase, I pass ‘-lower’ to preprocess.py and other parameters are default.
From the result of translate.py, I get:
BLEU = 29.21, 63.6/36.8/23.3/15.1 (BP=0.970, ratio=0.970, hyp_len=127268, ref_len=131141)
But the pretrained model gives 30.33 BLEU score.
I wonder where I set the model wrongly or are there anything I’m missing?
Thank you so much in advance.